2023-08-11T15:27:30.290 INFO:root:teuthology version: 0.0.1.dev181+g7fda959 2023-08-11T15:27:30.292 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} --name rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi -- /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066/orig.config.yaml 2023-08-11T15:27:30.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T15:27:30.433 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066 branch: wip-rishabh-2023Aug1-b4 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} email: null first_in_suite: false job_id: '7366066' kernel: flavor: default kdb: true sha1: a7fb1265323db972dd333f71b9a53e9479f62e37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-2023Aug1-b4 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Aug1-b4 sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Aug1-b4 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIeYzfE3NPS3k8lesPvSilWbv9aC2F0ABFsxvm0DjICD7uFmKzpbjIT4K0oyiq4G8JVQbSTXUg2NPF/FJWCfmoc= smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGd8+I5tYPlLin+mFo6DMZQ8wbSlDqoZJKOfAPw6zm6bEe2ff7EDCAnT3VVnHewnFpsguxwJp+ip/NUNaR9JwnI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_quota teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 7fda95956ac10132c9b74016ba832db907df09fa timestamp: 2023-08-10_20:16:46 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2224 2023-08-11T15:27:30.434 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa; will attempt to use it 2023-08-11T15:27:30.434 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks 2023-08-11T15:27:30.435 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-11T15:27:30.436 INFO:teuthology.task.internal:Checking packages... 2023-08-11T15:27:30.469 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '92684807f016ce045ac66e37fdde6f8c47857bcc' 2023-08-11T15:27:30.469 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-11T15:27:30.470 INFO:teuthology.packaging:ref: None 2023-08-11T15:27:30.470 INFO:teuthology.packaging:tag: None 2023-08-11T15:27:30.470 INFO:teuthology.packaging:branch: wip-rishabh-2023Aug1-b4 2023-08-11T15:27:30.470 INFO:teuthology.packaging:sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:27:30.471 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-2023Aug1-b4 2023-08-11T15:27:30.688 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5437.g92684807 2023-08-11T15:27:30.689 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-11T15:27:30.697 INFO:teuthology.task.internal:no buildpackages task found 2023-08-11T15:27:30.698 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-11T15:27:30.707 INFO:teuthology.task.internal:Saving configuration 2023-08-11T15:27:30.723 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-11T15:27:30.732 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T15:27:30.756 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 15:21:08.929080', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIeYzfE3NPS3k8lesPvSilWbv9aC2F0ABFsxvm0DjICD7uFmKzpbjIT4K0oyiq4G8JVQbSTXUg2NPF/FJWCfmoc='} 2023-08-11T15:27:30.780 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 15:21:08.930423', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGd8+I5tYPlLin+mFo6DMZQ8wbSlDqoZJKOfAPw6zm6bEe2ff7EDCAnT3VVnHewnFpsguxwJp+ip/NUNaR9JwnI='} 2023-08-11T15:27:30.781 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-11T15:27:30.790 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-08-11T15:27:30.790 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-08-11T15:27:30.791 INFO:teuthology.run_tasks:Running task console_log... 2023-08-11T15:27:30.874 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f73d4360f70>, signals=[15]) 2023-08-11T15:27:30.876 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-11T15:27:30.885 INFO:teuthology.task.internal:Opening connections... 2023-08-11T15:27:30.886 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:27:30.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:27:30.989 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:27:30.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:27:31.084 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-11T15:27:31.099 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2023-08-11T15:27:31.117 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2023-08-11T15:27:31.118 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2023-08-11T15:27:31.176 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2023-08-11T15:27:31.177 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2023-08-11T15:27:31.177 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2023-08-11T15:27:31.177 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2023-08-11T15:27:31.177 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2023-08-11T15:27:31.178 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2023-08-11T15:27:31.178 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-11T15:27:31.178 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2023-08-11T15:27:31.178 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2023-08-11T15:27:31.178 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-11T15:27:31.179 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2023-08-11T15:27:31.179 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T15:27:31.179 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-11T15:27:31.179 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-11T15:27:31.180 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2023-08-11T15:27:31.206 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2023-08-11T15:27:31.225 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2023-08-11T15:27:31.225 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2023-08-11T15:27:31.283 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2023-08-11T15:27:31.283 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2023-08-11T15:27:31.283 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2023-08-11T15:27:31.283 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2023-08-11T15:27:31.283 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2023-08-11T15:27:31.284 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2023-08-11T15:27:31.284 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-11T15:27:31.284 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2023-08-11T15:27:31.284 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2023-08-11T15:27:31.284 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-11T15:27:31.284 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2023-08-11T15:27:31.284 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T15:27:31.285 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-11T15:27:31.285 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-11T15:27:31.285 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2023-08-11T15:27:31.312 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-11T15:27:31.322 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-11T15:27:31.329 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-11T15:27:31.331 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2023-08-11T15:27:31.334 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2023-08-11T15:27:31.352 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-11T15:27:31.360 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-11T15:27:31.360 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2023-08-11T15:27:31.392 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2023-08-11T15:27:31.417 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-11T15:27:31.477 INFO:teuthology.run_tasks:Running task kernel... 2023-08-11T15:27:31.510 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'} 2023-08-11T15:27:31.511 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'}}, timeout 300 2023-08-11T15:27:31.511 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-11T15:27:31.513 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-11T15:27:31.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:27:31.530 DEBUG:teuthology.task.kernel:sha1 for mon.b is a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:27:31.531 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "a7fb1265323db972dd333f71b9a53e9479f62e37"... 2023-08-11T15:27:31.531 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-08-11T15:27:31.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:27:31.532 DEBUG:teuthology.task.kernel:sha1 for mon.a is a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:27:31.533 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "a7fb1265323db972dd333f71b9a53e9479f62e37"... 2023-08-11T15:27:31.533 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2023-08-11T15:27:31.588 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-239.el9.x86_64 2023-08-11T15:27:31.588 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-239.el9.x86_64 vs a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:27:31.589 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-08-11T15:27:31.589 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:27:31.593 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-239.el9.x86_64 2023-08-11T15:27:31.593 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-239.el9.x86_64 vs a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:27:31.594 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-08-11T15:27:31.594 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:27:31.745 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum upgrade -y linux-firmware 2023-08-11T15:27:31.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum upgrade -y linux-firmware 2023-08-11T15:27:33.801 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-08-11T15:27:34.041 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-08-11T15:27:35.160 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2023-08-11T15:27:35.375 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2023-08-11T15:27:37.409 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 7.8 MB 00:01 2023-08-11T15:27:40.901 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 1.4 MB/s | 7.8 MB 00:05 2023-08-11T15:27:46.285 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 18 MB 00:04 2023-08-11T15:27:50.077 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 14 kB 00:00 2023-08-11T15:27:51.228 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 18 MB/s | 18 MB 00:01 2023-08-11T15:27:55.556 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 18 MB 00:17 2023-08-11T15:27:57.585 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-08-11T15:27:57.597 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-08-11T15:27:57.597 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2023-08-11T15:27:57.598 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-08-11T15:27:57.598 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2023-08-11T15:27:57.598 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware noarch 20230726-138.el9 CentOS-BaseOS 294 M 2023-08-11T15:27:57.598 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:27:57.599 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-08-11T15:27:57.599 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-08-11T15:27:57.599 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 1 Package 2023-08-11T15:27:57.599 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:27:57.600 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 294 M 2023-08-11T15:27:57.600 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2023-08-11T15:27:58.947 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 14 kB 00:00 2023-08-11T15:28:01.658 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 18 MB 00:02 2023-08-11T15:28:05.179 INFO:teuthology.orchestra.run.smithi027.stdout:linux-firmware-20230726-138.el9.noarch.rpm 39 MB/s | 294 MB 00:07 2023-08-11T15:28:05.180 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:28:05.182 INFO:teuthology.orchestra.run.smithi027.stdout:Total 39 MB/s | 294 MB 00:07 2023-08-11T15:28:05.182 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2023-08-11T15:28:05.271 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2023-08-11T15:28:05.271 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2023-08-11T15:28:07.213 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2023-08-11T15:28:07.213 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2023-08-11T15:28:07.736 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-08-11T15:28:07.747 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-08-11T15:28:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-08-11T15:28:07.747 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-08-11T15:28:07.747 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2023-08-11T15:28:07.747 INFO:teuthology.orchestra.run.smithi119.stdout: linux-firmware noarch 20230726-138.el9 CentOS-BaseOS 294 M 2023-08-11T15:28:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:28:07.748 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-08-11T15:28:07.748 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-08-11T15:28:07.748 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 1 Package 2023-08-11T15:28:07.748 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:28:07.749 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 294 M 2023-08-11T15:28:07.749 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-08-11T15:28:13.455 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2023-08-11T15:28:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : linux-firmware-20230726-138.el9.noarch 1/2 2023-08-11T15:28:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-11T15:28:15.642 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: linux-firmware-20221214-129.el9.noarch 2/2 2023-08-11T15:28:15.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20230726-138.el9.noarch 1/2 2023-08-11T15:28:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-11T15:28:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:28:16.088 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2023-08-11T15:28:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware-20230726-138.el9.noarch 2023-08-11T15:28:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:28:16.088 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-08-11T15:28:16.200 INFO:teuthology.orchestra.run.smithi119.stdout:linux-firmware-20230726-138.el9.noarch.rpm 35 MB/s | 294 MB 00:08 2023-08-11T15:28:16.201 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:28:16.202 INFO:teuthology.orchestra.run.smithi119.stdout:Total 35 MB/s | 294 MB 00:08 2023-08-11T15:28:16.202 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-08-11T15:28:16.278 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-08-11T15:28:16.278 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-08-11T15:28:16.451 INFO:teuthology.task.kernel:Downloading kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on mon.a... 2023-08-11T15:28:16.452 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:28:16.625 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/ 2023-08-11T15:28:16.626 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/ --input-file=- 2023-08-11T15:28:16.630 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi027.front.sepia.ceph.com to complete... 2023-08-11T15:28:18.141 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-08-11T15:28:18.141 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-08-11T15:28:24.144 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-08-11T15:28:24.885 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : linux-firmware-20230726-138.el9.noarch 1/2 2023-08-11T15:28:24.973 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-11T15:28:26.245 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: linux-firmware-20221214-129.el9.noarch 2/2 2023-08-11T15:28:26.246 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : linux-firmware-20230726-138.el9.noarch 1/2 2023-08-11T15:28:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-11T15:28:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:28:26.662 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2023-08-11T15:28:26.662 INFO:teuthology.orchestra.run.smithi119.stdout: linux-firmware-20230726-138.el9.noarch 2023-08-11T15:28:26.663 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:28:26.663 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-08-11T15:28:27.015 INFO:teuthology.task.kernel:Downloading kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on mon.b... 2023-08-11T15:28:27.015 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:28:27.158 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/ 2023-08-11T15:28:27.158 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/ --input-file=- 2023-08-11T15:28:27.161 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi119.front.sepia.ceph.com to complete... 2023-08-11T15:28:29.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11 15:28:29 URL:https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm [319463472/319463472] -> "/tmp/kernel.x86_64.rpm" [1] 2023-08-11T15:28:29.234 INFO:teuthology.orchestra.run.smithi027.stderr:FINISHED --2023-08-11 15:28:29-- 2023-08-11T15:28:29.234 INFO:teuthology.orchestra.run.smithi027.stderr:Total wall clock time: 13s 2023-08-11T15:28:29.235 INFO:teuthology.orchestra.run.smithi027.stderr:Downloaded: 1 files, 305M in 12s (24.9 MB/s) 2023-08-11T15:28:29.236 INFO:teuthology.task.kernel:Installing kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on mon.a... 2023-08-11T15:28:29.237 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-08-11T15:28:31.165 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying... ######################################## 2023-08-11T15:28:31.209 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing... ######################################## 2023-08-11T15:28:31.260 INFO:teuthology.orchestra.run.smithi027.stdout:Updating / installing... 2023-08-11T15:28:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-6.5.0_rc4_ga7fb1265323d-1 ######################################## 2023-08-11T15:28:39.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11 15:28:39 URL:https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm [319463472/319463472] -> "/tmp/kernel.x86_64.rpm" [1] 2023-08-11T15:28:39.843 INFO:teuthology.orchestra.run.smithi119.stderr:FINISHED --2023-08-11 15:28:39-- 2023-08-11T15:28:39.844 INFO:teuthology.orchestra.run.smithi119.stderr:Total wall clock time: 13s 2023-08-11T15:28:39.844 INFO:teuthology.orchestra.run.smithi119.stderr:Downloaded: 1 files, 305M in 12s (25.0 MB/s) 2023-08-11T15:28:39.846 INFO:teuthology.task.kernel:Installing kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on mon.b... 2023-08-11T15:28:39.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-08-11T15:28:41.680 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying... ######################################## 2023-08-11T15:28:41.732 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing... ######################################## 2023-08-11T15:28:41.781 INFO:teuthology.orchestra.run.smithi119.stdout:Updating / installing... 2023-08-11T15:28:48.790 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-6.5.0_rc4_ga7fb1265323d-1 ######################################## 2023-08-11T15:29:19.485 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-08-11T15:29:19.485 DEBUG:teuthology.orchestra.run.smithi027:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-08-11T15:29:19.569 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:19.570 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/config-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:19.570 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/vmlinuz-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:19.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:19.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel 2023-08-11T15:29:19.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch 2023-08-11T15:29:19.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86 2023-08-11T15:29:19.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto 2023-08-11T15:29:19.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-08-11T15:29:19.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-08-11T15:29:19.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-08-11T15:29:19.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-08-11T15:29:19.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-08-11T15:29:19.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-08-11T15:29:19.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-08-11T15:29:19.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-08-11T15:29:19.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-08-11T15:29:19.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-08-11T15:29:19.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-08-11T15:29:19.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-08-11T15:29:19.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-08-11T15:29:19.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-08-11T15:29:19.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-08-11T15:29:19.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-08-11T15:29:19.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-08-11T15:29:19.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-08-11T15:29:19.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-08-11T15:29:19.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-08-11T15:29:19.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel 2023-08-11T15:29:19.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu 2023-08-11T15:29:19.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce 2023-08-11T15:29:19.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-08-11T15:29:19.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm 2023-08-11T15:29:19.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-amd.ko 2023-08-11T15:29:19.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-intel.ko 2023-08-11T15:29:19.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm.ko 2023-08-11T15:29:19.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto 2023-08-11T15:29:19.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ansi_cprng.ko 2023-08-11T15:29:19.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/anubis.ko 2023-08-11T15:29:19.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/arc4.ko 2023-08-11T15:29:19.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx 2023-08-11T15:29:19.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_memcpy.ko 2023-08-11T15:29:19.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_pq.ko 2023-08-11T15:29:19.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-08-11T15:29:19.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_tx.ko 2023-08-11T15:29:19.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_xor.ko 2023-08-11T15:29:19.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/raid6test.ko 2023-08-11T15:29:19.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authenc.ko 2023-08-11T15:29:19.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authencesn.ko 2023-08-11T15:29:19.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blake2b_generic.ko 2023-08-11T15:29:19.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_common.ko 2023-08-11T15:29:19.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_generic.ko 2023-08-11T15:29:19.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/camellia_generic.ko 2023-08-11T15:29:19.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast5_generic.ko 2023-08-11T15:29:19.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast6_generic.ko 2023-08-11T15:29:19.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast_common.ko 2023-08-11T15:29:19.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ccm.ko 2023-08-11T15:29:19.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cmac.ko 2023-08-11T15:29:19.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crc32_generic.ko 2023-08-11T15:29:19.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_engine.ko 2023-08-11T15:29:19.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_user.ko 2023-08-11T15:29:19.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cts.ko 2023-08-11T15:29:19.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/des_generic.ko 2023-08-11T15:29:19.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/dh_generic.ko 2023-08-11T15:29:19.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/echainiv.ko 2023-08-11T15:29:19.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/essiv.ko 2023-08-11T15:29:19.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/fcrypt.ko 2023-08-11T15:29:19.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/khazad.ko 2023-08-11T15:29:19.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4.ko 2023-08-11T15:29:19.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4hc.ko 2023-08-11T15:29:19.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/md4.ko 2023-08-11T15:29:19.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/michael_mic.ko 2023-08-11T15:29:19.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcbc.ko 2023-08-11T15:29:19.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcrypt.ko 2023-08-11T15:29:19.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/rmd160.ko 2023-08-11T15:29:19.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/seed.ko 2023-08-11T15:29:19.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/serpent_generic.ko 2023-08-11T15:29:19.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tcrypt.ko 2023-08-11T15:29:19.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tea.ko 2023-08-11T15:29:19.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_common.ko 2023-08-11T15:29:19.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_generic.ko 2023-08-11T15:29:19.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/vmac.ko 2023-08-11T15:29:19.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/wp512.ko 2023-08-11T15:29:19.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xcbc.ko 2023-08-11T15:29:19.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xor.ko 2023-08-11T15:29:19.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xxhash_generic.ko 2023-08-11T15:29:19.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers 2023-08-11T15:29:19.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi 2023-08-11T15:29:19.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_ipmi.ko 2023-08-11T15:29:19.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_pad.ko 2023-08-11T15:29:19.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/custom_method.ko 2023-08-11T15:29:19.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/ec_sys.ko 2023-08-11T15:29:19.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/platform_profile.ko 2023-08-11T15:29:19.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbs.ko 2023-08-11T15:29:19.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbshc.ko 2023-08-11T15:29:19.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/video.ko 2023-08-11T15:29:19.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata 2023-08-11T15:29:19.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/acard-ahci.ko 2023-08-11T15:29:19.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ahci_platform.ko 2023-08-11T15:29:19.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ata_generic.ko 2023-08-11T15:29:19.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/libahci_platform.ko 2023-08-11T15:29:19.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_acpi.ko 2023-08-11T15:29:19.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ali.ko 2023-08-11T15:29:19.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_amd.ko 2023-08-11T15:29:19.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_artop.ko 2023-08-11T15:29:19.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atiixp.ko 2023-08-11T15:29:19.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atp867x.ko 2023-08-11T15:29:19.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd640.ko 2023-08-11T15:29:19.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd64x.ko 2023-08-11T15:29:19.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cypress.ko 2023-08-11T15:29:19.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_efar.ko 2023-08-11T15:29:19.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt366.ko 2023-08-11T15:29:19.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt37x.ko 2023-08-11T15:29:19.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-08-11T15:29:19.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x3.ko 2023-08-11T15:29:19.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it8213.ko 2023-08-11T15:29:19.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it821x.ko 2023-08-11T15:29:19.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_jmicron.ko 2023-08-11T15:29:19.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_marvell.ko 2023-08-11T15:29:19.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_mpiix.ko 2023-08-11T15:29:19.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_netcell.ko 2023-08-11T15:29:19.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ninja32.ko 2023-08-11T15:29:19.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87410.ko 2023-08-11T15:29:19.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87415.ko 2023-08-11T15:29:19.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_oldpiix.ko 2023-08-11T15:29:19.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_opti.ko 2023-08-11T15:29:19.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_optidma.ko 2023-08-11T15:29:19.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pcmcia.ko 2023-08-11T15:29:19.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc2027x.ko 2023-08-11T15:29:19.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-08-11T15:29:19.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_piccolo.ko 2023-08-11T15:29:19.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_rdc.ko 2023-08-11T15:29:19.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sch.ko 2023-08-11T15:29:19.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_serverworks.ko 2023-08-11T15:29:19.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sil680.ko 2023-08-11T15:29:19.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sis.ko 2023-08-11T15:29:19.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sl82c105.ko 2023-08-11T15:29:19.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_triflex.ko 2023-08-11T15:29:19.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_via.ko 2023-08-11T15:29:19.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pdc_adma.ko 2023-08-11T15:29:19.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_inic162x.ko 2023-08-11T15:29:19.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_mv.ko 2023-08-11T15:29:19.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_nv.ko 2023-08-11T15:29:19.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_promise.ko 2023-08-11T15:29:19.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_qstor.ko 2023-08-11T15:29:19.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil.ko 2023-08-11T15:29:19.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil24.ko 2023-08-11T15:29:19.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sis.ko 2023-08-11T15:29:19.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_svw.ko 2023-08-11T15:29:19.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sx4.ko 2023-08-11T15:29:19.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_uli.ko 2023-08-11T15:29:19.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_via.ko 2023-08-11T15:29:19.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_vsc.ko 2023-08-11T15:29:19.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay 2023-08-11T15:29:19.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay/ks0108.ko 2023-08-11T15:29:19.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base 2023-08-11T15:29:19.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap 2023-08-11T15:29:19.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-08-11T15:29:19.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-08-11T15:29:19.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma 2023-08-11T15:29:19.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma/bcma.ko 2023-08-11T15:29:19.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block 2023-08-11T15:29:19.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe 2023-08-11T15:29:19.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe/aoe.ko 2023-08-11T15:29:19.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/brd.ko 2023-08-11T15:29:19.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd 2023-08-11T15:29:19.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd/drbd.ko 2023-08-11T15:29:19.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/floppy.ko 2023-08-11T15:29:19.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/loop.ko 2023-08-11T15:29:19.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx 2023-08-11T15:29:19.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-08-11T15:29:19.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/nbd.ko 2023-08-11T15:29:19.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk 2023-08-11T15:29:19.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk/null_blk.ko 2023-08-11T15:29:19.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/pktcdvd.ko 2023-08-11T15:29:19.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/rbd.ko 2023-08-11T15:29:19.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/virtio_blk.ko 2023-08-11T15:29:19.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback 2023-08-11T15:29:19.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-08-11T15:29:19.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkfront.ko 2023-08-11T15:29:19.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram 2023-08-11T15:29:19.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram/zram.ko 2023-08-11T15:29:19.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char 2023-08-11T15:29:19.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hangcheck-timer.ko 2023-08-11T15:29:19.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random 2023-08-11T15:29:19.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/amd-rng.ko 2023-08-11T15:29:19.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/intel-rng.ko 2023-08-11T15:29:19.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-08-11T15:29:19.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/via-rng.ko 2023-08-11T15:29:19.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-08-11T15:29:19.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi 2023-08-11T15:29:19.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-08-11T15:29:19.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-08-11T15:29:19.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-08-11T15:29:19.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-08-11T15:29:19.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-08-11T15:29:19.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-08-11T15:29:19.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/lp.ko 2023-08-11T15:29:19.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave 2023-08-11T15:29:19.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave/mwave.ko 2023-08-11T15:29:19.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ppdev.ko 2023-08-11T15:29:19.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tlclk.ko 2023-08-11T15:29:19.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm 2023-08-11T15:29:19.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm.ko 2023-08-11T15:29:19.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-08-11T15:29:19.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-08-11T15:29:19.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-08-11T15:29:19.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis.ko 2023-08-11T15:29:19.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-08-11T15:29:19.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/uv_mmtimer.ko 2023-08-11T15:29:19.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/virtio_console.ko 2023-08-11T15:29:19.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq 2023-08-11T15:29:19.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-08-11T15:29:19.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-08-11T15:29:19.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-08-11T15:29:19.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-08-11T15:29:19.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/powernow-k8.ko 2023-08-11T15:29:19.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-08-11T15:29:19.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto 2023-08-11T15:29:19.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp 2023-08-11T15:29:19.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-08-11T15:29:19.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp.ko 2023-08-11T15:29:19.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel 2023-08-11T15:29:19.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat 2023-08-11T15:29:19.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common 2023-08-11T15:29:19.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-08-11T15:29:19.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-08-11T15:29:19.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-08-11T15:29:19.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-aes.ko 2023-08-11T15:29:19.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-sha.ko 2023-08-11T15:29:19.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio 2023-08-11T15:29:19.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-08-11T15:29:19.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca 2023-08-11T15:29:19.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca/dca.ko 2023-08-11T15:29:19.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq 2023-08-11T15:29:19.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-08-11T15:29:19.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma 2023-08-11T15:29:19.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw 2023-08-11T15:29:19.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw/dw_dmac.ko 2023-08-11T15:29:19.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat 2023-08-11T15:29:19.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat/ioatdma.ko 2023-08-11T15:29:19.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac 2023-08-11T15:29:19.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/amd64_edac.ko 2023-08-11T15:29:19.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/e752x_edac.ko 2023-08-11T15:29:19.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/edac_mce_amd.ko 2023-08-11T15:29:19.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3000_edac.ko 2023-08-11T15:29:19.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3200_edac.ko 2023-08-11T15:29:19.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5100_edac.ko 2023-08-11T15:29:19.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5400_edac.ko 2023-08-11T15:29:19.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7300_edac.ko 2023-08-11T15:29:19.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7core_edac.ko 2023-08-11T15:29:19.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i82975x_edac.ko 2023-08-11T15:29:19.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/ie31200_edac.ko 2023-08-11T15:29:19.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/sb_edac.ko 2023-08-11T15:29:19.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/x38_edac.ko 2023-08-11T15:29:19.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware 2023-08-11T15:29:19.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/edd.ko 2023-08-11T15:29:19.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/iscsi_ibft.ko 2023-08-11T15:29:19.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio 2023-08-11T15:29:19.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-ich.ko 2023-08-11T15:29:19.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-viperboard.ko 2023-08-11T15:29:19.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid 2023-08-11T15:29:19.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-appleir.ko 2023-08-11T15:29:19.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-aureal.ko 2023-08-11T15:29:19.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-axff.ko 2023-08-11T15:29:19.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-dr.ko 2023-08-11T15:29:19.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elecom.ko 2023-08-11T15:29:19.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elo.ko 2023-08-11T15:29:19.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-emsff.ko 2023-08-11T15:29:19.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gaff.ko 2023-08-11T15:29:19.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gt683r.ko 2023-08-11T15:29:19.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gyration.ko 2023-08-11T15:29:19.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-08-11T15:29:19.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-08-11T15:29:19.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtekff.ko 2023-08-11T15:29:19.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-hyperv.ko 2023-08-11T15:29:19.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-icade.ko 2023-08-11T15:29:19.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-keytouch.ko 2023-08-11T15:29:19.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-kye.ko 2023-08-11T15:29:19.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lcpower.ko 2023-08-11T15:29:19.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-led.ko 2023-08-11T15:29:19.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lenovo.ko 2023-08-11T15:29:19.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-dj.ko 2023-08-11T15:29:19.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-08-11T15:29:19.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-multitouch.ko 2023-08-11T15:29:19.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-ortek.ko 2023-08-11T15:29:19.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-penmount.ko 2023-08-11T15:29:19.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-petalynx.ko 2023-08-11T15:29:19.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-picolcd.ko 2023-08-11T15:29:19.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-pl.ko 2023-08-11T15:29:19.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-plantronics.ko 2023-08-11T15:29:19.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-primax.ko 2023-08-11T15:29:19.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-rmi.ko 2023-08-11T15:29:19.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-08-11T15:29:19.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-common.ko 2023-08-11T15:29:19.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-isku.ko 2023-08-11T15:29:19.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kone.ko 2023-08-11T15:29:19.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-08-11T15:29:19.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-08-11T15:29:19.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-08-11T15:29:19.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-lua.ko 2023-08-11T15:29:19.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-08-11T15:29:19.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-08-11T15:29:19.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-savu.ko 2023-08-11T15:29:19.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat.ko 2023-08-11T15:29:19.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-saitek.ko 2023-08-11T15:29:19.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-samsung.ko 2023-08-11T15:29:19.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sensor-hub.ko 2023-08-11T15:29:19.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sjoy.ko 2023-08-11T15:29:19.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sony.ko 2023-08-11T15:29:19.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-speedlink.ko 2023-08-11T15:29:19.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-steelseries.ko 2023-08-11T15:29:19.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sunplus.ko 2023-08-11T15:29:19.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-thrustmaster.ko 2023-08-11T15:29:19.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tivo.ko 2023-08-11T15:29:19.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tmff.ko 2023-08-11T15:29:19.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-topseed.ko 2023-08-11T15:29:19.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-twinhan.ko 2023-08-11T15:29:19.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-uclogic.ko 2023-08-11T15:29:19.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-waltop.ko 2023-08-11T15:29:19.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-wiimote.ko 2023-08-11T15:29:19.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-xinmo.ko 2023-08-11T15:29:19.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zpff.ko 2023-08-11T15:29:19.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zydacron.ko 2023-08-11T15:29:19.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/uhid.ko 2023-08-11T15:29:19.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/wacom.ko 2023-08-11T15:29:19.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv 2023-08-11T15:29:19.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_balloon.ko 2023-08-11T15:29:19.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_utils.ko 2023-08-11T15:29:19.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_vmbus.ko 2023-08-11T15:29:19.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon 2023-08-11T15:29:19.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru.ko 2023-08-11T15:29:19.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru3.ko 2023-08-11T15:29:19.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-08-11T15:29:19.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7414.ko 2023-08-11T15:29:19.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7418.ko 2023-08-11T15:29:19.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adc128d818.ko 2023-08-11T15:29:19.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1025.ko 2023-08-11T15:29:19.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1026.ko 2023-08-11T15:29:19.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1029.ko 2023-08-11T15:29:19.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1031.ko 2023-08-11T15:29:19.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm9240.ko 2023-08-11T15:29:19.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ads7828.ko 2023-08-11T15:29:19.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7410.ko 2023-08-11T15:29:19.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7411.ko 2023-08-11T15:29:19.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7462.ko 2023-08-11T15:29:19.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7470.ko 2023-08-11T15:29:19.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7475.ko 2023-08-11T15:29:19.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7x10.ko 2023-08-11T15:29:19.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/amc6821.ko 2023-08-11T15:29:19.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/applesmc.ko 2023-08-11T15:29:19.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asb100.ko 2023-08-11T15:29:19.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asc7621.ko 2023-08-11T15:29:19.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asus_atk0110.ko 2023-08-11T15:29:19.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/atxp1.ko 2023-08-11T15:29:19.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/coretemp.ko 2023-08-11T15:29:19.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-08-11T15:29:19.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dme1737.ko 2023-08-11T15:29:19.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds1621.ko 2023-08-11T15:29:19.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds620.ko 2023-08-11T15:29:19.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc1403.ko 2023-08-11T15:29:19.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc6w201.ko 2023-08-11T15:29:19.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71805f.ko 2023-08-11T15:29:19.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71882fg.ko 2023-08-11T15:29:19.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f75375s.ko 2023-08-11T15:29:19.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fam15h_power.ko 2023-08-11T15:29:19.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fschmd.ko 2023-08-11T15:29:19.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g760a.ko 2023-08-11T15:29:19.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g762.ko 2023-08-11T15:29:19.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl518sm.ko 2023-08-11T15:29:19.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl520sm.ko 2023-08-11T15:29:19.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/hwmon-vid.ko 2023-08-11T15:29:19.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5500_temp.ko 2023-08-11T15:29:19.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5k_amb.ko 2023-08-11T15:29:19.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmaem.ko 2023-08-11T15:29:19.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmpex.ko 2023-08-11T15:29:19.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina209.ko 2023-08-11T15:29:19.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina2xx.ko 2023-08-11T15:29:19.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/it87.ko 2023-08-11T15:29:19.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k10temp.ko 2023-08-11T15:29:19.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k8temp.ko 2023-08-11T15:29:19.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lineage-pem.ko 2023-08-11T15:29:19.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm63.ko 2023-08-11T15:29:19.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm73.ko 2023-08-11T15:29:19.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm75.ko 2023-08-11T15:29:19.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm77.ko 2023-08-11T15:29:19.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm78.ko 2023-08-11T15:29:19.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm80.ko 2023-08-11T15:29:19.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm83.ko 2023-08-11T15:29:19.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm85.ko 2023-08-11T15:29:19.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm87.ko 2023-08-11T15:29:19.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm90.ko 2023-08-11T15:29:19.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm92.ko 2023-08-11T15:29:19.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm93.ko 2023-08-11T15:29:19.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95234.ko 2023-08-11T15:29:19.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95241.ko 2023-08-11T15:29:19.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95245.ko 2023-08-11T15:29:19.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc2945.ko 2023-08-11T15:29:19.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4151.ko 2023-08-11T15:29:19.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4215.ko 2023-08-11T15:29:19.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4222.ko 2023-08-11T15:29:19.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4245.ko 2023-08-11T15:29:19.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4260.ko 2023-08-11T15:29:19.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4261.ko 2023-08-11T15:29:19.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max16065.ko 2023-08-11T15:29:19.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1619.ko 2023-08-11T15:29:19.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1668.ko 2023-08-11T15:29:19.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max197.ko 2023-08-11T15:29:19.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6639.ko 2023-08-11T15:29:19.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6650.ko 2023-08-11T15:29:19.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6697.ko 2023-08-11T15:29:19.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/mcp3021.ko 2023-08-11T15:29:19.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6683.ko 2023-08-11T15:29:19.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775-core.ko 2023-08-11T15:29:19.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775.ko 2023-08-11T15:29:19.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct7802.ko 2023-08-11T15:29:19.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87360.ko 2023-08-11T15:29:19.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87427.ko 2023-08-11T15:29:19.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pcf8591.ko 2023-08-11T15:29:19.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus 2023-08-11T15:29:19.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-08-11T15:29:19.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-08-11T15:29:19.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-08-11T15:29:19.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-08-11T15:29:19.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-08-11T15:29:19.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-08-11T15:29:19.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-08-11T15:29:19.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-08-11T15:29:19.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-08-11T15:29:19.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-08-11T15:29:19.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-08-11T15:29:19.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-08-11T15:29:19.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/powr1220.ko 2023-08-11T15:29:19.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5627.ko 2023-08-11T15:29:19.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5636.ko 2023-08-11T15:29:19.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch56xx-common.ko 2023-08-11T15:29:19.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht15.ko 2023-08-11T15:29:19.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht21.ko 2023-08-11T15:29:19.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/shtc1.ko 2023-08-11T15:29:19.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sis5595.ko 2023-08-11T15:29:19.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47b397.ko 2023-08-11T15:29:19.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m1.ko 2023-08-11T15:29:19.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m192.ko 2023-08-11T15:29:19.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/thmc50.ko 2023-08-11T15:29:19.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp102.ko 2023-08-11T15:29:19.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp103.ko 2023-08-11T15:29:19.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp401.ko 2023-08-11T15:29:19.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp421.ko 2023-08-11T15:29:19.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via-cputemp.ko 2023-08-11T15:29:19.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via686a.ko 2023-08-11T15:29:19.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt1211.ko 2023-08-11T15:29:19.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt8231.ko 2023-08-11T15:29:19.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627ehf.ko 2023-08-11T15:29:19.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627hf.ko 2023-08-11T15:29:19.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83781d.ko 2023-08-11T15:29:19.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83791d.ko 2023-08-11T15:29:19.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83792d.ko 2023-08-11T15:29:19.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83793.ko 2023-08-11T15:29:19.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83795.ko 2023-08-11T15:29:19.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l785ts.ko 2023-08-11T15:29:19.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l786ng.ko 2023-08-11T15:29:19.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c 2023-08-11T15:29:19.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos 2023-08-11T15:29:19.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-08-11T15:29:19.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-08-11T15:29:19.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses 2023-08-11T15:29:19.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-08-11T15:29:19.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-08-11T15:29:19.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-08-11T15:29:19.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-08-11T15:29:19.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-08-11T15:29:19.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-08-11T15:29:19.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-08-11T15:29:19.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-08-11T15:29:19.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-08-11T15:29:19.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-08-11T15:29:19.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-08-11T15:29:19.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-08-11T15:29:19.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-08-11T15:29:19.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-08-11T15:29:19.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-08-11T15:29:19.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-08-11T15:29:19.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-08-11T15:29:19.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-08-11T15:29:19.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-08-11T15:29:19.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-08-11T15:29:19.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-via.ko 2023-08-11T15:29:19.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-08-11T15:29:19.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-08-11T15:29:19.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-dev.ko 2023-08-11T15:29:19.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-mux.ko 2023-08-11T15:29:19.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-08-11T15:29:19.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-smbus.ko 2023-08-11T15:29:19.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-stub.ko 2023-08-11T15:29:19.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband 2023-08-11T15:29:19.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core 2023-08-11T15:29:19.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_cm.ko 2023-08-11T15:29:19.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_core.ko 2023-08-11T15:29:19.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_umad.ko 2023-08-11T15:29:19.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-08-11T15:29:19.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/iw_cm.ko 2023-08-11T15:29:19.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-08-11T15:29:19.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-08-11T15:29:19.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw 2023-08-11T15:29:19.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4 2023-08-11T15:29:19.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-08-11T15:29:19.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4 2023-08-11T15:29:19.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-08-11T15:29:19.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5 2023-08-11T15:29:19.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-08-11T15:29:19.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca 2023-08-11T15:29:19.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-08-11T15:29:19.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp 2023-08-11T15:29:19.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib 2023-08-11T15:29:19.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-08-11T15:29:19.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser 2023-08-11T15:29:19.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-08-11T15:29:19.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert 2023-08-11T15:29:19.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-08-11T15:29:19.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp 2023-08-11T15:29:19.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-08-11T15:29:19.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt 2023-08-11T15:29:19.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-08-11T15:29:19.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input 2023-08-11T15:29:19.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport 2023-08-11T15:29:19.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-08-11T15:29:19.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/fm801-gp.ko 2023-08-11T15:29:19.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/gameport.ko 2023-08-11T15:29:19.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/lightning.ko 2023-08-11T15:29:19.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/ns558.ko 2023-08-11T15:29:19.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/joydev.ko 2023-08-11T15:29:19.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard 2023-08-11T15:29:19.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-08-11T15:29:19.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse 2023-08-11T15:29:19.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/appletouch.ko 2023-08-11T15:29:19.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/bcm5974.ko 2023-08-11T15:29:19.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/cyapatp.ko 2023-08-11T15:29:19.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/elan_i2c.ko 2023-08-11T15:29:19.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/sermouse.ko 2023-08-11T15:29:19.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-08-11T15:29:19.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-08-11T15:29:19.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-08-11T15:29:19.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4 2023-08-11T15:29:19.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4/rmi_core.ko 2023-08-11T15:29:19.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio 2023-08-11T15:29:19.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/altera_ps2.ko 2023-08-11T15:29:19.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/arc_ps2.ko 2023-08-11T15:29:19.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-08-11T15:29:19.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/serio_raw.ko 2023-08-11T15:29:19.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/sparse-keymap.ko 2023-08-11T15:29:19.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu 2023-08-11T15:29:19.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd 2023-08-11T15:29:19.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-08-11T15:29:19.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds 2023-08-11T15:29:19.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-blinkm.ko 2023-08-11T15:29:19.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lm3530.ko 2023-08-11T15:29:19.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lp3944.ko 2023-08-11T15:29:19.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-ss4200.ko 2023-08-11T15:29:19.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger 2023-08-11T15:29:19.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-08-11T15:29:19.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-08-11T15:29:19.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-08-11T15:29:19.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-08-11T15:29:19.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-08-11T15:29:19.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-08-11T15:29:19.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-08-11T15:29:19.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-08-11T15:29:19.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md 2023-08-11T15:29:19.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache 2023-08-11T15:29:19.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache/bcache.ko 2023-08-11T15:29:19.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-bio-prison.ko 2023-08-11T15:29:19.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache-smq.ko 2023-08-11T15:29:19.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache.ko 2023-08-11T15:29:19.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-crypt.ko 2023-08-11T15:29:19.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-delay.ko 2023-08-11T15:29:19.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-flakey.ko 2023-08-11T15:29:19.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-log-userspace.ko 2023-08-11T15:29:19.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-multipath.ko 2023-08-11T15:29:19.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-queue-length.ko 2023-08-11T15:29:19.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-raid.ko 2023-08-11T15:29:19.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-round-robin.ko 2023-08-11T15:29:19.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-service-time.ko 2023-08-11T15:29:19.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-switch.ko 2023-08-11T15:29:19.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-thin-pool.ko 2023-08-11T15:29:19.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-verity.ko 2023-08-11T15:29:19.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/faulty.ko 2023-08-11T15:29:19.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/linear.ko 2023-08-11T15:29:19.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/multipath.ko 2023-08-11T15:29:19.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data 2023-08-11T15:29:19.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-08-11T15:29:19.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid0.ko 2023-08-11T15:29:19.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid1.ko 2023-08-11T15:29:19.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid10.ko 2023-08-11T15:29:19.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid456.ko 2023-08-11T15:29:19.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message 2023-08-11T15:29:19.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion 2023-08-11T15:29:19.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptbase.ko 2023-08-11T15:29:19.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptctl.ko 2023-08-11T15:29:19.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptfc.ko 2023-08-11T15:29:19.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptlan.ko 2023-08-11T15:29:19.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptsas.ko 2023-08-11T15:29:19.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptscsih.ko 2023-08-11T15:29:19.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptspi.ko 2023-08-11T15:29:19.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd 2023-08-11T15:29:19.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_ich.ko 2023-08-11T15:29:19.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_sch.ko 2023-08-11T15:29:19.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/mfd-core.ko 2023-08-11T15:29:19.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/sm501.ko 2023-08-11T15:29:19.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/viperboard.ko 2023-08-11T15:29:19.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/vx855.ko 2023-08-11T15:29:19.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/wl1273-core.ko 2023-08-11T15:29:19.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc 2023-08-11T15:29:19.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl 2023-08-11T15:29:19.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-08-11T15:29:19.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds9802als.ko 2023-08-11T15:29:19.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds990x.ko 2023-08-11T15:29:19.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/bh1770glc.ko 2023-08-11T15:29:19.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710 2023-08-11T15:29:19.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710/cb710.ko 2023-08-11T15:29:19.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom 2023-08-11T15:29:19.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/at24.ko 2023-08-11T15:29:19.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom.ko 2023-08-11T15:29:19.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-08-11T15:29:19.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/max6875.ko 2023-08-11T15:29:19.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/enclosure.ko 2023-08-11T15:29:19.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/hpilo.ko 2023-08-11T15:29:19.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29003.ko 2023-08-11T15:29:19.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29020.ko 2023-08-11T15:29:19.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d 2023-08-11T15:29:19.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-08-11T15:29:19.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-08-11T15:29:19.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei 2023-08-11T15:29:19.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-me.ko 2023-08-11T15:29:19.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-txe.ko 2023-08-11T15:29:19.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei.ko 2023-08-11T15:29:19.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru 2023-08-11T15:29:19.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru/gru.ko 2023-08-11T15:29:19.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp 2023-08-11T15:29:19.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xp.ko 2023-08-11T15:29:19.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-08-11T15:29:19.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-08-11T15:29:19.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_7xx1.ko 2023-08-11T15:29:19.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_core.ko 2023-08-11T15:29:19.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tsl2550.ko 2023-08-11T15:29:19.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_balloon.ko 2023-08-11T15:29:19.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci 2023-08-11T15:29:19.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-08-11T15:29:19.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net 2023-08-11T15:29:19.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding 2023-08-11T15:29:19.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding/bonding.ko 2023-08-11T15:29:19.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa 2023-08-11T15:29:19.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53 2023-08-11T15:29:19.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-08-11T15:29:19.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-08-11T15:29:19.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6060.ko 2023-08-11T15:29:19.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx 2023-08-11T15:29:19.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-08-11T15:29:19.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dummy.ko 2023-08-11T15:29:19.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/eql.ko 2023-08-11T15:29:19.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet 2023-08-11T15:29:19.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com 2023-08-11T15:29:19.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-08-11T15:29:19.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-08-11T15:29:19.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-08-11T15:29:19.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-08-11T15:29:19.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390 2023-08-11T15:29:19.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/8390.ko 2023-08-11T15:29:19.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-08-11T15:29:19.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-08-11T15:29:19.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-08-11T15:29:19.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec 2023-08-11T15:29:19.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-08-11T15:29:19.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere 2023-08-11T15:29:19.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-08-11T15:29:19.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon 2023-08-11T15:29:19.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-08-11T15:29:19.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera 2023-08-11T15:29:19.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-08-11T15:29:19.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd 2023-08-11T15:29:19.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-08-11T15:29:19.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-08-11T15:29:19.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-08-11T15:29:19.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros 2023-08-11T15:29:19.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx 2023-08-11T15:29:19.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-08-11T15:29:19.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c 2023-08-11T15:29:19.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-08-11T15:29:19.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e 2023-08-11T15:29:19.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-08-11T15:29:19.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx 2023-08-11T15:29:19.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-08-11T15:29:19.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-08-11T15:29:19.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom 2023-08-11T15:29:19.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-08-11T15:29:19.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-08-11T15:29:19.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-08-11T15:29:19.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-08-11T15:29:19.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-08-11T15:29:19.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet 2023-08-11T15:29:19.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-08-11T15:29:19.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-08-11T15:29:19.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade 2023-08-11T15:29:19.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna 2023-08-11T15:29:19.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-08-11T15:29:19.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio 2023-08-11T15:29:19.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-08-11T15:29:19.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-08-11T15:29:19.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-08-11T15:29:19.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-08-11T15:29:19.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-08-11T15:29:19.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-08-11T15:29:19.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-08-11T15:29:19.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-08-11T15:29:19.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-08-11T15:29:19.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-08-11T15:29:19.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco 2023-08-11T15:29:19.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic 2023-08-11T15:29:19.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-08-11T15:29:19.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec 2023-08-11T15:29:19.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip 2023-08-11T15:29:19.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-08-11T15:29:19.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-08-11T15:29:19.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-08-11T15:29:19.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-08-11T15:29:19.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-08-11T15:29:19.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-08-11T15:29:19.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink 2023-08-11T15:29:19.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-08-11T15:29:19.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-08-11T15:29:19.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dnet.ko 2023-08-11T15:29:19.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex 2023-08-11T15:29:19.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet 2023-08-11T15:29:19.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-08-11T15:29:19.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ethoc.ko 2023-08-11T15:29:19.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/fealnx.ko 2023-08-11T15:29:19.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel 2023-08-11T15:29:19.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e100.ko 2023-08-11T15:29:19.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000 2023-08-11T15:29:19.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-08-11T15:29:19.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e 2023-08-11T15:29:19.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-08-11T15:29:19.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k 2023-08-11T15:29:19.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-08-11T15:29:19.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e 2023-08-11T15:29:19.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-08-11T15:29:19.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf 2023-08-11T15:29:19.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-08-11T15:29:19.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb 2023-08-11T15:29:19.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-08-11T15:29:19.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf 2023-08-11T15:29:19.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-08-11T15:29:19.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe 2023-08-11T15:29:19.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-08-11T15:29:19.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-08-11T15:29:19.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-08-11T15:29:19.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/jme.ko 2023-08-11T15:29:19.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell 2023-08-11T15:29:19.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-08-11T15:29:19.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-08-11T15:29:19.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-08-11T15:29:19.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox 2023-08-11T15:29:19.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-08-11T15:29:19.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-08-11T15:29:19.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-08-11T15:29:19.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-08-11T15:29:19.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-08-11T15:29:19.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-08-11T15:29:19.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel 2023-08-11T15:29:19.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-08-11T15:29:19.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom 2023-08-11T15:29:19.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-08-11T15:29:19.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-08-11T15:29:19.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi 2023-08-11T15:29:19.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-08-11T15:29:19.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-08-11T15:29:19.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion 2023-08-11T15:29:19.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-08-11T15:29:19.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia 2023-08-11T15:29:19.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-08-11T15:29:19.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines 2023-08-11T15:29:19.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-08-11T15:29:19.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-08-11T15:29:19.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic 2023-08-11T15:29:19.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen 2023-08-11T15:29:19.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-08-11T15:29:19.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-08-11T15:29:19.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-08-11T15:29:19.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-08-11T15:29:19.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc 2023-08-11T15:29:19.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-08-11T15:29:19.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek 2023-08-11T15:29:19.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-08-11T15:29:19.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-08-11T15:29:19.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-08-11T15:29:19.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-08-11T15:29:19.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker 2023-08-11T15:29:19.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-08-11T15:29:19.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc 2023-08-11T15:29:19.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-08-11T15:29:19.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan 2023-08-11T15:29:19.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-08-11T15:29:19.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis 2023-08-11T15:29:19.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-08-11T15:29:19.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-08-11T15:29:19.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc 2023-08-11T15:29:19.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-08-11T15:29:19.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-08-11T15:29:19.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-08-11T15:29:19.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-08-11T15:29:19.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro 2023-08-11T15:29:19.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-08-11T15:29:19.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-08-11T15:29:19.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-08-11T15:29:19.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun 2023-08-11T15:29:19.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-08-11T15:29:19.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/niu.ko 2023-08-11T15:29:19.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-08-11T15:29:19.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-08-11T15:29:19.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti 2023-08-11T15:29:19.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-08-11T15:29:19.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti 2023-08-11T15:29:19.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-08-11T15:29:19.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via 2023-08-11T15:29:19.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-08-11T15:29:19.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-08-11T15:29:19.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet 2023-08-11T15:29:19.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-08-11T15:29:19.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-08-11T15:29:19.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom 2023-08-11T15:29:19.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-08-11T15:29:19.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/geneve.ko 2023-08-11T15:29:19.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv 2023-08-11T15:29:19.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-08-11T15:29:19.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154 2023-08-11T15:29:19.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154/fakelb.ko 2023-08-11T15:29:19.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ifb.ko 2023-08-11T15:29:19.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan 2023-08-11T15:29:19.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-08-11T15:29:19.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvlan.ko 2023-08-11T15:29:19.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvtap.ko 2023-08-11T15:29:19.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio 2023-08-11T15:29:19.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio.ko 2023-08-11T15:29:19.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-08-11T15:29:19.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-08-11T15:29:19.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-08-11T15:29:19.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mii.ko 2023-08-11T15:29:19.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/net_failover.ko 2023-08-11T15:29:19.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/netconsole.ko 2023-08-11T15:29:19.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/nlmon.ko 2023-08-11T15:29:19.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs 2023-08-11T15:29:19.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-08-11T15:29:19.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-08-11T15:29:19.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy 2023-08-11T15:29:19.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/amd.ko 2023-08-11T15:29:19.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-08-11T15:29:19.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-08-11T15:29:19.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm7xxx.ko 2023-08-11T15:29:19.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm87xx.ko 2023-08-11T15:29:19.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/broadcom.ko 2023-08-11T15:29:19.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/cicada.ko 2023-08-11T15:29:19.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/davicom.ko 2023-08-11T15:29:19.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/dp83640.ko 2023-08-11T15:29:19.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/et1011c.ko 2023-08-11T15:29:19.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/icplus.ko 2023-08-11T15:29:19.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/lxt.ko 2023-08-11T15:29:19.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/marvell.ko 2023-08-11T15:29:19.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/micrel.ko 2023-08-11T15:29:19.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/national.ko 2023-08-11T15:29:19.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/phylink.ko 2023-08-11T15:29:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/qsemi.ko 2023-08-11T15:29:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/realtek.ko 2023-08-11T15:29:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/smsc.ko 2023-08-11T15:29:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/ste10Xp.ko 2023-08-11T15:29:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/vitesse.ko 2023-08-11T15:29:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp 2023-08-11T15:29:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/bsd_comp.ko 2023-08-11T15:29:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_async.ko 2023-08-11T15:29:19.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-08-11T15:29:19.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_generic.ko 2023-08-11T15:29:19.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-08-11T15:29:19.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-08-11T15:29:19.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppoe.ko 2023-08-11T15:29:19.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppox.ko 2023-08-11T15:29:19.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pptp.ko 2023-08-11T15:29:19.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip 2023-08-11T15:29:19.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slhc.ko 2023-08-11T15:29:19.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slip.ko 2023-08-11T15:29:19.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/sungem_phy.ko 2023-08-11T15:29:19.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tap.ko 2023-08-11T15:29:19.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team 2023-08-11T15:29:19.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team.ko 2023-08-11T15:29:19.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-08-11T15:29:19.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-08-11T15:29:19.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-08-11T15:29:19.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_random.ko 2023-08-11T15:29:19.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-08-11T15:29:19.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tun.ko 2023-08-11T15:29:19.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/veth.ko 2023-08-11T15:29:19.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/virtio_net.ko 2023-08-11T15:29:19.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3 2023-08-11T15:29:19.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-08-11T15:29:19.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan 2023-08-11T15:29:19.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan/vxlan.ko 2023-08-11T15:29:19.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback 2023-08-11T15:29:19.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-08-11T15:29:19.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netfront.ko 2023-08-11T15:29:19.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb 2023-08-11T15:29:19.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb/ntb.ko 2023-08-11T15:29:19.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme 2023-08-11T15:29:19.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host 2023-08-11T15:29:19.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme-core.ko 2023-08-11T15:29:19.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme.ko 2023-08-11T15:29:19.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport 2023-08-11T15:29:19.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport.ko 2023-08-11T15:29:19.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_cs.ko 2023-08-11T15:29:19.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_pc.ko 2023-08-11T15:29:19.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_serial.ko 2023-08-11T15:29:19.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci 2023-08-11T15:29:19.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug 2023-08-11T15:29:19.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-08-11T15:29:19.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie 2023-08-11T15:29:19.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie/aer_inject.ko 2023-08-11T15:29:19.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/xen-pcifront.ko 2023-08-11T15:29:19.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia 2023-08-11T15:29:19.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/i82092.ko 2023-08-11T15:29:19.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/pd6729.ko 2023-08-11T15:29:19.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/yenta_socket.ko 2023-08-11T15:29:19.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl 2023-08-11T15:29:19.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel 2023-08-11T15:29:19.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-08-11T15:29:19.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform 2023-08-11T15:29:19.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86 2023-08-11T15:29:19.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acer-wmi.ko 2023-08-11T15:29:19.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acerhdf.ko 2023-08-11T15:29:19.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/apple-gmux.ko 2023-08-11T15:29:19.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-laptop.ko 2023-08-11T15:29:19.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-08-11T15:29:19.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-wmi.ko 2023-08-11T15:29:19.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-08-11T15:29:19.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-08-11T15:29:19.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-08-11T15:29:19.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-08-11T15:29:19.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-08-11T15:29:19.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/hdaps.ko 2023-08-11T15:29:19.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel 2023-08-11T15:29:19.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-08-11T15:29:19.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel_ips.ko 2023-08-11T15:29:19.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/msi-wmi.ko 2023-08-11T15:29:19.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-08-11T15:29:19.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-08-11T15:29:19.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-08-11T15:29:19.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-q10.ko 2023-08-11T15:29:19.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-08-11T15:29:19.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-08-11T15:29:19.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-08-11T15:29:19.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-08-11T15:29:19.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi.ko 2023-08-11T15:29:19.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap 2023-08-11T15:29:19.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_common.ko 2023-08-11T15:29:19.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-08-11T15:29:19.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps 2023-08-11T15:29:19.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients 2023-08-11T15:29:19.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-gpio.ko 2023-08-11T15:29:19.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-08-11T15:29:19.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps_parport.ko 2023-08-11T15:29:19.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/pps_core.ko 2023-08-11T15:29:19.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp 2023-08-11T15:29:19.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp.ko 2023-08-11T15:29:19.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp_kvm.ko 2023-08-11T15:29:19.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc 2023-08-11T15:29:19.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq32k.ko 2023-08-11T15:29:19.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq4802.ko 2023-08-11T15:29:19.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1286.ko 2023-08-11T15:29:19.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1307.ko 2023-08-11T15:29:19.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1374.ko 2023-08-11T15:29:19.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1511.ko 2023-08-11T15:29:19.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1553.ko 2023-08-11T15:29:19.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1672.ko 2023-08-11T15:29:19.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1742.ko 2023-08-11T15:29:19.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds2404.ko 2023-08-11T15:29:19.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds3232.ko 2023-08-11T15:29:19.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-em3027.ko 2023-08-11T15:29:19.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-fm3130.ko 2023-08-11T15:29:19.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl12022.ko 2023-08-11T15:29:19.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl1208.ko 2023-08-11T15:29:19.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m41t80.ko 2023-08-11T15:29:19.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t35.ko 2023-08-11T15:29:19.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t59.ko 2023-08-11T15:29:19.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-max6900.ko 2023-08-11T15:29:19.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-msm6242.ko 2023-08-11T15:29:19.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-08-11T15:29:19.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-08-11T15:29:19.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-08-11T15:29:19.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-08-11T15:29:19.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-08-11T15:29:19.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-08-11T15:29:19.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-08-11T15:29:19.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-08-11T15:29:19.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8025.ko 2023-08-11T15:29:19.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8581.ko 2023-08-11T15:29:19.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-08-11T15:29:19.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-x1205.ko 2023-08-11T15:29:19.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi 2023-08-11T15:29:19.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-9xxx.ko 2023-08-11T15:29:19.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-sas.ko 2023-08-11T15:29:19.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-xxxx.ko 2023-08-11T15:29:19.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/BusLogic.ko 2023-08-11T15:29:19.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/a100u2w.ko 2023-08-11T15:29:19.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid 2023-08-11T15:29:19.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-08-11T15:29:19.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/advansys.ko 2023-08-11T15:29:19.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx 2023-08-11T15:29:19.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-08-11T15:29:19.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-08-11T15:29:19.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/am53c974.ko 2023-08-11T15:29:19.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr 2023-08-11T15:29:19.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-08-11T15:29:19.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/atp870u.ko 2023-08-11T15:29:19.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi 2023-08-11T15:29:19.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-08-11T15:29:19.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa 2023-08-11T15:29:19.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa/bfa.ko 2023-08-11T15:29:19.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc 2023-08-11T15:29:19.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-08-11T15:29:19.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i 2023-08-11T15:29:19.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-08-11T15:29:19.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ch.ko 2023-08-11T15:29:19.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor 2023-08-11T15:29:19.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-08-11T15:29:19.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi 2023-08-11T15:29:19.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-08-11T15:29:19.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-08-11T15:29:19.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-08-11T15:29:19.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-08-11T15:29:19.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-08-11T15:29:19.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dc395x.ko 2023-08-11T15:29:19.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler 2023-08-11T15:29:19.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-08-11T15:29:19.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-08-11T15:29:19.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-08-11T15:29:19.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-08-11T15:29:19.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dmx3191d.ko 2023-08-11T15:29:19.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r 2023-08-11T15:29:19.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-08-11T15:29:19.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esp_scsi.ko 2023-08-11T15:29:19.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe 2023-08-11T15:29:19.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-08-11T15:29:19.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-08-11T15:29:19.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic 2023-08-11T15:29:19.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic/fnic.ko 2023-08-11T15:29:19.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hpsa.ko 2023-08-11T15:29:19.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hptiop.ko 2023-08-11T15:29:19.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hv_storvsc.ko 2023-08-11T15:29:19.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/initio.ko 2023-08-11T15:29:19.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ipr.ko 2023-08-11T15:29:19.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ips.ko 2023-08-11T15:29:19.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci 2023-08-11T15:29:19.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci/isci.ko 2023-08-11T15:29:19.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-08-11T15:29:19.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_tcp.ko 2023-08-11T15:29:19.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc 2023-08-11T15:29:19.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc/libfc.ko 2023-08-11T15:29:19.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi.ko 2023-08-11T15:29:19.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-08-11T15:29:19.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas 2023-08-11T15:29:19.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas/libsas.ko 2023-08-11T15:29:19.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc 2023-08-11T15:29:19.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-08-11T15:29:19.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid 2023-08-11T15:29:19.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid.ko 2023-08-11T15:29:19.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-08-11T15:29:19.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-08-11T15:29:19.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-08-11T15:29:19.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas 2023-08-11T15:29:19.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-08-11T15:29:19.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas 2023-08-11T15:29:19.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-08-11T15:29:19.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvumi.ko 2023-08-11T15:29:19.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001 2023-08-11T15:29:19.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-08-11T15:29:19.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pmcraid.ko 2023-08-11T15:29:19.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla1280.ko 2023-08-11T15:29:19.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx 2023-08-11T15:29:19.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-08-11T15:29:19.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-08-11T15:29:19.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx 2023-08-11T15:29:19.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-08-11T15:29:19.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/raid_class.ko 2023-08-11T15:29:19.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_debug.ko 2023-08-11T15:29:19.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-08-11T15:29:19.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-08-11T15:29:19.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-08-11T15:29:19.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-08-11T15:29:19.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-08-11T15:29:19.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ses.ko 2023-08-11T15:29:19.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/st.ko 2023-08-11T15:29:19.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/stex.ko 2023-08-11T15:29:19.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2 2023-08-11T15:29:19.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-08-11T15:29:19.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/virtio_scsi.ko 2023-08-11T15:29:19.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-08-11T15:29:19.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/wd719x.ko 2023-08-11T15:29:19.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb 2023-08-11T15:29:19.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb/ssb.ko 2023-08-11T15:29:19.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target 2023-08-11T15:29:19.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi 2023-08-11T15:29:19.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-08-11T15:29:19.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback 2023-08-11T15:29:19.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback/tcm_loop.ko 2023-08-11T15:29:19.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_file.ko 2023-08-11T15:29:19.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_iblock.ko 2023-08-11T15:29:19.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_mod.ko 2023-08-11T15:29:19.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_pscsi.ko 2023-08-11T15:29:19.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_user.ko 2023-08-11T15:29:19.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc 2023-08-11T15:29:19.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-08-11T15:29:19.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal 2023-08-11T15:29:19.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel 2023-08-11T15:29:19.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal 2023-08-11T15:29:19.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-08-11T15:29:19.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-08-11T15:29:19.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-08-11T15:29:19.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-08-11T15:29:19.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-08-11T15:29:19.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-08-11T15:29:19.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-08-11T15:29:19.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-08-11T15:29:19.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-08-11T15:29:19.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-08-11T15:29:19.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-08-11T15:29:19.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-08-11T15:29:19.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-08-11T15:29:19.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-08-11T15:29:19.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-08-11T15:29:19.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty 2023-08-11T15:29:19.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless 2023-08-11T15:29:19.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-08-11T15:29:19.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_gsm.ko 2023-08-11T15:29:19.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_hdlc.ko 2023-08-11T15:29:19.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/nozomi.ko 2023-08-11T15:29:19.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial 2023-08-11T15:29:19.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250 2023-08-11T15:29:19.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-08-11T15:29:19.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/arc_uart.ko 2023-08-11T15:29:19.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm 2023-08-11T15:29:19.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-08-11T15:29:19.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/synclink_gt.ko 2023-08-11T15:29:19.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs 2023-08-11T15:29:19.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core 2023-08-11T15:29:19.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-08-11T15:29:19.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host 2023-08-11T15:29:19.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-08-11T15:29:19.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio 2023-08-11T15:29:19.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio.ko 2023-08-11T15:29:19.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_aec.ko 2023-08-11T15:29:19.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_cif.ko 2023-08-11T15:29:19.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_pci_generic.ko 2023-08-11T15:29:19.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_sercos3.ko 2023-08-11T15:29:19.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb 2023-08-11T15:29:19.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class 2023-08-11T15:29:19.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-acm.ko 2023-08-11T15:29:19.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-wdm.ko 2023-08-11T15:29:19.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usblp.ko 2023-08-11T15:29:19.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usbtmc.ko 2023-08-11T15:29:19.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host 2023-08-11T15:29:19.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host/sl811-hcd.ko 2023-08-11T15:29:19.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image 2023-08-11T15:29:19.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/mdc800.ko 2023-08-11T15:29:19.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/microtek.ko 2023-08-11T15:29:19.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc 2023-08-11T15:29:19.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/adutux.ko 2023-08-11T15:29:19.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/appledisplay.ko 2023-08-11T15:29:19.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi26.ko 2023-08-11T15:29:19.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi62.ko 2023-08-11T15:29:19.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ezusb.ko 2023-08-11T15:29:19.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/idmouse.ko 2023-08-11T15:29:19.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/iowarrior.ko 2023-08-11T15:29:19.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/isight_firmware.ko 2023-08-11T15:29:19.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ldusb.ko 2023-08-11T15:29:19.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/legousbtower.ko 2023-08-11T15:29:19.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga 2023-08-11T15:29:19.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-08-11T15:29:19.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/trancevibrator.ko 2023-08-11T15:29:19.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usb3503.ko 2023-08-11T15:29:19.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usblcd.ko 2023-08-11T15:29:19.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usbsevseg.ko 2023-08-11T15:29:19.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/uss720.ko 2023-08-11T15:29:19.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/yurex.ko 2023-08-11T15:29:19.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy 2023-08-11T15:29:19.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy/phy-generic.ko 2023-08-11T15:29:19.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial 2023-08-11T15:29:19.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/aircable.ko 2023-08-11T15:29:19.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ark3116.ko 2023-08-11T15:29:19.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/belkin_sa.ko 2023-08-11T15:29:19.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ch341.ko 2023-08-11T15:29:19.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cp210x.ko 2023-08-11T15:29:19.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cyberjack.ko 2023-08-11T15:29:19.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cypress_m8.ko 2023-08-11T15:29:19.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-08-11T15:29:19.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/empeg.ko 2023-08-11T15:29:19.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-08-11T15:29:19.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/garmin_gps.ko 2023-08-11T15:29:19.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_edgeport.ko 2023-08-11T15:29:19.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_ti.ko 2023-08-11T15:29:19.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipaq.ko 2023-08-11T15:29:19.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipw.ko 2023-08-11T15:29:19.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ir-usb.ko 2023-08-11T15:29:19.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-08-11T15:29:19.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan.ko 2023-08-11T15:29:19.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-08-11T15:29:19.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-08-11T15:29:19.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kobil_sct.ko 2023-08-11T15:29:19.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mct_u232.ko 2023-08-11T15:29:19.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7720.ko 2023-08-11T15:29:19.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7840.ko 2023-08-11T15:29:19.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/navman.ko 2023-08-11T15:29:19.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/omninet.ko 2023-08-11T15:29:19.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/opticon.ko 2023-08-11T15:29:19.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/option.ko 2023-08-11T15:29:19.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/oti6858.ko 2023-08-11T15:29:19.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/pl2303.ko 2023-08-11T15:29:19.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcaux.ko 2023-08-11T15:29:19.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcserial.ko 2023-08-11T15:29:19.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/quatech2.ko 2023-08-11T15:29:19.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/safe_serial.ko 2023-08-11T15:29:19.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/sierra.ko 2023-08-11T15:29:19.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/spcp8x5.ko 2023-08-11T15:29:19.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ssu100.ko 2023-08-11T15:29:19.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/symbolserial.ko 2023-08-11T15:29:19.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-08-11T15:29:19.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-08-11T15:29:19.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_debug.ko 2023-08-11T15:29:19.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_wwan.ko 2023-08-11T15:29:19.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/visor.ko 2023-08-11T15:29:19.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/whiteheat.ko 2023-08-11T15:29:19.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/xsens_mt.ko 2023-08-11T15:29:19.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage 2023-08-11T15:29:19.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/uas.ko 2023-08-11T15:29:19.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-alauda.ko 2023-08-11T15:29:19.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-cypress.ko 2023-08-11T15:29:19.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-datafab.ko 2023-08-11T15:29:19.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-08-11T15:29:19.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-freecom.ko 2023-08-11T15:29:19.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-isd200.ko 2023-08-11T15:29:19.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-08-11T15:29:19.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-karma.ko 2023-08-11T15:29:19.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-08-11T15:29:19.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-realtek.ko 2023-08-11T15:29:19.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-08-11T15:29:19.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-08-11T15:29:19.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-usbat.ko 2023-08-11T15:29:19.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/usb-storage.ko 2023-08-11T15:29:19.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip 2023-08-11T15:29:19.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-core.ko 2023-08-11T15:29:19.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-host.ko 2023-08-11T15:29:19.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-08-11T15:29:19.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio 2023-08-11T15:29:19.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci 2023-08-11T15:29:19.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-08-11T15:29:19.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-08-11T15:29:19.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio.ko 2023-08-11T15:29:19.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-08-11T15:29:19.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost 2023-08-11T15:29:19.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost.ko 2023-08-11T15:29:19.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_iotlb.ko 2023-08-11T15:29:19.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_net.ko 2023-08-11T15:29:19.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_scsi.ko 2023-08-11T15:29:19.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video 2023-08-11T15:29:19.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight 2023-08-11T15:29:19.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/apple_bl.ko 2023-08-11T15:29:19.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/lcd.ko 2023-08-11T15:29:19.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/platform_lcd.ko 2023-08-11T15:29:19.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio 2023-08-11T15:29:19.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio.ko 2023-08-11T15:29:19.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_balloon.ko 2023-08-11T15:29:19.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_mmio.ko 2023-08-11T15:29:19.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci.ko 2023-08-11T15:29:19.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-08-11T15:29:19.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-08-11T15:29:19.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_ring.ko 2023-08-11T15:29:19.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1 2023-08-11T15:29:19.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters 2023-08-11T15:29:19.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2482.ko 2023-08-11T15:29:19.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2490.ko 2023-08-11T15:29:19.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves 2023-08-11T15:29:19.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-08-11T15:29:19.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-08-11T15:29:19.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-08-11T15:29:19.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-08-11T15:29:19.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-08-11T15:29:19.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-08-11T15:29:19.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-08-11T15:29:19.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-08-11T15:29:19.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-08-11T15:29:19.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_smem.ko 2023-08-11T15:29:19.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_therm.ko 2023-08-11T15:29:19.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/wire.ko 2023-08-11T15:29:19.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog 2023-08-11T15:29:19.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-08-11T15:29:19.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-08-11T15:29:19.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-08-11T15:29:19.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/hpwdt.ko 2023-08-11T15:29:19.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/i6300esb.ko 2023-08-11T15:29:19.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-08-11T15:29:19.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-08-11T15:29:19.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ib700wdt.ko 2023-08-11T15:29:19.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ibmasr.ko 2023-08-11T15:29:19.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-08-11T15:29:19.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-08-11T15:29:19.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it87_wdt.ko 2023-08-11T15:29:19.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/machzwd.ko 2023-08-11T15:29:19.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/nv_tco.ko 2023-08-11T15:29:19.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_pci.ko 2023-08-11T15:29:19.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_usb.ko 2023-08-11T15:29:19.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-08-11T15:29:19.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-08-11T15:29:19.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/softdog.ko 2023-08-11T15:29:19.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sp5100_tco.ko 2023-08-11T15:29:19.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/via_wdt.ko 2023-08-11T15:29:19.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-08-11T15:29:19.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-08-11T15:29:19.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-08-11T15:29:19.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/wdt_pci.ko 2023-08-11T15:29:19.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/xen_wdt.ko 2023-08-11T15:29:19.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen 2023-08-11T15:29:19.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-acpi-processor.ko 2023-08-11T15:29:19.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-evtchn.ko 2023-08-11T15:29:19.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntalloc.ko 2023-08-11T15:29:19.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntdev.ko 2023-08-11T15:29:19.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback 2023-08-11T15:29:19.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-08-11T15:29:19.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-privcmd.ko 2023-08-11T15:29:19.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs 2023-08-11T15:29:19.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs/xenfs.ko 2023-08-11T15:29:19.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs 2023-08-11T15:29:19.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p 2023-08-11T15:29:19.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p/9p.ko 2023-08-11T15:29:19.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs 2023-08-11T15:29:19.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs/affs.ko 2023-08-11T15:29:19.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs 2023-08-11T15:29:19.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs/befs.ko 2023-08-11T15:29:19.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/binfmt_misc.ko 2023-08-11T15:29:19.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs 2023-08-11T15:29:19.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs/btrfs.ko 2023-08-11T15:29:19.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles 2023-08-11T15:29:19.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles/cachefiles.ko 2023-08-11T15:29:19.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph 2023-08-11T15:29:19.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph/ceph.ko 2023-08-11T15:29:19.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda 2023-08-11T15:29:19.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda/coda.ko 2023-08-11T15:29:19.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs 2023-08-11T15:29:19.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs/cramfs.ko 2023-08-11T15:29:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm 2023-08-11T15:29:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm/dlm.ko 2023-08-11T15:29:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs 2023-08-11T15:29:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs/ecryptfs.ko 2023-08-11T15:29:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs 2023-08-11T15:29:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs/f2fs.ko 2023-08-11T15:29:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat 2023-08-11T15:29:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/fat.ko 2023-08-11T15:29:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/msdos.ko 2023-08-11T15:29:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/vfat.ko 2023-08-11T15:29:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache 2023-08-11T15:29:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache/fscache.ko 2023-08-11T15:29:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse 2023-08-11T15:29:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/cuse.ko 2023-08-11T15:29:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/fuse.ko 2023-08-11T15:29:19.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2 2023-08-11T15:29:19.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2/gfs2.ko 2023-08-11T15:29:19.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs 2023-08-11T15:29:19.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs/hfs.ko 2023-08-11T15:29:19.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus 2023-08-11T15:29:19.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus/hfsplus.ko 2023-08-11T15:29:19.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs 2023-08-11T15:29:19.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs/isofs.ko 2023-08-11T15:29:19.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs 2023-08-11T15:29:19.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs/jfs.ko 2023-08-11T15:29:19.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd 2023-08-11T15:29:19.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd/lockd.ko 2023-08-11T15:29:19.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix 2023-08-11T15:29:19.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix/minix.ko 2023-08-11T15:29:19.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs 2023-08-11T15:29:19.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs/netfs.ko 2023-08-11T15:29:19.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs 2023-08-11T15:29:19.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout 2023-08-11T15:29:19.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-08-11T15:29:19.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout 2023-08-11T15:29:19.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-08-11T15:29:19.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout 2023-08-11T15:29:19.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-08-11T15:29:19.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfs.ko 2023-08-11T15:29:19.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv3.ko 2023-08-11T15:29:19.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv4.ko 2023-08-11T15:29:19.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common 2023-08-11T15:29:19.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/grace.ko 2023-08-11T15:29:19.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/nfs_acl.ko 2023-08-11T15:29:19.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd 2023-08-11T15:29:19.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd/nfsd.ko 2023-08-11T15:29:19.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2 2023-08-11T15:29:19.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2/nilfs2.ko 2023-08-11T15:29:19.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls 2023-08-11T15:29:19.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-celtic.ko 2023-08-11T15:29:19.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-centeuro.ko 2023-08-11T15:29:19.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-croatian.ko 2023-08-11T15:29:19.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-cyrillic.ko 2023-08-11T15:29:19.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-gaelic.ko 2023-08-11T15:29:19.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-greek.ko 2023-08-11T15:29:19.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-iceland.ko 2023-08-11T15:29:19.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-inuit.ko 2023-08-11T15:29:19.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-roman.ko 2023-08-11T15:29:19.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-romanian.ko 2023-08-11T15:29:19.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-turkish.ko 2023-08-11T15:29:19.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1250.ko 2023-08-11T15:29:19.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1251.ko 2023-08-11T15:29:19.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1255.ko 2023-08-11T15:29:19.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp737.ko 2023-08-11T15:29:19.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp775.ko 2023-08-11T15:29:19.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp850.ko 2023-08-11T15:29:19.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp852.ko 2023-08-11T15:29:19.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp855.ko 2023-08-11T15:29:19.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp857.ko 2023-08-11T15:29:19.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp860.ko 2023-08-11T15:29:19.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp861.ko 2023-08-11T15:29:19.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp862.ko 2023-08-11T15:29:19.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp863.ko 2023-08-11T15:29:19.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp864.ko 2023-08-11T15:29:19.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp865.ko 2023-08-11T15:29:19.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp866.ko 2023-08-11T15:29:19.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp869.ko 2023-08-11T15:29:19.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp874.ko 2023-08-11T15:29:19.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp932.ko 2023-08-11T15:29:19.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp936.ko 2023-08-11T15:29:19.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp949.ko 2023-08-11T15:29:19.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp950.ko 2023-08-11T15:29:19.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_euc-jp.ko 2023-08-11T15:29:19.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-1.ko 2023-08-11T15:29:19.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-13.ko 2023-08-11T15:29:19.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-14.ko 2023-08-11T15:29:19.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-15.ko 2023-08-11T15:29:19.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-2.ko 2023-08-11T15:29:19.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-3.ko 2023-08-11T15:29:19.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-4.ko 2023-08-11T15:29:19.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-5.ko 2023-08-11T15:29:19.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-6.ko 2023-08-11T15:29:19.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-7.ko 2023-08-11T15:29:19.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-9.ko 2023-08-11T15:29:19.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-r.ko 2023-08-11T15:29:19.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-ru.ko 2023-08-11T15:29:19.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-u.ko 2023-08-11T15:29:19.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_utf8.ko 2023-08-11T15:29:19.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2 2023-08-11T15:29:19.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster 2023-08-11T15:29:19.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-08-11T15:29:19.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm 2023-08-11T15:29:19.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-08-11T15:29:19.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs 2023-08-11T15:29:19.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-08-11T15:29:19.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2.ko 2023-08-11T15:29:19.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-08-11T15:29:19.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-08-11T15:29:19.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-08-11T15:29:19.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs 2023-08-11T15:29:19.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs/overlay.ko 2023-08-11T15:29:19.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore 2023-08-11T15:29:19.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore/ramoops.ko 2023-08-11T15:29:19.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs 2023-08-11T15:29:19.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs/reiserfs.ko 2023-08-11T15:29:19.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs 2023-08-11T15:29:19.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs/romfs.ko 2023-08-11T15:29:19.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb 2023-08-11T15:29:19.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client 2023-08-11T15:29:19.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client/cifs.ko 2023-08-11T15:29:19.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common 2023-08-11T15:29:19.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_arc4.ko 2023-08-11T15:29:19.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_md4.ko 2023-08-11T15:29:19.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs 2023-08-11T15:29:19.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs/squashfs.ko 2023-08-11T15:29:19.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv 2023-08-11T15:29:19.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv/sysv.ko 2023-08-11T15:29:19.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf 2023-08-11T15:29:19.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf/udf.ko 2023-08-11T15:29:19.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs 2023-08-11T15:29:19.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs/ufs.ko 2023-08-11T15:29:19.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs 2023-08-11T15:29:19.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs/xfs.ko 2023-08-11T15:29:19.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel 2023-08-11T15:29:19.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu 2023-08-11T15:29:19.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu/rcutorture.ko 2023-08-11T15:29:19.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/torture.ko 2023-08-11T15:29:19.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace 2023-08-11T15:29:19.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-08-11T15:29:19.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib 2023-08-11T15:29:19.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/asn1_encoder.ko 2023-08-11T15:29:19.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc-itu-t.ko 2023-08-11T15:29:19.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc8.ko 2023-08-11T15:29:19.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto 2023-08-11T15:29:19.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libarc4.ko 2023-08-11T15:29:19.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libdes.ko 2023-08-11T15:29:19.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/libcrc32c.ko 2023-08-11T15:29:19.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lru_cache.ko 2023-08-11T15:29:19.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4 2023-08-11T15:29:19.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4_compress.ko 2023-08-11T15:29:19.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4hc_compress.ko 2023-08-11T15:29:19.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math 2023-08-11T15:29:19.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math/cordic.ko 2023-08-11T15:29:19.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6 2023-08-11T15:29:19.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6/raid6_pq.ko 2023-08-11T15:29:19.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon 2023-08-11T15:29:19.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon/reed_solomon.ko 2023-08-11T15:29:19.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_bm.ko 2023-08-11T15:29:19.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_fsm.ko 2023-08-11T15:29:19.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_kmp.ko 2023-08-11T15:29:19.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd 2023-08-11T15:29:19.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd/zstd_compress.ko 2023-08-11T15:29:19.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm 2023-08-11T15:29:19.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm/hwpoison-inject.ko 2023-08-11T15:29:19.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net 2023-08-11T15:29:19.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan 2023-08-11T15:29:19.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/6lowpan.ko 2023-08-11T15:29:19.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_dest.ko 2023-08-11T15:29:19.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_fragment.ko 2023-08-11T15:29:19.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_hop.ko 2023-08-11T15:29:19.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_ipv6.ko 2023-08-11T15:29:19.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_mobility.ko 2023-08-11T15:29:19.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_routing.ko 2023-08-11T15:29:19.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_udp.ko 2023-08-11T15:29:19.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802 2023-08-11T15:29:19.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/garp.ko 2023-08-11T15:29:19.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/mrp.ko 2023-08-11T15:29:19.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/p8022.ko 2023-08-11T15:29:19.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/psnap.ko 2023-08-11T15:29:19.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/stp.ko 2023-08-11T15:29:19.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q 2023-08-11T15:29:19.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q/8021q.ko 2023-08-11T15:29:19.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p 2023-08-11T15:29:19.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet.ko 2023-08-11T15:29:19.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_fd.ko 2023-08-11T15:29:19.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_rdma.ko 2023-08-11T15:29:19.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_virtio.ko 2023-08-11T15:29:19.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv 2023-08-11T15:29:19.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv/batman-adv.ko 2023-08-11T15:29:19.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge 2023-08-11T15:29:19.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/br_netfilter.ko 2023-08-11T15:29:19.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/bridge.ko 2023-08-11T15:29:19.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter 2023-08-11T15:29:19.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-08-11T15:29:19.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_among.ko 2023-08-11T15:29:19.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-08-11T15:29:19.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-08-11T15:29:19.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-08-11T15:29:19.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-08-11T15:29:19.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-08-11T15:29:19.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-08-11T15:29:19.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_log.ko 2023-08-11T15:29:19.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-08-11T15:29:19.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-08-11T15:29:19.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-08-11T15:29:19.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-08-11T15:29:19.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-08-11T15:29:19.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-08-11T15:29:19.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-08-11T15:29:19.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-08-11T15:29:19.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-08-11T15:29:19.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-08-11T15:29:19.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-08-11T15:29:19.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtables.ko 2023-08-11T15:29:19.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-08-11T15:29:19.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-08-11T15:29:19.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-08-11T15:29:19.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph 2023-08-11T15:29:19.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph/libceph.ko 2023-08-11T15:29:19.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core 2023-08-11T15:29:19.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/failover.ko 2023-08-11T15:29:19.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/pktgen.ko 2023-08-11T15:29:19.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp 2023-08-11T15:29:19.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp.ko 2023-08-11T15:29:19.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_diag.ko 2023-08-11T15:29:19.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv4.ko 2023-08-11T15:29:19.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv6.ko 2023-08-11T15:29:19.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver 2023-08-11T15:29:19.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver/dns_resolver.ko 2023-08-11T15:29:19.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa 2023-08-11T15:29:19.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/dsa_core.ko 2023-08-11T15:29:19.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_brcm.ko 2023-08-11T15:29:19.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_dsa.ko 2023-08-11T15:29:19.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_none.ko 2023-08-11T15:29:19.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_trailer.ko 2023-08-11T15:29:19.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154 2023-08-11T15:29:19.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan 2023-08-11T15:29:19.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-08-11T15:29:19.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154.ko 2023-08-11T15:29:19.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154_socket.ko 2023-08-11T15:29:19.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4 2023-08-11T15:29:19.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ah4.ko 2023-08-11T15:29:19.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/esp4.ko 2023-08-11T15:29:19.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/fou.ko 2023-08-11T15:29:19.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/gre.ko 2023-08-11T15:29:19.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/inet_diag.ko 2023-08-11T15:29:19.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_gre.ko 2023-08-11T15:29:19.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_tunnel.ko 2023-08-11T15:29:19.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_vti.ko 2023-08-11T15:29:19.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipcomp.ko 2023-08-11T15:29:19.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipip.ko 2023-08-11T15:29:19.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter 2023-08-11T15:29:19.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-08-11T15:29:19.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-08-11T15:29:19.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-08-11T15:29:19.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-08-11T15:29:19.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-08-11T15:29:19.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-08-11T15:29:19.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-08-11T15:29:19.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-08-11T15:29:19.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-08-11T15:29:19.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-08-11T15:29:19.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-08-11T15:29:19.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-08-11T15:29:19.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-08-11T15:29:19.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-08-11T15:29:19.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-08-11T15:29:19.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-08-11T15:29:19.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-08-11T15:29:19.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-08-11T15:29:19.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-08-11T15:29:19.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-08-11T15:29:19.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-08-11T15:29:19.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_bic.ko 2023-08-11T15:29:19.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_dctcp.ko 2023-08-11T15:29:19.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_diag.ko 2023-08-11T15:29:19.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_highspeed.ko 2023-08-11T15:29:19.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_htcp.ko 2023-08-11T15:29:19.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_hybla.ko 2023-08-11T15:29:19.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_illinois.ko 2023-08-11T15:29:19.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_lp.ko 2023-08-11T15:29:19.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_scalable.ko 2023-08-11T15:29:19.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_vegas.ko 2023-08-11T15:29:19.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_veno.ko 2023-08-11T15:29:19.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_westwood.ko 2023-08-11T15:29:19.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_yeah.ko 2023-08-11T15:29:19.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tunnel4.ko 2023-08-11T15:29:19.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_diag.ko 2023-08-11T15:29:19.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_tunnel.ko 2023-08-11T15:29:19.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-08-11T15:29:19.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6 2023-08-11T15:29:19.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ah6.ko 2023-08-11T15:29:19.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/esp6.ko 2023-08-11T15:29:19.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/fou6.ko 2023-08-11T15:29:19.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_tunnel.ko 2023-08-11T15:29:19.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-08-11T15:29:19.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_vti.ko 2023-08-11T15:29:19.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ipcomp6.ko 2023-08-11T15:29:19.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter 2023-08-11T15:29:19.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-08-11T15:29:19.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-08-11T15:29:19.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-08-11T15:29:19.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-08-11T15:29:19.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-08-11T15:29:19.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-08-11T15:29:19.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-08-11T15:29:19.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-08-11T15:29:19.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-08-11T15:29:19.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-08-11T15:29:19.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-08-11T15:29:19.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-08-11T15:29:19.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-08-11T15:29:19.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-08-11T15:29:19.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-08-11T15:29:19.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-08-11T15:29:19.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-08-11T15:29:19.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-08-11T15:29:19.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-08-11T15:29:19.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-08-11T15:29:19.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-08-11T15:29:19.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-08-11T15:29:19.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-08-11T15:29:19.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-08-11T15:29:19.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/sit.ko 2023-08-11T15:29:19.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/tunnel6.ko 2023-08-11T15:29:19.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-08-11T15:29:19.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key 2023-08-11T15:29:19.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key/af_key.ko 2023-08-11T15:29:19.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp 2023-08-11T15:29:19.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_core.ko 2023-08-11T15:29:19.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_debugfs.ko 2023-08-11T15:29:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_eth.ko 2023-08-11T15:29:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip.ko 2023-08-11T15:29:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip6.ko 2023-08-11T15:29:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_netlink.ko 2023-08-11T15:29:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ppp.ko 2023-08-11T15:29:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc 2023-08-11T15:29:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc/llc.ko 2023-08-11T15:29:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154 2023-08-11T15:29:19.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154/mac802154.ko 2023-08-11T15:29:19.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls 2023-08-11T15:29:19.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls/mpls_gso.ko 2023-08-11T15:29:19.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter 2023-08-11T15:29:19.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset 2023-08-11T15:29:19.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set.ko 2023-08-11T15:29:19.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-08-11T15:29:19.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-08-11T15:29:19.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-08-11T15:29:19.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-08-11T15:29:19.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-08-11T15:29:19.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-08-11T15:29:19.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-08-11T15:29:19.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-08-11T15:29:19.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-08-11T15:29:19.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-08-11T15:29:19.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-08-11T15:29:19.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-08-11T15:29:19.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-08-11T15:29:19.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-08-11T15:29:19.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-08-11T15:29:19.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs 2023-08-11T15:29:19.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-08-11T15:29:19.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-08-11T15:29:19.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-08-11T15:29:19.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-08-11T15:29:19.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-08-11T15:29:19.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-08-11T15:29:19.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-08-11T15:29:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-08-11T15:29:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-08-11T15:29:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-08-11T15:29:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-08-11T15:29:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-08-11T15:29:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-08-11T15:29:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-08-11T15:29:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conncount.ko 2023-08-11T15:29:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack.ko 2023-08-11T15:29:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-08-11T15:29:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-08-11T15:29:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-08-11T15:29:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-08-11T15:29:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-08-11T15:29:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-08-11T15:29:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-08-11T15:29:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-08-11T15:29:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-08-11T15:29:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-08-11T15:29:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-08-11T15:29:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-08-11T15:29:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_dup_netdev.ko 2023-08-11T15:29:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table.ko 2023-08-11T15:29:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-08-11T15:29:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_log_syslog.ko 2023-08-11T15:29:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat.ko 2023-08-11T15:29:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_amanda.ko 2023-08-11T15:29:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_ftp.ko 2023-08-11T15:29:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_irc.ko 2023-08-11T15:29:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_sip.ko 2023-08-11T15:29:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_tftp.ko 2023-08-11T15:29:19.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_synproxy_core.ko 2023-08-11T15:29:19.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_tables.ko 2023-08-11T15:29:19.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink.ko 2023-08-11T15:29:19.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_acct.ko 2023-08-11T15:29:19.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_log.ko 2023-08-11T15:29:19.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_osf.ko 2023-08-11T15:29:19.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_queue.ko 2023-08-11T15:29:19.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_chain_nat.ko 2023-08-11T15:29:19.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_compat.ko 2023-08-11T15:29:19.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_ct.ko 2023-08-11T15:29:19.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_dup_netdev.ko 2023-08-11T15:29:19.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fib.ko 2023-08-11T15:29:19.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-08-11T15:29:19.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_hash.ko 2023-08-11T15:29:19.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_limit.ko 2023-08-11T15:29:19.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_log.ko 2023-08-11T15:29:19.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_masq.ko 2023-08-11T15:29:19.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_nat.ko 2023-08-11T15:29:19.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_numgen.ko 2023-08-11T15:29:19.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_queue.ko 2023-08-11T15:29:19.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_quota.ko 2023-08-11T15:29:19.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_redir.ko 2023-08-11T15:29:19.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject.ko 2023-08-11T15:29:19.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject_inet.ko 2023-08-11T15:29:19.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_socket.ko 2023-08-11T15:29:19.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_tunnel.ko 2023-08-11T15:29:19.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_AUDIT.ko 2023-08-11T15:29:19.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-08-11T15:29:19.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-08-11T15:29:19.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-08-11T15:29:19.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CT.ko 2023-08-11T15:29:19.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_DSCP.ko 2023-08-11T15:29:19.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HL.ko 2023-08-11T15:29:19.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HMARK.ko 2023-08-11T15:29:19.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-08-11T15:29:19.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LED.ko 2023-08-11T15:29:19.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LOG.ko 2023-08-11T15:29:19.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-08-11T15:29:19.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NETMAP.ko 2023-08-11T15:29:19.860 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFLOG.ko 2023-08-11T15:29:19.860 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-08-11T15:29:19.860 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_RATEEST.ko 2023-08-11T15:29:19.860 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_REDIRECT.ko 2023-08-11T15:29:19.860 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_SECMARK.ko 2023-08-11T15:29:19.860 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPMSS.ko 2023-08-11T15:29:19.860 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-08-11T15:29:19.861 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TEE.ko 2023-08-11T15:29:19.861 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TPROXY.ko 2023-08-11T15:29:19.861 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TRACE.ko 2023-08-11T15:29:19.861 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_addrtype.ko 2023-08-11T15:29:19.861 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_bpf.ko 2023-08-11T15:29:19.861 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cgroup.ko 2023-08-11T15:29:19.861 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cluster.ko 2023-08-11T15:29:19.861 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_comment.ko 2023-08-11T15:29:19.862 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connbytes.ko 2023-08-11T15:29:19.862 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlabel.ko 2023-08-11T15:29:19.862 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlimit.ko 2023-08-11T15:29:19.862 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connmark.ko 2023-08-11T15:29:19.862 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_conntrack.ko 2023-08-11T15:29:19.862 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cpu.ko 2023-08-11T15:29:19.862 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dccp.ko 2023-08-11T15:29:19.862 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_devgroup.ko 2023-08-11T15:29:19.863 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dscp.ko 2023-08-11T15:29:19.863 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ecn.ko 2023-08-11T15:29:19.863 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_esp.ko 2023-08-11T15:29:19.863 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hashlimit.ko 2023-08-11T15:29:19.863 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_helper.ko 2023-08-11T15:29:19.863 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hl.ko 2023-08-11T15:29:19.863 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipcomp.ko 2023-08-11T15:29:19.863 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_iprange.ko 2023-08-11T15:29:19.864 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipvs.ko 2023-08-11T15:29:19.864 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_l2tp.ko 2023-08-11T15:29:19.864 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_length.ko 2023-08-11T15:29:19.864 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_limit.ko 2023-08-11T15:29:19.864 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mac.ko 2023-08-11T15:29:19.864 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mark.ko 2023-08-11T15:29:19.864 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_multiport.ko 2023-08-11T15:29:19.865 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nat.ko 2023-08-11T15:29:19.865 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nfacct.ko 2023-08-11T15:29:19.865 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_osf.ko 2023-08-11T15:29:19.865 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_owner.ko 2023-08-11T15:29:19.865 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_physdev.ko 2023-08-11T15:29:19.865 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_pkttype.ko 2023-08-11T15:29:19.865 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_policy.ko 2023-08-11T15:29:19.865 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_quota.ko 2023-08-11T15:29:19.866 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_rateest.ko 2023-08-11T15:29:19.866 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_realm.ko 2023-08-11T15:29:19.866 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_recent.ko 2023-08-11T15:29:19.866 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_sctp.ko 2023-08-11T15:29:19.866 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_set.ko 2023-08-11T15:29:19.866 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_socket.ko 2023-08-11T15:29:19.866 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_state.ko 2023-08-11T15:29:19.866 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_statistic.ko 2023-08-11T15:29:19.867 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_string.ko 2023-08-11T15:29:19.867 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_tcpmss.ko 2023-08-11T15:29:19.867 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_time.ko 2023-08-11T15:29:19.867 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_u32.ko 2023-08-11T15:29:19.867 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink 2023-08-11T15:29:19.867 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink/netlink_diag.ko 2023-08-11T15:29:19.867 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh 2023-08-11T15:29:19.867 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh/nsh.ko 2023-08-11T15:29:19.868 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch 2023-08-11T15:29:19.868 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/openvswitch.ko 2023-08-11T15:29:19.868 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-geneve.ko 2023-08-11T15:29:19.868 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-gre.ko 2023-08-11T15:29:19.868 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-vxlan.ko 2023-08-11T15:29:19.868 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet 2023-08-11T15:29:19.868 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet/af_packet_diag.ko 2023-08-11T15:29:19.869 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds 2023-08-11T15:29:19.869 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds.ko 2023-08-11T15:29:19.869 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_rdma.ko 2023-08-11T15:29:19.869 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_tcp.ko 2023-08-11T15:29:19.869 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched 2023-08-11T15:29:19.869 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_csum.ko 2023-08-11T15:29:19.869 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_gact.ko 2023-08-11T15:29:19.870 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_ipt.ko 2023-08-11T15:29:19.870 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_mirred.ko 2023-08-11T15:29:19.870 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_nat.ko 2023-08-11T15:29:19.870 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_pedit.ko 2023-08-11T15:29:19.870 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_police.ko 2023-08-11T15:29:19.870 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_simple.ko 2023-08-11T15:29:19.870 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_skbedit.ko 2023-08-11T15:29:19.870 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_vlan.ko 2023-08-11T15:29:19.871 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_basic.ko 2023-08-11T15:29:19.871 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_bpf.ko 2023-08-11T15:29:19.871 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_flow.ko 2023-08-11T15:29:19.871 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_fw.ko 2023-08-11T15:29:19.871 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_route.ko 2023-08-11T15:29:19.871 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_u32.ko 2023-08-11T15:29:19.871 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_cmp.ko 2023-08-11T15:29:19.871 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_ipset.ko 2023-08-11T15:29:19.872 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_meta.ko 2023-08-11T15:29:19.872 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_nbyte.ko 2023-08-11T15:29:19.872 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_text.ko 2023-08-11T15:29:19.872 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_u32.ko 2023-08-11T15:29:19.872 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_choke.ko 2023-08-11T15:29:19.872 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_codel.ko 2023-08-11T15:29:19.872 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_drr.ko 2023-08-11T15:29:19.873 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_fq.ko 2023-08-11T15:29:19.873 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_gred.ko 2023-08-11T15:29:19.873 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hfsc.ko 2023-08-11T15:29:19.873 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hhf.ko 2023-08-11T15:29:19.873 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_htb.ko 2023-08-11T15:29:19.873 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_ingress.ko 2023-08-11T15:29:19.873 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio.ko 2023-08-11T15:29:19.873 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio_lib.ko 2023-08-11T15:29:19.874 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_multiq.ko 2023-08-11T15:29:19.874 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_netem.ko 2023-08-11T15:29:19.874 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_pie.ko 2023-08-11T15:29:19.874 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_plug.ko 2023-08-11T15:29:19.874 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_prio.ko 2023-08-11T15:29:19.874 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_qfq.ko 2023-08-11T15:29:19.874 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_red.ko 2023-08-11T15:29:19.874 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfb.ko 2023-08-11T15:29:19.875 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfq.ko 2023-08-11T15:29:19.875 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_tbf.ko 2023-08-11T15:29:19.875 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_teql.ko 2023-08-11T15:29:19.875 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp 2023-08-11T15:29:19.875 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp.ko 2023-08-11T15:29:19.875 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp_diag.ko 2023-08-11T15:29:19.875 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc 2023-08-11T15:29:19.876 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss 2023-08-11T15:29:19.876 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-08-11T15:29:19.876 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-08-11T15:29:19.876 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/sunrpc.ko 2023-08-11T15:29:19.876 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma 2023-08-11T15:29:19.876 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-08-11T15:29:19.876 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc 2023-08-11T15:29:19.876 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/diag.ko 2023-08-11T15:29:19.877 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/tipc.ko 2023-08-11T15:29:19.877 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix 2023-08-11T15:29:19.877 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix/unix_diag.ko 2023-08-11T15:29:19.877 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock 2023-08-11T15:29:19.877 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-08-11T15:29:19.877 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-08-11T15:29:19.877 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock.ko 2023-08-11T15:29:19.878 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_diag.ko 2023-08-11T15:29:19.878 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-08-11T15:29:19.878 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm 2023-08-11T15:29:19.878 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-08-11T15:29:19.878 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security 2023-08-11T15:29:19.878 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys 2023-08-11T15:29:19.878 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys 2023-08-11T15:29:19.878 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-08-11T15:29:19.879 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys 2023-08-11T15:29:19.879 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys/trusted.ko 2023-08-11T15:29:19.879 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt 2023-08-11T15:29:19.879 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib 2023-08-11T15:29:19.879 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib/irqbypass.ko 2023-08-11T15:29:19.879 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias 2023-08-11T15:29:19.879 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias.bin 2023-08-11T15:29:19.879 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin 2023-08-11T15:29:19.880 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.alias.bin 2023-08-11T15:29:19.880 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.bin 2023-08-11T15:29:19.880 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.modinfo 2023-08-11T15:29:19.880 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep 2023-08-11T15:29:19.880 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep.bin 2023-08-11T15:29:19.880 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.devname 2023-08-11T15:29:19.880 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.order 2023-08-11T15:29:19.880 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.softdep 2023-08-11T15:29:19.881 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols 2023-08-11T15:29:19.881 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols.bin 2023-08-11T15:29:19.882 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:19.883 DEBUG:teuthology.orchestra.run.smithi027:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-08-11T15:29:19.907 INFO:teuthology.orchestra.run.smithi027.stdout:postinstall scriptlet (using /bin/sh): 2023-08-11T15:29:19.908 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d -a -r /boot/System.map-6.5.0-rc4-ga7fb1265323d ]; then 2023-08-11T15:29:19.908 INFO:teuthology.orchestra.run.smithi027.stdout:cp /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:29:19.909 INFO:teuthology.orchestra.run.smithi027.stdout:cp /boot/System.map-6.5.0-rc4-ga7fb1265323d /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:29:19.909 INFO:teuthology.orchestra.run.smithi027.stdout:rm -f /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:19.909 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/installkernel 6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:29:19.909 INFO:teuthology.orchestra.run.smithi027.stdout:rm -f /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:29:19.909 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-08-11T15:29:19.910 INFO:teuthology.orchestra.run.smithi027.stdout:preuninstall scriptlet (using /bin/sh): 2023-08-11T15:29:19.910 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-08-11T15:29:19.910 INFO:teuthology.orchestra.run.smithi027.stdout:new-kernel-pkg --remove 6.5.0-rc4-ga7fb1265323d --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc4-ga7fb1265323d.img 2023-08-11T15:29:19.910 INFO:teuthology.orchestra.run.smithi027.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-08-11T15:29:19.910 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-install remove 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:19.910 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-08-11T15:29:19.910 INFO:teuthology.orchestra.run.smithi027.stdout:postuninstall scriptlet (using /bin/sh): 2023-08-11T15:29:19.911 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/update-bootloader ]; then 2023-08-11T15:29:19.911 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/update-bootloader --remove 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:19.911 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-08-11T15:29:19.912 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2023-08-11T15:29:20.003 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2023-08-11T15:29:20.003 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2023-08-11T15:29:20.003 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2023-08-11T15:29:20.004 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 53.el9 2023-08-11T15:29:20.004 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2023-08-11T15:29:20.004 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Mon 30 Jan 2023 10:38:40 AM UTC 2023-08-11T15:29:20.004 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2023-08-11T15:29:20.004 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8551923 2023-08-11T15:29:20.004 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2023-08-11T15:29:20.004 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Wed 18 Jan 2023 04:40:07 PM UTC, Key ID 05b555b38483c65d 2023-08-11T15:29:20.005 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-53.el9.src.rpm 2023-08-11T15:29:20.005 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Fri 13 Jan 2023 09:07:28 PM UTC 2023-08-11T15:29:20.005 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-08-11T15:29:20.005 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2023-08-11T15:29:20.005 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2023-08-11T15:29:20.005 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-11T15:29:20.005 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2023-08-11T15:29:20.006 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2023-08-11T15:29:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:29:20.006 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-11T15:29:20.006 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-11T15:29:20.006 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-11T15:29:20.006 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2023-08-11T15:29:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:29:20.007 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2023-08-11T15:29:20.008 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-11T15:29:20.008 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:20.008 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-11T15:29:20.574 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2023-08-11T15:29:21.947 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-08-11T15:29:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:done 2023-08-11T15:29:22.020 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2023-08-11T15:29:22.036 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.rIOjx4oLjN 2023-08-11T15:29:22.037 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rIOjx4oLjN 2023-08-11T15:29:22.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.rIOjx4oLjN 2023-08-11T15:29:22.260 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.rIOjx4oLjN is 6KB 2023-08-11T15:29:22.282 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.rIOjx4oLjN 2023-08-11T15:29:22.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2023-08-11T15:29:22.367 INFO:teuthology.orchestra.run.smithi027.stdout:e13cba1d25f14c51a1d04da8db823a4e-0-rescue.conf 2023-08-11T15:29:22.368 INFO:teuthology.orchestra.run.smithi027.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-239.el9.x86_64.conf 2023-08-11T15:29:22.368 INFO:teuthology.orchestra.run.smithi027.stdout:e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc4-ga7fb1265323d.conf 2023-08-11T15:29:22.369 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:22.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2023-08-11T15:29:30.088 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-08-11T15:29:30.088 DEBUG:teuthology.orchestra.run.smithi119:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-08-11T15:29:30.177 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:30.178 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/config-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:30.178 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/vmlinuz-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:30.178 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:30.178 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel 2023-08-11T15:29:30.178 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch 2023-08-11T15:29:30.178 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86 2023-08-11T15:29:30.179 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto 2023-08-11T15:29:30.179 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-08-11T15:29:30.179 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-08-11T15:29:30.179 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-08-11T15:29:30.179 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-08-11T15:29:30.179 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-08-11T15:29:30.180 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-08-11T15:29:30.180 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-08-11T15:29:30.180 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-08-11T15:29:30.180 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-08-11T15:29:30.180 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-08-11T15:29:30.181 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-08-11T15:29:30.181 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-08-11T15:29:30.181 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-08-11T15:29:30.181 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-08-11T15:29:30.181 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-08-11T15:29:30.181 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-08-11T15:29:30.181 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-08-11T15:29:30.182 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-08-11T15:29:30.182 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-08-11T15:29:30.182 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-08-11T15:29:30.182 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel 2023-08-11T15:29:30.182 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu 2023-08-11T15:29:30.182 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce 2023-08-11T15:29:30.182 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-08-11T15:29:30.183 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm 2023-08-11T15:29:30.183 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-amd.ko 2023-08-11T15:29:30.183 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-intel.ko 2023-08-11T15:29:30.183 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm.ko 2023-08-11T15:29:30.183 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto 2023-08-11T15:29:30.183 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ansi_cprng.ko 2023-08-11T15:29:30.184 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/anubis.ko 2023-08-11T15:29:30.184 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/arc4.ko 2023-08-11T15:29:30.184 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx 2023-08-11T15:29:30.184 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_memcpy.ko 2023-08-11T15:29:30.185 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_pq.ko 2023-08-11T15:29:30.185 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-08-11T15:29:30.186 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_tx.ko 2023-08-11T15:29:30.186 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_xor.ko 2023-08-11T15:29:30.186 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/raid6test.ko 2023-08-11T15:29:30.186 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authenc.ko 2023-08-11T15:29:30.187 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authencesn.ko 2023-08-11T15:29:30.187 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blake2b_generic.ko 2023-08-11T15:29:30.187 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_common.ko 2023-08-11T15:29:30.188 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_generic.ko 2023-08-11T15:29:30.189 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/camellia_generic.ko 2023-08-11T15:29:30.189 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast5_generic.ko 2023-08-11T15:29:30.189 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast6_generic.ko 2023-08-11T15:29:30.189 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast_common.ko 2023-08-11T15:29:30.190 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ccm.ko 2023-08-11T15:29:30.190 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cmac.ko 2023-08-11T15:29:30.191 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crc32_generic.ko 2023-08-11T15:29:30.191 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_engine.ko 2023-08-11T15:29:30.192 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_user.ko 2023-08-11T15:29:30.192 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cts.ko 2023-08-11T15:29:30.192 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/des_generic.ko 2023-08-11T15:29:30.192 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/dh_generic.ko 2023-08-11T15:29:30.193 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/echainiv.ko 2023-08-11T15:29:30.193 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/essiv.ko 2023-08-11T15:29:30.193 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/fcrypt.ko 2023-08-11T15:29:30.194 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/khazad.ko 2023-08-11T15:29:30.194 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4.ko 2023-08-11T15:29:30.194 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4hc.ko 2023-08-11T15:29:30.194 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/md4.ko 2023-08-11T15:29:30.194 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/michael_mic.ko 2023-08-11T15:29:30.194 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcbc.ko 2023-08-11T15:29:30.195 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcrypt.ko 2023-08-11T15:29:30.195 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/rmd160.ko 2023-08-11T15:29:30.195 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/seed.ko 2023-08-11T15:29:30.195 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/serpent_generic.ko 2023-08-11T15:29:30.195 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tcrypt.ko 2023-08-11T15:29:30.196 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tea.ko 2023-08-11T15:29:30.196 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_common.ko 2023-08-11T15:29:30.196 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_generic.ko 2023-08-11T15:29:30.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/vmac.ko 2023-08-11T15:29:30.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/wp512.ko 2023-08-11T15:29:30.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xcbc.ko 2023-08-11T15:29:30.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xor.ko 2023-08-11T15:29:30.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xxhash_generic.ko 2023-08-11T15:29:30.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers 2023-08-11T15:29:30.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi 2023-08-11T15:29:30.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_ipmi.ko 2023-08-11T15:29:30.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_pad.ko 2023-08-11T15:29:30.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/custom_method.ko 2023-08-11T15:29:30.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/ec_sys.ko 2023-08-11T15:29:30.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/platform_profile.ko 2023-08-11T15:29:30.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbs.ko 2023-08-11T15:29:30.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbshc.ko 2023-08-11T15:29:30.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/video.ko 2023-08-11T15:29:30.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata 2023-08-11T15:29:30.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/acard-ahci.ko 2023-08-11T15:29:30.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ahci_platform.ko 2023-08-11T15:29:30.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ata_generic.ko 2023-08-11T15:29:30.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/libahci_platform.ko 2023-08-11T15:29:30.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_acpi.ko 2023-08-11T15:29:30.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ali.ko 2023-08-11T15:29:30.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_amd.ko 2023-08-11T15:29:30.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_artop.ko 2023-08-11T15:29:30.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atiixp.ko 2023-08-11T15:29:30.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atp867x.ko 2023-08-11T15:29:30.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd640.ko 2023-08-11T15:29:30.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd64x.ko 2023-08-11T15:29:30.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cypress.ko 2023-08-11T15:29:30.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_efar.ko 2023-08-11T15:29:30.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt366.ko 2023-08-11T15:29:30.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt37x.ko 2023-08-11T15:29:30.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-08-11T15:29:30.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x3.ko 2023-08-11T15:29:30.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it8213.ko 2023-08-11T15:29:30.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it821x.ko 2023-08-11T15:29:30.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_jmicron.ko 2023-08-11T15:29:30.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_marvell.ko 2023-08-11T15:29:30.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_mpiix.ko 2023-08-11T15:29:30.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_netcell.ko 2023-08-11T15:29:30.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ninja32.ko 2023-08-11T15:29:30.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87410.ko 2023-08-11T15:29:30.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87415.ko 2023-08-11T15:29:30.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_oldpiix.ko 2023-08-11T15:29:30.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_opti.ko 2023-08-11T15:29:30.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_optidma.ko 2023-08-11T15:29:30.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pcmcia.ko 2023-08-11T15:29:30.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc2027x.ko 2023-08-11T15:29:30.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-08-11T15:29:30.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_piccolo.ko 2023-08-11T15:29:30.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_rdc.ko 2023-08-11T15:29:30.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sch.ko 2023-08-11T15:29:30.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_serverworks.ko 2023-08-11T15:29:30.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sil680.ko 2023-08-11T15:29:30.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sis.ko 2023-08-11T15:29:30.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sl82c105.ko 2023-08-11T15:29:30.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_triflex.ko 2023-08-11T15:29:30.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_via.ko 2023-08-11T15:29:30.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pdc_adma.ko 2023-08-11T15:29:30.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_inic162x.ko 2023-08-11T15:29:30.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_mv.ko 2023-08-11T15:29:30.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_nv.ko 2023-08-11T15:29:30.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_promise.ko 2023-08-11T15:29:30.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_qstor.ko 2023-08-11T15:29:30.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil.ko 2023-08-11T15:29:30.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil24.ko 2023-08-11T15:29:30.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sis.ko 2023-08-11T15:29:30.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_svw.ko 2023-08-11T15:29:30.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sx4.ko 2023-08-11T15:29:30.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_uli.ko 2023-08-11T15:29:30.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_via.ko 2023-08-11T15:29:30.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_vsc.ko 2023-08-11T15:29:30.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay 2023-08-11T15:29:30.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay/ks0108.ko 2023-08-11T15:29:30.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base 2023-08-11T15:29:30.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap 2023-08-11T15:29:30.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-08-11T15:29:30.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-08-11T15:29:30.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma 2023-08-11T15:29:30.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma/bcma.ko 2023-08-11T15:29:30.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block 2023-08-11T15:29:30.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe 2023-08-11T15:29:30.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe/aoe.ko 2023-08-11T15:29:30.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/brd.ko 2023-08-11T15:29:30.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd 2023-08-11T15:29:30.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd/drbd.ko 2023-08-11T15:29:30.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/floppy.ko 2023-08-11T15:29:30.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/loop.ko 2023-08-11T15:29:30.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx 2023-08-11T15:29:30.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-08-11T15:29:30.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/nbd.ko 2023-08-11T15:29:30.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk 2023-08-11T15:29:30.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk/null_blk.ko 2023-08-11T15:29:30.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/pktcdvd.ko 2023-08-11T15:29:30.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/rbd.ko 2023-08-11T15:29:30.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/virtio_blk.ko 2023-08-11T15:29:30.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback 2023-08-11T15:29:30.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-08-11T15:29:30.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkfront.ko 2023-08-11T15:29:30.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram 2023-08-11T15:29:30.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram/zram.ko 2023-08-11T15:29:30.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char 2023-08-11T15:29:30.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hangcheck-timer.ko 2023-08-11T15:29:30.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random 2023-08-11T15:29:30.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/amd-rng.ko 2023-08-11T15:29:30.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/intel-rng.ko 2023-08-11T15:29:30.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-08-11T15:29:30.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/via-rng.ko 2023-08-11T15:29:30.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-08-11T15:29:30.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi 2023-08-11T15:29:30.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-08-11T15:29:30.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-08-11T15:29:30.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-08-11T15:29:30.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-08-11T15:29:30.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-08-11T15:29:30.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-08-11T15:29:30.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/lp.ko 2023-08-11T15:29:30.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave 2023-08-11T15:29:30.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave/mwave.ko 2023-08-11T15:29:30.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ppdev.ko 2023-08-11T15:29:30.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tlclk.ko 2023-08-11T15:29:30.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm 2023-08-11T15:29:30.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm.ko 2023-08-11T15:29:30.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-08-11T15:29:30.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-08-11T15:29:30.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-08-11T15:29:30.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis.ko 2023-08-11T15:29:30.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-08-11T15:29:30.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/uv_mmtimer.ko 2023-08-11T15:29:30.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/virtio_console.ko 2023-08-11T15:29:30.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq 2023-08-11T15:29:30.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-08-11T15:29:30.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-08-11T15:29:30.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-08-11T15:29:30.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-08-11T15:29:30.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/powernow-k8.ko 2023-08-11T15:29:30.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-08-11T15:29:30.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto 2023-08-11T15:29:30.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp 2023-08-11T15:29:30.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-08-11T15:29:30.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp.ko 2023-08-11T15:29:30.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel 2023-08-11T15:29:30.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat 2023-08-11T15:29:30.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common 2023-08-11T15:29:30.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-08-11T15:29:30.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-08-11T15:29:30.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-08-11T15:29:30.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-aes.ko 2023-08-11T15:29:30.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-sha.ko 2023-08-11T15:29:30.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio 2023-08-11T15:29:30.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-08-11T15:29:30.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca 2023-08-11T15:29:30.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca/dca.ko 2023-08-11T15:29:30.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq 2023-08-11T15:29:30.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-08-11T15:29:30.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma 2023-08-11T15:29:30.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw 2023-08-11T15:29:30.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw/dw_dmac.ko 2023-08-11T15:29:30.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat 2023-08-11T15:29:30.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat/ioatdma.ko 2023-08-11T15:29:30.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac 2023-08-11T15:29:30.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/amd64_edac.ko 2023-08-11T15:29:30.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/e752x_edac.ko 2023-08-11T15:29:30.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/edac_mce_amd.ko 2023-08-11T15:29:30.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3000_edac.ko 2023-08-11T15:29:30.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3200_edac.ko 2023-08-11T15:29:30.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5100_edac.ko 2023-08-11T15:29:30.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5400_edac.ko 2023-08-11T15:29:30.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7300_edac.ko 2023-08-11T15:29:30.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7core_edac.ko 2023-08-11T15:29:30.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i82975x_edac.ko 2023-08-11T15:29:30.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/ie31200_edac.ko 2023-08-11T15:29:30.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/sb_edac.ko 2023-08-11T15:29:30.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/x38_edac.ko 2023-08-11T15:29:30.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware 2023-08-11T15:29:30.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/edd.ko 2023-08-11T15:29:30.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/iscsi_ibft.ko 2023-08-11T15:29:30.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio 2023-08-11T15:29:30.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-ich.ko 2023-08-11T15:29:30.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-viperboard.ko 2023-08-11T15:29:30.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid 2023-08-11T15:29:30.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-appleir.ko 2023-08-11T15:29:30.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-aureal.ko 2023-08-11T15:29:30.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-axff.ko 2023-08-11T15:29:30.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-dr.ko 2023-08-11T15:29:30.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elecom.ko 2023-08-11T15:29:30.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elo.ko 2023-08-11T15:29:30.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-emsff.ko 2023-08-11T15:29:30.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gaff.ko 2023-08-11T15:29:30.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gt683r.ko 2023-08-11T15:29:30.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gyration.ko 2023-08-11T15:29:30.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-08-11T15:29:30.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-08-11T15:29:30.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtekff.ko 2023-08-11T15:29:30.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-hyperv.ko 2023-08-11T15:29:30.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-icade.ko 2023-08-11T15:29:30.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-keytouch.ko 2023-08-11T15:29:30.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-kye.ko 2023-08-11T15:29:30.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lcpower.ko 2023-08-11T15:29:30.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-led.ko 2023-08-11T15:29:30.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lenovo.ko 2023-08-11T15:29:30.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-dj.ko 2023-08-11T15:29:30.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-08-11T15:29:30.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-multitouch.ko 2023-08-11T15:29:30.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-ortek.ko 2023-08-11T15:29:30.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-penmount.ko 2023-08-11T15:29:30.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-petalynx.ko 2023-08-11T15:29:30.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-picolcd.ko 2023-08-11T15:29:30.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-pl.ko 2023-08-11T15:29:30.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-plantronics.ko 2023-08-11T15:29:30.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-primax.ko 2023-08-11T15:29:30.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-rmi.ko 2023-08-11T15:29:30.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-08-11T15:29:30.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-common.ko 2023-08-11T15:29:30.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-isku.ko 2023-08-11T15:29:30.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kone.ko 2023-08-11T15:29:30.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-08-11T15:29:30.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-08-11T15:29:30.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-08-11T15:29:30.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-lua.ko 2023-08-11T15:29:30.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-08-11T15:29:30.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-08-11T15:29:30.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-savu.ko 2023-08-11T15:29:30.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat.ko 2023-08-11T15:29:30.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-saitek.ko 2023-08-11T15:29:30.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-samsung.ko 2023-08-11T15:29:30.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sensor-hub.ko 2023-08-11T15:29:30.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sjoy.ko 2023-08-11T15:29:30.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sony.ko 2023-08-11T15:29:30.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-speedlink.ko 2023-08-11T15:29:30.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-steelseries.ko 2023-08-11T15:29:30.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sunplus.ko 2023-08-11T15:29:30.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-thrustmaster.ko 2023-08-11T15:29:30.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tivo.ko 2023-08-11T15:29:30.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tmff.ko 2023-08-11T15:29:30.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-topseed.ko 2023-08-11T15:29:30.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-twinhan.ko 2023-08-11T15:29:30.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-uclogic.ko 2023-08-11T15:29:30.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-waltop.ko 2023-08-11T15:29:30.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-wiimote.ko 2023-08-11T15:29:30.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-xinmo.ko 2023-08-11T15:29:30.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zpff.ko 2023-08-11T15:29:30.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zydacron.ko 2023-08-11T15:29:30.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/uhid.ko 2023-08-11T15:29:30.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/wacom.ko 2023-08-11T15:29:30.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv 2023-08-11T15:29:30.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_balloon.ko 2023-08-11T15:29:30.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_utils.ko 2023-08-11T15:29:30.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_vmbus.ko 2023-08-11T15:29:30.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon 2023-08-11T15:29:30.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru.ko 2023-08-11T15:29:30.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru3.ko 2023-08-11T15:29:30.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-08-11T15:29:30.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7414.ko 2023-08-11T15:29:30.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7418.ko 2023-08-11T15:29:30.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adc128d818.ko 2023-08-11T15:29:30.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1025.ko 2023-08-11T15:29:30.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1026.ko 2023-08-11T15:29:30.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1029.ko 2023-08-11T15:29:30.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1031.ko 2023-08-11T15:29:30.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm9240.ko 2023-08-11T15:29:30.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ads7828.ko 2023-08-11T15:29:30.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7410.ko 2023-08-11T15:29:30.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7411.ko 2023-08-11T15:29:30.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7462.ko 2023-08-11T15:29:30.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7470.ko 2023-08-11T15:29:30.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7475.ko 2023-08-11T15:29:30.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7x10.ko 2023-08-11T15:29:30.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/amc6821.ko 2023-08-11T15:29:30.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/applesmc.ko 2023-08-11T15:29:30.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asb100.ko 2023-08-11T15:29:30.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asc7621.ko 2023-08-11T15:29:30.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asus_atk0110.ko 2023-08-11T15:29:30.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/atxp1.ko 2023-08-11T15:29:30.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/coretemp.ko 2023-08-11T15:29:30.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-08-11T15:29:30.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dme1737.ko 2023-08-11T15:29:30.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds1621.ko 2023-08-11T15:29:30.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds620.ko 2023-08-11T15:29:30.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc1403.ko 2023-08-11T15:29:30.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc6w201.ko 2023-08-11T15:29:30.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71805f.ko 2023-08-11T15:29:30.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71882fg.ko 2023-08-11T15:29:30.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f75375s.ko 2023-08-11T15:29:30.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fam15h_power.ko 2023-08-11T15:29:30.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fschmd.ko 2023-08-11T15:29:30.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g760a.ko 2023-08-11T15:29:30.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g762.ko 2023-08-11T15:29:30.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl518sm.ko 2023-08-11T15:29:30.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl520sm.ko 2023-08-11T15:29:30.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/hwmon-vid.ko 2023-08-11T15:29:30.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5500_temp.ko 2023-08-11T15:29:30.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5k_amb.ko 2023-08-11T15:29:30.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmaem.ko 2023-08-11T15:29:30.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmpex.ko 2023-08-11T15:29:30.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina209.ko 2023-08-11T15:29:30.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina2xx.ko 2023-08-11T15:29:30.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/it87.ko 2023-08-11T15:29:30.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k10temp.ko 2023-08-11T15:29:30.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k8temp.ko 2023-08-11T15:29:30.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lineage-pem.ko 2023-08-11T15:29:30.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm63.ko 2023-08-11T15:29:30.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm73.ko 2023-08-11T15:29:30.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm75.ko 2023-08-11T15:29:30.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm77.ko 2023-08-11T15:29:30.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm78.ko 2023-08-11T15:29:30.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm80.ko 2023-08-11T15:29:30.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm83.ko 2023-08-11T15:29:30.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm85.ko 2023-08-11T15:29:30.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm87.ko 2023-08-11T15:29:30.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm90.ko 2023-08-11T15:29:30.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm92.ko 2023-08-11T15:29:30.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm93.ko 2023-08-11T15:29:30.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95234.ko 2023-08-11T15:29:30.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95241.ko 2023-08-11T15:29:30.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95245.ko 2023-08-11T15:29:30.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc2945.ko 2023-08-11T15:29:30.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4151.ko 2023-08-11T15:29:30.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4215.ko 2023-08-11T15:29:30.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4222.ko 2023-08-11T15:29:30.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4245.ko 2023-08-11T15:29:30.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4260.ko 2023-08-11T15:29:30.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4261.ko 2023-08-11T15:29:30.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max16065.ko 2023-08-11T15:29:30.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1619.ko 2023-08-11T15:29:30.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1668.ko 2023-08-11T15:29:30.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max197.ko 2023-08-11T15:29:30.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6639.ko 2023-08-11T15:29:30.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6650.ko 2023-08-11T15:29:30.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6697.ko 2023-08-11T15:29:30.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/mcp3021.ko 2023-08-11T15:29:30.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6683.ko 2023-08-11T15:29:30.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775-core.ko 2023-08-11T15:29:30.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775.ko 2023-08-11T15:29:30.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct7802.ko 2023-08-11T15:29:30.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87360.ko 2023-08-11T15:29:30.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87427.ko 2023-08-11T15:29:30.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pcf8591.ko 2023-08-11T15:29:30.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus 2023-08-11T15:29:30.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-08-11T15:29:30.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-08-11T15:29:30.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-08-11T15:29:30.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-08-11T15:29:30.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-08-11T15:29:30.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-08-11T15:29:30.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-08-11T15:29:30.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-08-11T15:29:30.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-08-11T15:29:30.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-08-11T15:29:30.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-08-11T15:29:30.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-08-11T15:29:30.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/powr1220.ko 2023-08-11T15:29:30.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5627.ko 2023-08-11T15:29:30.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5636.ko 2023-08-11T15:29:30.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch56xx-common.ko 2023-08-11T15:29:30.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht15.ko 2023-08-11T15:29:30.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht21.ko 2023-08-11T15:29:30.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/shtc1.ko 2023-08-11T15:29:30.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sis5595.ko 2023-08-11T15:29:30.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47b397.ko 2023-08-11T15:29:30.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m1.ko 2023-08-11T15:29:30.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m192.ko 2023-08-11T15:29:30.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/thmc50.ko 2023-08-11T15:29:30.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp102.ko 2023-08-11T15:29:30.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp103.ko 2023-08-11T15:29:30.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp401.ko 2023-08-11T15:29:30.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp421.ko 2023-08-11T15:29:30.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via-cputemp.ko 2023-08-11T15:29:30.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via686a.ko 2023-08-11T15:29:30.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt1211.ko 2023-08-11T15:29:30.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt8231.ko 2023-08-11T15:29:30.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627ehf.ko 2023-08-11T15:29:30.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627hf.ko 2023-08-11T15:29:30.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83781d.ko 2023-08-11T15:29:30.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83791d.ko 2023-08-11T15:29:30.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83792d.ko 2023-08-11T15:29:30.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83793.ko 2023-08-11T15:29:30.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83795.ko 2023-08-11T15:29:30.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l785ts.ko 2023-08-11T15:29:30.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l786ng.ko 2023-08-11T15:29:30.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c 2023-08-11T15:29:30.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos 2023-08-11T15:29:30.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-08-11T15:29:30.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-08-11T15:29:30.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses 2023-08-11T15:29:30.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-08-11T15:29:30.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-08-11T15:29:30.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-08-11T15:29:30.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-08-11T15:29:30.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-08-11T15:29:30.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-08-11T15:29:30.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-08-11T15:29:30.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-08-11T15:29:30.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-08-11T15:29:30.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-08-11T15:29:30.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-08-11T15:29:30.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-08-11T15:29:30.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-08-11T15:29:30.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-08-11T15:29:30.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-08-11T15:29:30.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-08-11T15:29:30.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-08-11T15:29:30.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-08-11T15:29:30.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-08-11T15:29:30.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-08-11T15:29:30.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-via.ko 2023-08-11T15:29:30.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-08-11T15:29:30.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-08-11T15:29:30.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-dev.ko 2023-08-11T15:29:30.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-mux.ko 2023-08-11T15:29:30.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-08-11T15:29:30.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-smbus.ko 2023-08-11T15:29:30.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-stub.ko 2023-08-11T15:29:30.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband 2023-08-11T15:29:30.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core 2023-08-11T15:29:30.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_cm.ko 2023-08-11T15:29:30.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_core.ko 2023-08-11T15:29:30.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_umad.ko 2023-08-11T15:29:30.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-08-11T15:29:30.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/iw_cm.ko 2023-08-11T15:29:30.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-08-11T15:29:30.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-08-11T15:29:30.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw 2023-08-11T15:29:30.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4 2023-08-11T15:29:30.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-08-11T15:29:30.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4 2023-08-11T15:29:30.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-08-11T15:29:30.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5 2023-08-11T15:29:30.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-08-11T15:29:30.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca 2023-08-11T15:29:30.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-08-11T15:29:30.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp 2023-08-11T15:29:30.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib 2023-08-11T15:29:30.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-08-11T15:29:30.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser 2023-08-11T15:29:30.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-08-11T15:29:30.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert 2023-08-11T15:29:30.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-08-11T15:29:30.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp 2023-08-11T15:29:30.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-08-11T15:29:30.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt 2023-08-11T15:29:30.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-08-11T15:29:30.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input 2023-08-11T15:29:30.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport 2023-08-11T15:29:30.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-08-11T15:29:30.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/fm801-gp.ko 2023-08-11T15:29:30.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/gameport.ko 2023-08-11T15:29:30.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/lightning.ko 2023-08-11T15:29:30.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/ns558.ko 2023-08-11T15:29:30.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/joydev.ko 2023-08-11T15:29:30.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard 2023-08-11T15:29:30.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-08-11T15:29:30.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse 2023-08-11T15:29:30.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/appletouch.ko 2023-08-11T15:29:30.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/bcm5974.ko 2023-08-11T15:29:30.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/cyapatp.ko 2023-08-11T15:29:30.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/elan_i2c.ko 2023-08-11T15:29:30.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/sermouse.ko 2023-08-11T15:29:30.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-08-11T15:29:30.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-08-11T15:29:30.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-08-11T15:29:30.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4 2023-08-11T15:29:30.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4/rmi_core.ko 2023-08-11T15:29:30.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio 2023-08-11T15:29:30.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/altera_ps2.ko 2023-08-11T15:29:30.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/arc_ps2.ko 2023-08-11T15:29:30.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-08-11T15:29:30.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/serio_raw.ko 2023-08-11T15:29:30.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/sparse-keymap.ko 2023-08-11T15:29:30.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu 2023-08-11T15:29:30.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd 2023-08-11T15:29:30.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-08-11T15:29:30.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds 2023-08-11T15:29:30.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-blinkm.ko 2023-08-11T15:29:30.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lm3530.ko 2023-08-11T15:29:30.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lp3944.ko 2023-08-11T15:29:30.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-ss4200.ko 2023-08-11T15:29:30.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger 2023-08-11T15:29:30.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-08-11T15:29:30.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-08-11T15:29:30.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-08-11T15:29:30.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-08-11T15:29:30.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-08-11T15:29:30.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-08-11T15:29:30.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-08-11T15:29:30.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-08-11T15:29:30.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md 2023-08-11T15:29:30.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache 2023-08-11T15:29:30.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache/bcache.ko 2023-08-11T15:29:30.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-bio-prison.ko 2023-08-11T15:29:30.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache-smq.ko 2023-08-11T15:29:30.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache.ko 2023-08-11T15:29:30.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-crypt.ko 2023-08-11T15:29:30.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-delay.ko 2023-08-11T15:29:30.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-flakey.ko 2023-08-11T15:29:30.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-log-userspace.ko 2023-08-11T15:29:30.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-multipath.ko 2023-08-11T15:29:30.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-queue-length.ko 2023-08-11T15:29:30.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-raid.ko 2023-08-11T15:29:30.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-round-robin.ko 2023-08-11T15:29:30.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-service-time.ko 2023-08-11T15:29:30.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-switch.ko 2023-08-11T15:29:30.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-thin-pool.ko 2023-08-11T15:29:30.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-verity.ko 2023-08-11T15:29:30.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/faulty.ko 2023-08-11T15:29:30.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/linear.ko 2023-08-11T15:29:30.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/multipath.ko 2023-08-11T15:29:30.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data 2023-08-11T15:29:30.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-08-11T15:29:30.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid0.ko 2023-08-11T15:29:30.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid1.ko 2023-08-11T15:29:30.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid10.ko 2023-08-11T15:29:30.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid456.ko 2023-08-11T15:29:30.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message 2023-08-11T15:29:30.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion 2023-08-11T15:29:30.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptbase.ko 2023-08-11T15:29:30.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptctl.ko 2023-08-11T15:29:30.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptfc.ko 2023-08-11T15:29:30.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptlan.ko 2023-08-11T15:29:30.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptsas.ko 2023-08-11T15:29:30.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptscsih.ko 2023-08-11T15:29:30.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptspi.ko 2023-08-11T15:29:30.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd 2023-08-11T15:29:30.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_ich.ko 2023-08-11T15:29:30.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_sch.ko 2023-08-11T15:29:30.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/mfd-core.ko 2023-08-11T15:29:30.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/sm501.ko 2023-08-11T15:29:30.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/viperboard.ko 2023-08-11T15:29:30.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/vx855.ko 2023-08-11T15:29:30.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/wl1273-core.ko 2023-08-11T15:29:30.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc 2023-08-11T15:29:30.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl 2023-08-11T15:29:30.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-08-11T15:29:30.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds9802als.ko 2023-08-11T15:29:30.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds990x.ko 2023-08-11T15:29:30.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/bh1770glc.ko 2023-08-11T15:29:30.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710 2023-08-11T15:29:30.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710/cb710.ko 2023-08-11T15:29:30.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom 2023-08-11T15:29:30.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/at24.ko 2023-08-11T15:29:30.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom.ko 2023-08-11T15:29:30.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-08-11T15:29:30.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/max6875.ko 2023-08-11T15:29:30.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/enclosure.ko 2023-08-11T15:29:30.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/hpilo.ko 2023-08-11T15:29:30.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29003.ko 2023-08-11T15:29:30.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29020.ko 2023-08-11T15:29:30.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d 2023-08-11T15:29:30.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-08-11T15:29:30.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-08-11T15:29:30.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei 2023-08-11T15:29:30.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-me.ko 2023-08-11T15:29:30.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-txe.ko 2023-08-11T15:29:30.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei.ko 2023-08-11T15:29:30.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru 2023-08-11T15:29:30.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru/gru.ko 2023-08-11T15:29:30.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp 2023-08-11T15:29:30.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xp.ko 2023-08-11T15:29:30.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-08-11T15:29:30.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-08-11T15:29:30.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_7xx1.ko 2023-08-11T15:29:30.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_core.ko 2023-08-11T15:29:30.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tsl2550.ko 2023-08-11T15:29:30.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_balloon.ko 2023-08-11T15:29:30.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci 2023-08-11T15:29:30.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-08-11T15:29:30.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net 2023-08-11T15:29:30.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding 2023-08-11T15:29:30.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding/bonding.ko 2023-08-11T15:29:30.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa 2023-08-11T15:29:30.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53 2023-08-11T15:29:30.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-08-11T15:29:30.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-08-11T15:29:30.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6060.ko 2023-08-11T15:29:30.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx 2023-08-11T15:29:30.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-08-11T15:29:30.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dummy.ko 2023-08-11T15:29:30.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/eql.ko 2023-08-11T15:29:30.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet 2023-08-11T15:29:30.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com 2023-08-11T15:29:30.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-08-11T15:29:30.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-08-11T15:29:30.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-08-11T15:29:30.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-08-11T15:29:30.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390 2023-08-11T15:29:30.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/8390.ko 2023-08-11T15:29:30.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-08-11T15:29:30.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-08-11T15:29:30.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-08-11T15:29:30.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec 2023-08-11T15:29:30.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-08-11T15:29:30.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere 2023-08-11T15:29:30.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-08-11T15:29:30.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon 2023-08-11T15:29:30.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-08-11T15:29:30.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera 2023-08-11T15:29:30.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-08-11T15:29:30.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd 2023-08-11T15:29:30.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-08-11T15:29:30.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-08-11T15:29:30.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-08-11T15:29:30.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros 2023-08-11T15:29:30.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx 2023-08-11T15:29:30.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-08-11T15:29:30.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c 2023-08-11T15:29:30.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-08-11T15:29:30.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e 2023-08-11T15:29:30.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-08-11T15:29:30.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx 2023-08-11T15:29:30.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-08-11T15:29:30.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-08-11T15:29:30.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom 2023-08-11T15:29:30.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-08-11T15:29:30.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-08-11T15:29:30.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-08-11T15:29:30.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-08-11T15:29:30.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-08-11T15:29:30.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet 2023-08-11T15:29:30.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-08-11T15:29:30.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-08-11T15:29:30.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade 2023-08-11T15:29:30.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna 2023-08-11T15:29:30.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-08-11T15:29:30.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio 2023-08-11T15:29:30.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-08-11T15:29:30.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-08-11T15:29:30.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-08-11T15:29:30.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-08-11T15:29:30.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-08-11T15:29:30.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-08-11T15:29:30.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-08-11T15:29:30.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-08-11T15:29:30.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-08-11T15:29:30.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-08-11T15:29:30.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco 2023-08-11T15:29:30.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic 2023-08-11T15:29:30.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-08-11T15:29:30.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec 2023-08-11T15:29:30.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip 2023-08-11T15:29:30.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-08-11T15:29:30.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-08-11T15:29:30.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-08-11T15:29:30.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-08-11T15:29:30.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-08-11T15:29:30.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-08-11T15:29:30.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink 2023-08-11T15:29:30.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-08-11T15:29:30.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-08-11T15:29:30.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dnet.ko 2023-08-11T15:29:30.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex 2023-08-11T15:29:30.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet 2023-08-11T15:29:30.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-08-11T15:29:30.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ethoc.ko 2023-08-11T15:29:30.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/fealnx.ko 2023-08-11T15:29:30.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel 2023-08-11T15:29:30.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e100.ko 2023-08-11T15:29:30.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000 2023-08-11T15:29:30.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-08-11T15:29:30.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e 2023-08-11T15:29:30.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-08-11T15:29:30.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k 2023-08-11T15:29:30.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-08-11T15:29:30.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e 2023-08-11T15:29:30.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-08-11T15:29:30.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf 2023-08-11T15:29:30.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-08-11T15:29:30.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb 2023-08-11T15:29:30.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-08-11T15:29:30.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf 2023-08-11T15:29:30.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-08-11T15:29:30.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe 2023-08-11T15:29:30.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-08-11T15:29:30.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-08-11T15:29:30.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-08-11T15:29:30.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/jme.ko 2023-08-11T15:29:30.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell 2023-08-11T15:29:30.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-08-11T15:29:30.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-08-11T15:29:30.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-08-11T15:29:30.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox 2023-08-11T15:29:30.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-08-11T15:29:30.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-08-11T15:29:30.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-08-11T15:29:30.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-08-11T15:29:30.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-08-11T15:29:30.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-08-11T15:29:30.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel 2023-08-11T15:29:30.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-08-11T15:29:30.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom 2023-08-11T15:29:30.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-08-11T15:29:30.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-08-11T15:29:30.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi 2023-08-11T15:29:30.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-08-11T15:29:30.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-08-11T15:29:30.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion 2023-08-11T15:29:30.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-08-11T15:29:30.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia 2023-08-11T15:29:30.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-08-11T15:29:30.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines 2023-08-11T15:29:30.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-08-11T15:29:30.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-08-11T15:29:30.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic 2023-08-11T15:29:30.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen 2023-08-11T15:29:30.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-08-11T15:29:30.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-08-11T15:29:30.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-08-11T15:29:30.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-08-11T15:29:30.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc 2023-08-11T15:29:30.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-08-11T15:29:30.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek 2023-08-11T15:29:30.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-08-11T15:29:30.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-08-11T15:29:30.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-08-11T15:29:30.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-08-11T15:29:30.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker 2023-08-11T15:29:30.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-08-11T15:29:30.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc 2023-08-11T15:29:30.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-08-11T15:29:30.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan 2023-08-11T15:29:30.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-08-11T15:29:30.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis 2023-08-11T15:29:30.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-08-11T15:29:30.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-08-11T15:29:30.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc 2023-08-11T15:29:30.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-08-11T15:29:30.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-08-11T15:29:30.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-08-11T15:29:30.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-08-11T15:29:30.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro 2023-08-11T15:29:30.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-08-11T15:29:30.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-08-11T15:29:30.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-08-11T15:29:30.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun 2023-08-11T15:29:30.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-08-11T15:29:30.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/niu.ko 2023-08-11T15:29:30.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-08-11T15:29:30.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-08-11T15:29:30.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti 2023-08-11T15:29:30.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-08-11T15:29:30.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti 2023-08-11T15:29:30.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-08-11T15:29:30.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via 2023-08-11T15:29:30.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-08-11T15:29:30.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-08-11T15:29:30.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet 2023-08-11T15:29:30.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-08-11T15:29:30.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-08-11T15:29:30.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom 2023-08-11T15:29:30.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-08-11T15:29:30.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/geneve.ko 2023-08-11T15:29:30.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv 2023-08-11T15:29:30.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-08-11T15:29:30.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154 2023-08-11T15:29:30.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154/fakelb.ko 2023-08-11T15:29:30.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ifb.ko 2023-08-11T15:29:30.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan 2023-08-11T15:29:30.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-08-11T15:29:30.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvlan.ko 2023-08-11T15:29:30.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvtap.ko 2023-08-11T15:29:30.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio 2023-08-11T15:29:30.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio.ko 2023-08-11T15:29:30.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-08-11T15:29:30.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-08-11T15:29:30.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-08-11T15:29:30.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mii.ko 2023-08-11T15:29:30.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/net_failover.ko 2023-08-11T15:29:30.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/netconsole.ko 2023-08-11T15:29:30.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/nlmon.ko 2023-08-11T15:29:30.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs 2023-08-11T15:29:30.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-08-11T15:29:30.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-08-11T15:29:30.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy 2023-08-11T15:29:30.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/amd.ko 2023-08-11T15:29:30.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-08-11T15:29:30.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-08-11T15:29:30.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm7xxx.ko 2023-08-11T15:29:30.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm87xx.ko 2023-08-11T15:29:30.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/broadcom.ko 2023-08-11T15:29:30.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/cicada.ko 2023-08-11T15:29:30.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/davicom.ko 2023-08-11T15:29:30.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/dp83640.ko 2023-08-11T15:29:30.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/et1011c.ko 2023-08-11T15:29:30.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/icplus.ko 2023-08-11T15:29:30.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/lxt.ko 2023-08-11T15:29:30.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/marvell.ko 2023-08-11T15:29:30.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/micrel.ko 2023-08-11T15:29:30.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/national.ko 2023-08-11T15:29:30.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/phylink.ko 2023-08-11T15:29:30.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/qsemi.ko 2023-08-11T15:29:30.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/realtek.ko 2023-08-11T15:29:30.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/smsc.ko 2023-08-11T15:29:30.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/ste10Xp.ko 2023-08-11T15:29:30.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/vitesse.ko 2023-08-11T15:29:30.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp 2023-08-11T15:29:30.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/bsd_comp.ko 2023-08-11T15:29:30.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_async.ko 2023-08-11T15:29:30.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-08-11T15:29:30.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_generic.ko 2023-08-11T15:29:30.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-08-11T15:29:30.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-08-11T15:29:30.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppoe.ko 2023-08-11T15:29:30.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppox.ko 2023-08-11T15:29:30.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pptp.ko 2023-08-11T15:29:30.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip 2023-08-11T15:29:30.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slhc.ko 2023-08-11T15:29:30.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slip.ko 2023-08-11T15:29:30.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/sungem_phy.ko 2023-08-11T15:29:30.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tap.ko 2023-08-11T15:29:30.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team 2023-08-11T15:29:30.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team.ko 2023-08-11T15:29:30.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-08-11T15:29:30.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-08-11T15:29:30.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-08-11T15:29:30.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_random.ko 2023-08-11T15:29:30.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-08-11T15:29:30.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tun.ko 2023-08-11T15:29:30.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/veth.ko 2023-08-11T15:29:30.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/virtio_net.ko 2023-08-11T15:29:30.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3 2023-08-11T15:29:30.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-08-11T15:29:30.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan 2023-08-11T15:29:30.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan/vxlan.ko 2023-08-11T15:29:30.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback 2023-08-11T15:29:30.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-08-11T15:29:30.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netfront.ko 2023-08-11T15:29:30.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb 2023-08-11T15:29:30.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb/ntb.ko 2023-08-11T15:29:30.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme 2023-08-11T15:29:30.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host 2023-08-11T15:29:30.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme-core.ko 2023-08-11T15:29:30.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme.ko 2023-08-11T15:29:30.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport 2023-08-11T15:29:30.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport.ko 2023-08-11T15:29:30.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_cs.ko 2023-08-11T15:29:30.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_pc.ko 2023-08-11T15:29:30.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_serial.ko 2023-08-11T15:29:30.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci 2023-08-11T15:29:30.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug 2023-08-11T15:29:30.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-08-11T15:29:30.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie 2023-08-11T15:29:30.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie/aer_inject.ko 2023-08-11T15:29:30.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/xen-pcifront.ko 2023-08-11T15:29:30.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia 2023-08-11T15:29:30.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/i82092.ko 2023-08-11T15:29:30.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/pd6729.ko 2023-08-11T15:29:30.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/yenta_socket.ko 2023-08-11T15:29:30.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl 2023-08-11T15:29:30.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel 2023-08-11T15:29:30.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-08-11T15:29:30.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform 2023-08-11T15:29:30.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86 2023-08-11T15:29:30.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acer-wmi.ko 2023-08-11T15:29:30.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acerhdf.ko 2023-08-11T15:29:30.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/apple-gmux.ko 2023-08-11T15:29:30.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-laptop.ko 2023-08-11T15:29:30.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-08-11T15:29:30.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-wmi.ko 2023-08-11T15:29:30.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-08-11T15:29:30.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-08-11T15:29:30.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-08-11T15:29:30.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-08-11T15:29:30.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-08-11T15:29:30.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/hdaps.ko 2023-08-11T15:29:30.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel 2023-08-11T15:29:30.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-08-11T15:29:30.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel_ips.ko 2023-08-11T15:29:30.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/msi-wmi.ko 2023-08-11T15:29:30.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-08-11T15:29:30.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-08-11T15:29:30.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-08-11T15:29:30.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-q10.ko 2023-08-11T15:29:30.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-08-11T15:29:30.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-08-11T15:29:30.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-08-11T15:29:30.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-08-11T15:29:30.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi.ko 2023-08-11T15:29:30.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap 2023-08-11T15:29:30.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_common.ko 2023-08-11T15:29:30.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-08-11T15:29:30.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps 2023-08-11T15:29:30.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients 2023-08-11T15:29:30.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-gpio.ko 2023-08-11T15:29:30.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-08-11T15:29:30.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps_parport.ko 2023-08-11T15:29:30.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/pps_core.ko 2023-08-11T15:29:30.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp 2023-08-11T15:29:30.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp.ko 2023-08-11T15:29:30.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp_kvm.ko 2023-08-11T15:29:30.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc 2023-08-11T15:29:30.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq32k.ko 2023-08-11T15:29:30.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq4802.ko 2023-08-11T15:29:30.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1286.ko 2023-08-11T15:29:30.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1307.ko 2023-08-11T15:29:30.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1374.ko 2023-08-11T15:29:30.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1511.ko 2023-08-11T15:29:30.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1553.ko 2023-08-11T15:29:30.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1672.ko 2023-08-11T15:29:30.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1742.ko 2023-08-11T15:29:30.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds2404.ko 2023-08-11T15:29:30.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds3232.ko 2023-08-11T15:29:30.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-em3027.ko 2023-08-11T15:29:30.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-fm3130.ko 2023-08-11T15:29:30.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl12022.ko 2023-08-11T15:29:30.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl1208.ko 2023-08-11T15:29:30.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m41t80.ko 2023-08-11T15:29:30.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t35.ko 2023-08-11T15:29:30.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t59.ko 2023-08-11T15:29:30.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-max6900.ko 2023-08-11T15:29:30.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-msm6242.ko 2023-08-11T15:29:30.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-08-11T15:29:30.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-08-11T15:29:30.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-08-11T15:29:30.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-08-11T15:29:30.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-08-11T15:29:30.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-08-11T15:29:30.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-08-11T15:29:30.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-08-11T15:29:30.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8025.ko 2023-08-11T15:29:30.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8581.ko 2023-08-11T15:29:30.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-08-11T15:29:30.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-x1205.ko 2023-08-11T15:29:30.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi 2023-08-11T15:29:30.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-9xxx.ko 2023-08-11T15:29:30.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-sas.ko 2023-08-11T15:29:30.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-xxxx.ko 2023-08-11T15:29:30.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/BusLogic.ko 2023-08-11T15:29:30.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/a100u2w.ko 2023-08-11T15:29:30.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid 2023-08-11T15:29:30.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-08-11T15:29:30.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/advansys.ko 2023-08-11T15:29:30.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx 2023-08-11T15:29:30.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-08-11T15:29:30.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-08-11T15:29:30.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/am53c974.ko 2023-08-11T15:29:30.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr 2023-08-11T15:29:30.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-08-11T15:29:30.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/atp870u.ko 2023-08-11T15:29:30.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi 2023-08-11T15:29:30.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-08-11T15:29:30.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa 2023-08-11T15:29:30.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa/bfa.ko 2023-08-11T15:29:30.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc 2023-08-11T15:29:30.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-08-11T15:29:30.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i 2023-08-11T15:29:30.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-08-11T15:29:30.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ch.ko 2023-08-11T15:29:30.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor 2023-08-11T15:29:30.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-08-11T15:29:30.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi 2023-08-11T15:29:30.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-08-11T15:29:30.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-08-11T15:29:30.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-08-11T15:29:30.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-08-11T15:29:30.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-08-11T15:29:30.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dc395x.ko 2023-08-11T15:29:30.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler 2023-08-11T15:29:30.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-08-11T15:29:30.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-08-11T15:29:30.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-08-11T15:29:30.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-08-11T15:29:30.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dmx3191d.ko 2023-08-11T15:29:30.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r 2023-08-11T15:29:30.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-08-11T15:29:30.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esp_scsi.ko 2023-08-11T15:29:30.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe 2023-08-11T15:29:30.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-08-11T15:29:30.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-08-11T15:29:30.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic 2023-08-11T15:29:30.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic/fnic.ko 2023-08-11T15:29:30.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hpsa.ko 2023-08-11T15:29:30.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hptiop.ko 2023-08-11T15:29:30.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hv_storvsc.ko 2023-08-11T15:29:30.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/initio.ko 2023-08-11T15:29:30.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ipr.ko 2023-08-11T15:29:30.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ips.ko 2023-08-11T15:29:30.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci 2023-08-11T15:29:30.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci/isci.ko 2023-08-11T15:29:30.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-08-11T15:29:30.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_tcp.ko 2023-08-11T15:29:30.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc 2023-08-11T15:29:30.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc/libfc.ko 2023-08-11T15:29:30.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi.ko 2023-08-11T15:29:30.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-08-11T15:29:30.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas 2023-08-11T15:29:30.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas/libsas.ko 2023-08-11T15:29:30.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc 2023-08-11T15:29:30.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-08-11T15:29:30.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid 2023-08-11T15:29:30.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid.ko 2023-08-11T15:29:30.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-08-11T15:29:30.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-08-11T15:29:30.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-08-11T15:29:30.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas 2023-08-11T15:29:30.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-08-11T15:29:30.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas 2023-08-11T15:29:30.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-08-11T15:29:30.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvumi.ko 2023-08-11T15:29:30.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001 2023-08-11T15:29:30.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-08-11T15:29:30.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pmcraid.ko 2023-08-11T15:29:30.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla1280.ko 2023-08-11T15:29:30.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx 2023-08-11T15:29:30.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-08-11T15:29:30.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-08-11T15:29:30.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx 2023-08-11T15:29:30.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-08-11T15:29:30.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/raid_class.ko 2023-08-11T15:29:30.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_debug.ko 2023-08-11T15:29:30.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-08-11T15:29:30.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-08-11T15:29:30.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-08-11T15:29:30.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-08-11T15:29:30.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-08-11T15:29:30.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ses.ko 2023-08-11T15:29:30.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/st.ko 2023-08-11T15:29:30.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/stex.ko 2023-08-11T15:29:30.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2 2023-08-11T15:29:30.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-08-11T15:29:30.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/virtio_scsi.ko 2023-08-11T15:29:30.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-08-11T15:29:30.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/wd719x.ko 2023-08-11T15:29:30.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb 2023-08-11T15:29:30.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb/ssb.ko 2023-08-11T15:29:30.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target 2023-08-11T15:29:30.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi 2023-08-11T15:29:30.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-08-11T15:29:30.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback 2023-08-11T15:29:30.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback/tcm_loop.ko 2023-08-11T15:29:30.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_file.ko 2023-08-11T15:29:30.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_iblock.ko 2023-08-11T15:29:30.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_mod.ko 2023-08-11T15:29:30.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_pscsi.ko 2023-08-11T15:29:30.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_user.ko 2023-08-11T15:29:30.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc 2023-08-11T15:29:30.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-08-11T15:29:30.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal 2023-08-11T15:29:30.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel 2023-08-11T15:29:30.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal 2023-08-11T15:29:30.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-08-11T15:29:30.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-08-11T15:29:30.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-08-11T15:29:30.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-08-11T15:29:30.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-08-11T15:29:30.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-08-11T15:29:30.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-08-11T15:29:30.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-08-11T15:29:30.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-08-11T15:29:30.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-08-11T15:29:30.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-08-11T15:29:30.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-08-11T15:29:30.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-08-11T15:29:30.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-08-11T15:29:30.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-08-11T15:29:30.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty 2023-08-11T15:29:30.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless 2023-08-11T15:29:30.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-08-11T15:29:30.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_gsm.ko 2023-08-11T15:29:30.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_hdlc.ko 2023-08-11T15:29:30.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/nozomi.ko 2023-08-11T15:29:30.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial 2023-08-11T15:29:30.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250 2023-08-11T15:29:30.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-08-11T15:29:30.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/arc_uart.ko 2023-08-11T15:29:30.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm 2023-08-11T15:29:30.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-08-11T15:29:30.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/synclink_gt.ko 2023-08-11T15:29:30.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs 2023-08-11T15:29:30.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core 2023-08-11T15:29:30.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-08-11T15:29:30.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host 2023-08-11T15:29:30.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-08-11T15:29:30.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio 2023-08-11T15:29:30.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio.ko 2023-08-11T15:29:30.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_aec.ko 2023-08-11T15:29:30.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_cif.ko 2023-08-11T15:29:30.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_pci_generic.ko 2023-08-11T15:29:30.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_sercos3.ko 2023-08-11T15:29:30.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb 2023-08-11T15:29:30.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class 2023-08-11T15:29:30.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-acm.ko 2023-08-11T15:29:30.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-wdm.ko 2023-08-11T15:29:30.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usblp.ko 2023-08-11T15:29:30.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usbtmc.ko 2023-08-11T15:29:30.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host 2023-08-11T15:29:30.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host/sl811-hcd.ko 2023-08-11T15:29:30.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image 2023-08-11T15:29:30.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/mdc800.ko 2023-08-11T15:29:30.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/microtek.ko 2023-08-11T15:29:30.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc 2023-08-11T15:29:30.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/adutux.ko 2023-08-11T15:29:30.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/appledisplay.ko 2023-08-11T15:29:30.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi26.ko 2023-08-11T15:29:30.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi62.ko 2023-08-11T15:29:30.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ezusb.ko 2023-08-11T15:29:30.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/idmouse.ko 2023-08-11T15:29:30.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/iowarrior.ko 2023-08-11T15:29:30.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/isight_firmware.ko 2023-08-11T15:29:30.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ldusb.ko 2023-08-11T15:29:30.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/legousbtower.ko 2023-08-11T15:29:30.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga 2023-08-11T15:29:30.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-08-11T15:29:30.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/trancevibrator.ko 2023-08-11T15:29:30.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usb3503.ko 2023-08-11T15:29:30.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usblcd.ko 2023-08-11T15:29:30.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usbsevseg.ko 2023-08-11T15:29:30.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/uss720.ko 2023-08-11T15:29:30.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/yurex.ko 2023-08-11T15:29:30.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy 2023-08-11T15:29:30.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy/phy-generic.ko 2023-08-11T15:29:30.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial 2023-08-11T15:29:30.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/aircable.ko 2023-08-11T15:29:30.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ark3116.ko 2023-08-11T15:29:30.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/belkin_sa.ko 2023-08-11T15:29:30.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ch341.ko 2023-08-11T15:29:30.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cp210x.ko 2023-08-11T15:29:30.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cyberjack.ko 2023-08-11T15:29:30.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cypress_m8.ko 2023-08-11T15:29:30.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-08-11T15:29:30.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/empeg.ko 2023-08-11T15:29:30.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-08-11T15:29:30.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/garmin_gps.ko 2023-08-11T15:29:30.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_edgeport.ko 2023-08-11T15:29:30.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_ti.ko 2023-08-11T15:29:30.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipaq.ko 2023-08-11T15:29:30.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipw.ko 2023-08-11T15:29:30.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ir-usb.ko 2023-08-11T15:29:30.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-08-11T15:29:30.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan.ko 2023-08-11T15:29:30.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-08-11T15:29:30.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-08-11T15:29:30.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kobil_sct.ko 2023-08-11T15:29:30.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mct_u232.ko 2023-08-11T15:29:30.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7720.ko 2023-08-11T15:29:30.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7840.ko 2023-08-11T15:29:30.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/navman.ko 2023-08-11T15:29:30.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/omninet.ko 2023-08-11T15:29:30.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/opticon.ko 2023-08-11T15:29:30.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/option.ko 2023-08-11T15:29:30.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/oti6858.ko 2023-08-11T15:29:30.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/pl2303.ko 2023-08-11T15:29:30.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcaux.ko 2023-08-11T15:29:30.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcserial.ko 2023-08-11T15:29:30.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/quatech2.ko 2023-08-11T15:29:30.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/safe_serial.ko 2023-08-11T15:29:30.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/sierra.ko 2023-08-11T15:29:30.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/spcp8x5.ko 2023-08-11T15:29:30.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ssu100.ko 2023-08-11T15:29:30.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/symbolserial.ko 2023-08-11T15:29:30.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-08-11T15:29:30.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-08-11T15:29:30.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_debug.ko 2023-08-11T15:29:30.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_wwan.ko 2023-08-11T15:29:30.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/visor.ko 2023-08-11T15:29:30.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/whiteheat.ko 2023-08-11T15:29:30.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/xsens_mt.ko 2023-08-11T15:29:30.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage 2023-08-11T15:29:30.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/uas.ko 2023-08-11T15:29:30.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-alauda.ko 2023-08-11T15:29:30.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-cypress.ko 2023-08-11T15:29:30.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-datafab.ko 2023-08-11T15:29:30.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-08-11T15:29:30.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-freecom.ko 2023-08-11T15:29:30.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-isd200.ko 2023-08-11T15:29:30.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-08-11T15:29:30.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-karma.ko 2023-08-11T15:29:30.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-08-11T15:29:30.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-realtek.ko 2023-08-11T15:29:30.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-08-11T15:29:30.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-08-11T15:29:30.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-usbat.ko 2023-08-11T15:29:30.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/usb-storage.ko 2023-08-11T15:29:30.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip 2023-08-11T15:29:30.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-core.ko 2023-08-11T15:29:30.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-host.ko 2023-08-11T15:29:30.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-08-11T15:29:30.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio 2023-08-11T15:29:30.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci 2023-08-11T15:29:30.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-08-11T15:29:30.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-08-11T15:29:30.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio.ko 2023-08-11T15:29:30.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-08-11T15:29:30.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost 2023-08-11T15:29:30.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost.ko 2023-08-11T15:29:30.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_iotlb.ko 2023-08-11T15:29:30.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_net.ko 2023-08-11T15:29:30.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_scsi.ko 2023-08-11T15:29:30.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video 2023-08-11T15:29:30.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight 2023-08-11T15:29:30.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/apple_bl.ko 2023-08-11T15:29:30.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/lcd.ko 2023-08-11T15:29:30.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/platform_lcd.ko 2023-08-11T15:29:30.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio 2023-08-11T15:29:30.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio.ko 2023-08-11T15:29:30.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_balloon.ko 2023-08-11T15:29:30.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_mmio.ko 2023-08-11T15:29:30.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci.ko 2023-08-11T15:29:30.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-08-11T15:29:30.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-08-11T15:29:30.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_ring.ko 2023-08-11T15:29:30.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1 2023-08-11T15:29:30.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters 2023-08-11T15:29:30.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2482.ko 2023-08-11T15:29:30.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2490.ko 2023-08-11T15:29:30.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves 2023-08-11T15:29:30.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-08-11T15:29:30.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-08-11T15:29:30.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-08-11T15:29:30.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-08-11T15:29:30.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-08-11T15:29:30.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-08-11T15:29:30.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-08-11T15:29:30.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-08-11T15:29:30.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-08-11T15:29:30.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_smem.ko 2023-08-11T15:29:30.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_therm.ko 2023-08-11T15:29:30.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/wire.ko 2023-08-11T15:29:30.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog 2023-08-11T15:29:30.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-08-11T15:29:30.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-08-11T15:29:30.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-08-11T15:29:30.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/hpwdt.ko 2023-08-11T15:29:30.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/i6300esb.ko 2023-08-11T15:29:30.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-08-11T15:29:30.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-08-11T15:29:30.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ib700wdt.ko 2023-08-11T15:29:30.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ibmasr.ko 2023-08-11T15:29:30.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-08-11T15:29:30.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-08-11T15:29:30.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it87_wdt.ko 2023-08-11T15:29:30.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/machzwd.ko 2023-08-11T15:29:30.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/nv_tco.ko 2023-08-11T15:29:30.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_pci.ko 2023-08-11T15:29:30.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_usb.ko 2023-08-11T15:29:30.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-08-11T15:29:30.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-08-11T15:29:30.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/softdog.ko 2023-08-11T15:29:30.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sp5100_tco.ko 2023-08-11T15:29:30.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/via_wdt.ko 2023-08-11T15:29:30.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-08-11T15:29:30.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-08-11T15:29:30.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-08-11T15:29:30.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/wdt_pci.ko 2023-08-11T15:29:30.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/xen_wdt.ko 2023-08-11T15:29:30.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen 2023-08-11T15:29:30.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-acpi-processor.ko 2023-08-11T15:29:30.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-evtchn.ko 2023-08-11T15:29:30.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntalloc.ko 2023-08-11T15:29:30.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntdev.ko 2023-08-11T15:29:30.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback 2023-08-11T15:29:30.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-08-11T15:29:30.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-privcmd.ko 2023-08-11T15:29:30.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs 2023-08-11T15:29:30.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs/xenfs.ko 2023-08-11T15:29:30.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs 2023-08-11T15:29:30.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p 2023-08-11T15:29:30.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p/9p.ko 2023-08-11T15:29:30.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs 2023-08-11T15:29:30.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs/affs.ko 2023-08-11T15:29:30.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs 2023-08-11T15:29:30.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs/befs.ko 2023-08-11T15:29:30.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/binfmt_misc.ko 2023-08-11T15:29:30.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs 2023-08-11T15:29:30.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs/btrfs.ko 2023-08-11T15:29:30.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles 2023-08-11T15:29:30.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles/cachefiles.ko 2023-08-11T15:29:30.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph 2023-08-11T15:29:30.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph/ceph.ko 2023-08-11T15:29:30.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda 2023-08-11T15:29:30.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda/coda.ko 2023-08-11T15:29:30.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs 2023-08-11T15:29:30.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs/cramfs.ko 2023-08-11T15:29:30.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm 2023-08-11T15:29:30.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm/dlm.ko 2023-08-11T15:29:30.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs 2023-08-11T15:29:30.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs/ecryptfs.ko 2023-08-11T15:29:30.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs 2023-08-11T15:29:30.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs/f2fs.ko 2023-08-11T15:29:30.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat 2023-08-11T15:29:30.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/fat.ko 2023-08-11T15:29:30.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/msdos.ko 2023-08-11T15:29:30.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/vfat.ko 2023-08-11T15:29:30.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache 2023-08-11T15:29:30.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache/fscache.ko 2023-08-11T15:29:30.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse 2023-08-11T15:29:30.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/cuse.ko 2023-08-11T15:29:30.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/fuse.ko 2023-08-11T15:29:30.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2 2023-08-11T15:29:30.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2/gfs2.ko 2023-08-11T15:29:30.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs 2023-08-11T15:29:30.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs/hfs.ko 2023-08-11T15:29:30.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus 2023-08-11T15:29:30.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus/hfsplus.ko 2023-08-11T15:29:30.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs 2023-08-11T15:29:30.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs/isofs.ko 2023-08-11T15:29:30.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs 2023-08-11T15:29:30.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs/jfs.ko 2023-08-11T15:29:30.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd 2023-08-11T15:29:30.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd/lockd.ko 2023-08-11T15:29:30.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix 2023-08-11T15:29:30.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix/minix.ko 2023-08-11T15:29:30.382 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs 2023-08-11T15:29:30.382 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs/netfs.ko 2023-08-11T15:29:30.382 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs 2023-08-11T15:29:30.382 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout 2023-08-11T15:29:30.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-08-11T15:29:30.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout 2023-08-11T15:29:30.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-08-11T15:29:30.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout 2023-08-11T15:29:30.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-08-11T15:29:30.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfs.ko 2023-08-11T15:29:30.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv3.ko 2023-08-11T15:29:30.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv4.ko 2023-08-11T15:29:30.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common 2023-08-11T15:29:30.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/grace.ko 2023-08-11T15:29:30.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/nfs_acl.ko 2023-08-11T15:29:30.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd 2023-08-11T15:29:30.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd/nfsd.ko 2023-08-11T15:29:30.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2 2023-08-11T15:29:30.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2/nilfs2.ko 2023-08-11T15:29:30.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls 2023-08-11T15:29:30.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-celtic.ko 2023-08-11T15:29:30.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-centeuro.ko 2023-08-11T15:29:30.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-croatian.ko 2023-08-11T15:29:30.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-cyrillic.ko 2023-08-11T15:29:30.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-gaelic.ko 2023-08-11T15:29:30.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-greek.ko 2023-08-11T15:29:30.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-iceland.ko 2023-08-11T15:29:30.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-inuit.ko 2023-08-11T15:29:30.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-roman.ko 2023-08-11T15:29:30.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-romanian.ko 2023-08-11T15:29:30.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-turkish.ko 2023-08-11T15:29:30.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1250.ko 2023-08-11T15:29:30.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1251.ko 2023-08-11T15:29:30.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1255.ko 2023-08-11T15:29:30.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp737.ko 2023-08-11T15:29:30.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp775.ko 2023-08-11T15:29:30.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp850.ko 2023-08-11T15:29:30.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp852.ko 2023-08-11T15:29:30.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp855.ko 2023-08-11T15:29:30.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp857.ko 2023-08-11T15:29:30.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp860.ko 2023-08-11T15:29:30.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp861.ko 2023-08-11T15:29:30.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp862.ko 2023-08-11T15:29:30.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp863.ko 2023-08-11T15:29:30.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp864.ko 2023-08-11T15:29:30.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp865.ko 2023-08-11T15:29:30.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp866.ko 2023-08-11T15:29:30.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp869.ko 2023-08-11T15:29:30.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp874.ko 2023-08-11T15:29:30.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp932.ko 2023-08-11T15:29:30.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp936.ko 2023-08-11T15:29:30.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp949.ko 2023-08-11T15:29:30.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp950.ko 2023-08-11T15:29:30.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_euc-jp.ko 2023-08-11T15:29:30.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-1.ko 2023-08-11T15:29:30.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-13.ko 2023-08-11T15:29:30.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-14.ko 2023-08-11T15:29:30.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-15.ko 2023-08-11T15:29:30.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-2.ko 2023-08-11T15:29:30.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-3.ko 2023-08-11T15:29:30.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-4.ko 2023-08-11T15:29:30.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-5.ko 2023-08-11T15:29:30.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-6.ko 2023-08-11T15:29:30.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-7.ko 2023-08-11T15:29:30.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-9.ko 2023-08-11T15:29:30.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-r.ko 2023-08-11T15:29:30.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-ru.ko 2023-08-11T15:29:30.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-u.ko 2023-08-11T15:29:30.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_utf8.ko 2023-08-11T15:29:30.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2 2023-08-11T15:29:30.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster 2023-08-11T15:29:30.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-08-11T15:29:30.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm 2023-08-11T15:29:30.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-08-11T15:29:30.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs 2023-08-11T15:29:30.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-08-11T15:29:30.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2.ko 2023-08-11T15:29:30.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-08-11T15:29:30.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-08-11T15:29:30.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-08-11T15:29:30.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs 2023-08-11T15:29:30.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs/overlay.ko 2023-08-11T15:29:30.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore 2023-08-11T15:29:30.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore/ramoops.ko 2023-08-11T15:29:30.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs 2023-08-11T15:29:30.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs/reiserfs.ko 2023-08-11T15:29:30.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs 2023-08-11T15:29:30.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs/romfs.ko 2023-08-11T15:29:30.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb 2023-08-11T15:29:30.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client 2023-08-11T15:29:30.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client/cifs.ko 2023-08-11T15:29:30.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common 2023-08-11T15:29:30.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_arc4.ko 2023-08-11T15:29:30.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_md4.ko 2023-08-11T15:29:30.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs 2023-08-11T15:29:30.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs/squashfs.ko 2023-08-11T15:29:30.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv 2023-08-11T15:29:30.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv/sysv.ko 2023-08-11T15:29:30.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf 2023-08-11T15:29:30.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf/udf.ko 2023-08-11T15:29:30.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs 2023-08-11T15:29:30.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs/ufs.ko 2023-08-11T15:29:30.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs 2023-08-11T15:29:30.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs/xfs.ko 2023-08-11T15:29:30.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel 2023-08-11T15:29:30.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu 2023-08-11T15:29:30.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu/rcutorture.ko 2023-08-11T15:29:30.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/torture.ko 2023-08-11T15:29:30.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace 2023-08-11T15:29:30.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-08-11T15:29:30.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib 2023-08-11T15:29:30.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/asn1_encoder.ko 2023-08-11T15:29:30.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc-itu-t.ko 2023-08-11T15:29:30.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc8.ko 2023-08-11T15:29:30.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto 2023-08-11T15:29:30.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libarc4.ko 2023-08-11T15:29:30.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libdes.ko 2023-08-11T15:29:30.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/libcrc32c.ko 2023-08-11T15:29:30.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lru_cache.ko 2023-08-11T15:29:30.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4 2023-08-11T15:29:30.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4_compress.ko 2023-08-11T15:29:30.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4hc_compress.ko 2023-08-11T15:29:30.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math 2023-08-11T15:29:30.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math/cordic.ko 2023-08-11T15:29:30.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6 2023-08-11T15:29:30.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6/raid6_pq.ko 2023-08-11T15:29:30.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon 2023-08-11T15:29:30.399 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon/reed_solomon.ko 2023-08-11T15:29:30.399 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_bm.ko 2023-08-11T15:29:30.399 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_fsm.ko 2023-08-11T15:29:30.399 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_kmp.ko 2023-08-11T15:29:30.399 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd 2023-08-11T15:29:30.399 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd/zstd_compress.ko 2023-08-11T15:29:30.399 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm 2023-08-11T15:29:30.399 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm/hwpoison-inject.ko 2023-08-11T15:29:30.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net 2023-08-11T15:29:30.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan 2023-08-11T15:29:30.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/6lowpan.ko 2023-08-11T15:29:30.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_dest.ko 2023-08-11T15:29:30.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_fragment.ko 2023-08-11T15:29:30.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_hop.ko 2023-08-11T15:29:30.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_ipv6.ko 2023-08-11T15:29:30.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_mobility.ko 2023-08-11T15:29:30.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_routing.ko 2023-08-11T15:29:30.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_udp.ko 2023-08-11T15:29:30.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802 2023-08-11T15:29:30.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/garp.ko 2023-08-11T15:29:30.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/mrp.ko 2023-08-11T15:29:30.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/p8022.ko 2023-08-11T15:29:30.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/psnap.ko 2023-08-11T15:29:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/stp.ko 2023-08-11T15:29:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q 2023-08-11T15:29:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q/8021q.ko 2023-08-11T15:29:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p 2023-08-11T15:29:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet.ko 2023-08-11T15:29:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_fd.ko 2023-08-11T15:29:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_rdma.ko 2023-08-11T15:29:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_virtio.ko 2023-08-11T15:29:30.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv 2023-08-11T15:29:30.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv/batman-adv.ko 2023-08-11T15:29:30.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge 2023-08-11T15:29:30.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/br_netfilter.ko 2023-08-11T15:29:30.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/bridge.ko 2023-08-11T15:29:30.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter 2023-08-11T15:29:30.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-08-11T15:29:30.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_among.ko 2023-08-11T15:29:30.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-08-11T15:29:30.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-08-11T15:29:30.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-08-11T15:29:30.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-08-11T15:29:30.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-08-11T15:29:30.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-08-11T15:29:30.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_log.ko 2023-08-11T15:29:30.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-08-11T15:29:30.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-08-11T15:29:30.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-08-11T15:29:30.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-08-11T15:29:30.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-08-11T15:29:30.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-08-11T15:29:30.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-08-11T15:29:30.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-08-11T15:29:30.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-08-11T15:29:30.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-08-11T15:29:30.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-08-11T15:29:30.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtables.ko 2023-08-11T15:29:30.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-08-11T15:29:30.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-08-11T15:29:30.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-08-11T15:29:30.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph 2023-08-11T15:29:30.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph/libceph.ko 2023-08-11T15:29:30.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core 2023-08-11T15:29:30.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/failover.ko 2023-08-11T15:29:30.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/pktgen.ko 2023-08-11T15:29:30.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp 2023-08-11T15:29:30.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp.ko 2023-08-11T15:29:30.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_diag.ko 2023-08-11T15:29:30.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv4.ko 2023-08-11T15:29:30.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv6.ko 2023-08-11T15:29:30.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver 2023-08-11T15:29:30.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver/dns_resolver.ko 2023-08-11T15:29:30.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa 2023-08-11T15:29:30.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/dsa_core.ko 2023-08-11T15:29:30.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_brcm.ko 2023-08-11T15:29:30.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_dsa.ko 2023-08-11T15:29:30.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_none.ko 2023-08-11T15:29:30.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_trailer.ko 2023-08-11T15:29:30.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154 2023-08-11T15:29:30.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan 2023-08-11T15:29:30.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-08-11T15:29:30.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154.ko 2023-08-11T15:29:30.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154_socket.ko 2023-08-11T15:29:30.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4 2023-08-11T15:29:30.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ah4.ko 2023-08-11T15:29:30.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/esp4.ko 2023-08-11T15:29:30.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/fou.ko 2023-08-11T15:29:30.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/gre.ko 2023-08-11T15:29:30.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/inet_diag.ko 2023-08-11T15:29:30.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_gre.ko 2023-08-11T15:29:30.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_tunnel.ko 2023-08-11T15:29:30.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_vti.ko 2023-08-11T15:29:30.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipcomp.ko 2023-08-11T15:29:30.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipip.ko 2023-08-11T15:29:30.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter 2023-08-11T15:29:30.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-08-11T15:29:30.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-08-11T15:29:30.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-08-11T15:29:30.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-08-11T15:29:30.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-08-11T15:29:30.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-08-11T15:29:30.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-08-11T15:29:30.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-08-11T15:29:30.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-08-11T15:29:30.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-08-11T15:29:30.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-08-11T15:29:30.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-08-11T15:29:30.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-08-11T15:29:30.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-08-11T15:29:30.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-08-11T15:29:30.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-08-11T15:29:30.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-08-11T15:29:30.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-08-11T15:29:30.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-08-11T15:29:30.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-08-11T15:29:30.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-08-11T15:29:30.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_bic.ko 2023-08-11T15:29:30.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_dctcp.ko 2023-08-11T15:29:30.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_diag.ko 2023-08-11T15:29:30.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_highspeed.ko 2023-08-11T15:29:30.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_htcp.ko 2023-08-11T15:29:30.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_hybla.ko 2023-08-11T15:29:30.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_illinois.ko 2023-08-11T15:29:30.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_lp.ko 2023-08-11T15:29:30.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_scalable.ko 2023-08-11T15:29:30.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_vegas.ko 2023-08-11T15:29:30.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_veno.ko 2023-08-11T15:29:30.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_westwood.ko 2023-08-11T15:29:30.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_yeah.ko 2023-08-11T15:29:30.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tunnel4.ko 2023-08-11T15:29:30.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_diag.ko 2023-08-11T15:29:30.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_tunnel.ko 2023-08-11T15:29:30.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-08-11T15:29:30.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6 2023-08-11T15:29:30.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ah6.ko 2023-08-11T15:29:30.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/esp6.ko 2023-08-11T15:29:30.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/fou6.ko 2023-08-11T15:29:30.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_tunnel.ko 2023-08-11T15:29:30.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-08-11T15:29:30.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_vti.ko 2023-08-11T15:29:30.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ipcomp6.ko 2023-08-11T15:29:30.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter 2023-08-11T15:29:30.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-08-11T15:29:30.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-08-11T15:29:30.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-08-11T15:29:30.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-08-11T15:29:30.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-08-11T15:29:30.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-08-11T15:29:30.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-08-11T15:29:30.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-08-11T15:29:30.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-08-11T15:29:30.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-08-11T15:29:30.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-08-11T15:29:30.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-08-11T15:29:30.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-08-11T15:29:30.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-08-11T15:29:30.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-08-11T15:29:30.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-08-11T15:29:30.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-08-11T15:29:30.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-08-11T15:29:30.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-08-11T15:29:30.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-08-11T15:29:30.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-08-11T15:29:30.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-08-11T15:29:30.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-08-11T15:29:30.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-08-11T15:29:30.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/sit.ko 2023-08-11T15:29:30.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/tunnel6.ko 2023-08-11T15:29:30.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-08-11T15:29:30.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key 2023-08-11T15:29:30.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key/af_key.ko 2023-08-11T15:29:30.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp 2023-08-11T15:29:30.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_core.ko 2023-08-11T15:29:30.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_debugfs.ko 2023-08-11T15:29:30.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_eth.ko 2023-08-11T15:29:30.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip.ko 2023-08-11T15:29:30.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip6.ko 2023-08-11T15:29:30.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_netlink.ko 2023-08-11T15:29:30.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ppp.ko 2023-08-11T15:29:30.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc 2023-08-11T15:29:30.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc/llc.ko 2023-08-11T15:29:30.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154 2023-08-11T15:29:30.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154/mac802154.ko 2023-08-11T15:29:30.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls 2023-08-11T15:29:30.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls/mpls_gso.ko 2023-08-11T15:29:30.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter 2023-08-11T15:29:30.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset 2023-08-11T15:29:30.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set.ko 2023-08-11T15:29:30.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-08-11T15:29:30.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-08-11T15:29:30.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-08-11T15:29:30.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-08-11T15:29:30.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-08-11T15:29:30.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-08-11T15:29:30.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-08-11T15:29:30.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-08-11T15:29:30.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-08-11T15:29:30.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-08-11T15:29:30.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-08-11T15:29:30.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-08-11T15:29:30.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-08-11T15:29:30.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-08-11T15:29:30.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-08-11T15:29:30.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs 2023-08-11T15:29:30.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-08-11T15:29:30.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-08-11T15:29:30.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-08-11T15:29:30.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-08-11T15:29:30.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-08-11T15:29:30.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-08-11T15:29:30.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-08-11T15:29:30.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-08-11T15:29:30.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-08-11T15:29:30.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-08-11T15:29:30.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-08-11T15:29:30.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-08-11T15:29:30.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-08-11T15:29:30.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-08-11T15:29:30.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conncount.ko 2023-08-11T15:29:30.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack.ko 2023-08-11T15:29:30.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-08-11T15:29:30.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-08-11T15:29:30.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-08-11T15:29:30.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-08-11T15:29:30.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-08-11T15:29:30.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-08-11T15:29:30.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-08-11T15:29:30.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-08-11T15:29:30.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-08-11T15:29:30.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-08-11T15:29:30.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-08-11T15:29:30.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-08-11T15:29:30.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_dup_netdev.ko 2023-08-11T15:29:30.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table.ko 2023-08-11T15:29:30.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-08-11T15:29:30.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_log_syslog.ko 2023-08-11T15:29:30.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat.ko 2023-08-11T15:29:30.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_amanda.ko 2023-08-11T15:29:30.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_ftp.ko 2023-08-11T15:29:30.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_irc.ko 2023-08-11T15:29:30.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_sip.ko 2023-08-11T15:29:30.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_tftp.ko 2023-08-11T15:29:30.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_synproxy_core.ko 2023-08-11T15:29:30.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_tables.ko 2023-08-11T15:29:30.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink.ko 2023-08-11T15:29:30.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_acct.ko 2023-08-11T15:29:30.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_log.ko 2023-08-11T15:29:30.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_osf.ko 2023-08-11T15:29:30.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_queue.ko 2023-08-11T15:29:30.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_chain_nat.ko 2023-08-11T15:29:30.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_compat.ko 2023-08-11T15:29:30.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_ct.ko 2023-08-11T15:29:30.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_dup_netdev.ko 2023-08-11T15:29:30.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fib.ko 2023-08-11T15:29:30.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-08-11T15:29:30.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_hash.ko 2023-08-11T15:29:30.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_limit.ko 2023-08-11T15:29:30.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_log.ko 2023-08-11T15:29:30.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_masq.ko 2023-08-11T15:29:30.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_nat.ko 2023-08-11T15:29:30.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_numgen.ko 2023-08-11T15:29:30.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_queue.ko 2023-08-11T15:29:30.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_quota.ko 2023-08-11T15:29:30.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_redir.ko 2023-08-11T15:29:30.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject.ko 2023-08-11T15:29:30.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject_inet.ko 2023-08-11T15:29:30.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_socket.ko 2023-08-11T15:29:30.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_tunnel.ko 2023-08-11T15:29:30.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_AUDIT.ko 2023-08-11T15:29:30.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-08-11T15:29:30.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-08-11T15:29:30.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-08-11T15:29:30.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CT.ko 2023-08-11T15:29:30.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_DSCP.ko 2023-08-11T15:29:30.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HL.ko 2023-08-11T15:29:30.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HMARK.ko 2023-08-11T15:29:30.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-08-11T15:29:30.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LED.ko 2023-08-11T15:29:30.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LOG.ko 2023-08-11T15:29:30.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-08-11T15:29:30.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NETMAP.ko 2023-08-11T15:29:30.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFLOG.ko 2023-08-11T15:29:30.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-08-11T15:29:30.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_RATEEST.ko 2023-08-11T15:29:30.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_REDIRECT.ko 2023-08-11T15:29:30.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_SECMARK.ko 2023-08-11T15:29:30.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPMSS.ko 2023-08-11T15:29:30.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-08-11T15:29:30.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TEE.ko 2023-08-11T15:29:30.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TPROXY.ko 2023-08-11T15:29:30.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TRACE.ko 2023-08-11T15:29:30.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_addrtype.ko 2023-08-11T15:29:30.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_bpf.ko 2023-08-11T15:29:30.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cgroup.ko 2023-08-11T15:29:30.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cluster.ko 2023-08-11T15:29:30.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_comment.ko 2023-08-11T15:29:30.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connbytes.ko 2023-08-11T15:29:30.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlabel.ko 2023-08-11T15:29:30.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlimit.ko 2023-08-11T15:29:30.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connmark.ko 2023-08-11T15:29:30.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_conntrack.ko 2023-08-11T15:29:30.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cpu.ko 2023-08-11T15:29:30.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dccp.ko 2023-08-11T15:29:30.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_devgroup.ko 2023-08-11T15:29:30.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dscp.ko 2023-08-11T15:29:30.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ecn.ko 2023-08-11T15:29:30.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_esp.ko 2023-08-11T15:29:30.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hashlimit.ko 2023-08-11T15:29:30.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_helper.ko 2023-08-11T15:29:30.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hl.ko 2023-08-11T15:29:30.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipcomp.ko 2023-08-11T15:29:30.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_iprange.ko 2023-08-11T15:29:30.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipvs.ko 2023-08-11T15:29:30.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_l2tp.ko 2023-08-11T15:29:30.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_length.ko 2023-08-11T15:29:30.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_limit.ko 2023-08-11T15:29:30.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mac.ko 2023-08-11T15:29:30.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mark.ko 2023-08-11T15:29:30.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_multiport.ko 2023-08-11T15:29:30.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nat.ko 2023-08-11T15:29:30.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nfacct.ko 2023-08-11T15:29:30.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_osf.ko 2023-08-11T15:29:30.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_owner.ko 2023-08-11T15:29:30.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_physdev.ko 2023-08-11T15:29:30.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_pkttype.ko 2023-08-11T15:29:30.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_policy.ko 2023-08-11T15:29:30.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_quota.ko 2023-08-11T15:29:30.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_rateest.ko 2023-08-11T15:29:30.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_realm.ko 2023-08-11T15:29:30.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_recent.ko 2023-08-11T15:29:30.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_sctp.ko 2023-08-11T15:29:30.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_set.ko 2023-08-11T15:29:30.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_socket.ko 2023-08-11T15:29:30.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_state.ko 2023-08-11T15:29:30.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_statistic.ko 2023-08-11T15:29:30.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_string.ko 2023-08-11T15:29:30.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_tcpmss.ko 2023-08-11T15:29:30.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_time.ko 2023-08-11T15:29:30.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_u32.ko 2023-08-11T15:29:30.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink 2023-08-11T15:29:30.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink/netlink_diag.ko 2023-08-11T15:29:30.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh 2023-08-11T15:29:30.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh/nsh.ko 2023-08-11T15:29:30.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch 2023-08-11T15:29:30.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/openvswitch.ko 2023-08-11T15:29:30.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-geneve.ko 2023-08-11T15:29:30.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-gre.ko 2023-08-11T15:29:30.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-vxlan.ko 2023-08-11T15:29:30.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet 2023-08-11T15:29:30.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet/af_packet_diag.ko 2023-08-11T15:29:30.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds 2023-08-11T15:29:30.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds.ko 2023-08-11T15:29:30.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_rdma.ko 2023-08-11T15:29:30.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_tcp.ko 2023-08-11T15:29:30.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched 2023-08-11T15:29:30.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_csum.ko 2023-08-11T15:29:30.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_gact.ko 2023-08-11T15:29:30.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_ipt.ko 2023-08-11T15:29:30.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_mirred.ko 2023-08-11T15:29:30.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_nat.ko 2023-08-11T15:29:30.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_pedit.ko 2023-08-11T15:29:30.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_police.ko 2023-08-11T15:29:30.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_simple.ko 2023-08-11T15:29:30.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_skbedit.ko 2023-08-11T15:29:30.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_vlan.ko 2023-08-11T15:29:30.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_basic.ko 2023-08-11T15:29:30.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_bpf.ko 2023-08-11T15:29:30.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_flow.ko 2023-08-11T15:29:30.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_fw.ko 2023-08-11T15:29:30.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_route.ko 2023-08-11T15:29:30.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_u32.ko 2023-08-11T15:29:30.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_cmp.ko 2023-08-11T15:29:30.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_ipset.ko 2023-08-11T15:29:30.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_meta.ko 2023-08-11T15:29:30.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_nbyte.ko 2023-08-11T15:29:30.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_text.ko 2023-08-11T15:29:30.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_u32.ko 2023-08-11T15:29:30.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_choke.ko 2023-08-11T15:29:30.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_codel.ko 2023-08-11T15:29:30.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_drr.ko 2023-08-11T15:29:30.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_fq.ko 2023-08-11T15:29:30.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_gred.ko 2023-08-11T15:29:30.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hfsc.ko 2023-08-11T15:29:30.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hhf.ko 2023-08-11T15:29:30.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_htb.ko 2023-08-11T15:29:30.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_ingress.ko 2023-08-11T15:29:30.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio.ko 2023-08-11T15:29:30.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio_lib.ko 2023-08-11T15:29:30.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_multiq.ko 2023-08-11T15:29:30.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_netem.ko 2023-08-11T15:29:30.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_pie.ko 2023-08-11T15:29:30.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_plug.ko 2023-08-11T15:29:30.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_prio.ko 2023-08-11T15:29:30.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_qfq.ko 2023-08-11T15:29:30.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_red.ko 2023-08-11T15:29:30.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfb.ko 2023-08-11T15:29:30.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfq.ko 2023-08-11T15:29:30.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_tbf.ko 2023-08-11T15:29:30.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_teql.ko 2023-08-11T15:29:30.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp 2023-08-11T15:29:30.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp.ko 2023-08-11T15:29:30.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp_diag.ko 2023-08-11T15:29:30.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc 2023-08-11T15:29:30.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss 2023-08-11T15:29:30.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-08-11T15:29:30.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-08-11T15:29:30.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/sunrpc.ko 2023-08-11T15:29:30.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma 2023-08-11T15:29:30.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-08-11T15:29:30.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc 2023-08-11T15:29:30.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/diag.ko 2023-08-11T15:29:30.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/tipc.ko 2023-08-11T15:29:30.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix 2023-08-11T15:29:30.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix/unix_diag.ko 2023-08-11T15:29:30.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock 2023-08-11T15:29:30.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-08-11T15:29:30.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-08-11T15:29:30.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock.ko 2023-08-11T15:29:30.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_diag.ko 2023-08-11T15:29:30.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-08-11T15:29:30.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm 2023-08-11T15:29:30.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-08-11T15:29:30.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security 2023-08-11T15:29:30.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys 2023-08-11T15:29:30.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys 2023-08-11T15:29:30.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-08-11T15:29:30.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys 2023-08-11T15:29:30.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys/trusted.ko 2023-08-11T15:29:30.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt 2023-08-11T15:29:30.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib 2023-08-11T15:29:30.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib/irqbypass.ko 2023-08-11T15:29:30.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias 2023-08-11T15:29:30.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias.bin 2023-08-11T15:29:30.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin 2023-08-11T15:29:30.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.alias.bin 2023-08-11T15:29:30.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.bin 2023-08-11T15:29:30.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.modinfo 2023-08-11T15:29:30.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep 2023-08-11T15:29:30.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep.bin 2023-08-11T15:29:30.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.devname 2023-08-11T15:29:30.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.order 2023-08-11T15:29:30.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.softdep 2023-08-11T15:29:30.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols 2023-08-11T15:29:30.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols.bin 2023-08-11T15:29:30.456 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:30.456 DEBUG:teuthology.orchestra.run.smithi119:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-08-11T15:29:30.480 INFO:teuthology.orchestra.run.smithi119.stdout:postinstall scriptlet (using /bin/sh): 2023-08-11T15:29:30.480 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d -a -r /boot/System.map-6.5.0-rc4-ga7fb1265323d ]; then 2023-08-11T15:29:30.480 INFO:teuthology.orchestra.run.smithi119.stdout:cp /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:29:30.480 INFO:teuthology.orchestra.run.smithi119.stdout:cp /boot/System.map-6.5.0-rc4-ga7fb1265323d /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:29:30.480 INFO:teuthology.orchestra.run.smithi119.stdout:rm -f /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:30.481 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/installkernel 6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:29:30.481 INFO:teuthology.orchestra.run.smithi119.stdout:rm -f /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:29:30.481 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-08-11T15:29:30.481 INFO:teuthology.orchestra.run.smithi119.stdout:preuninstall scriptlet (using /bin/sh): 2023-08-11T15:29:30.481 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-08-11T15:29:30.481 INFO:teuthology.orchestra.run.smithi119.stdout:new-kernel-pkg --remove 6.5.0-rc4-ga7fb1265323d --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc4-ga7fb1265323d.img 2023-08-11T15:29:30.481 INFO:teuthology.orchestra.run.smithi119.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-08-11T15:29:30.481 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-install remove 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:30.482 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-08-11T15:29:30.482 INFO:teuthology.orchestra.run.smithi119.stdout:postuninstall scriptlet (using /bin/sh): 2023-08-11T15:29:30.482 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/update-bootloader ]; then 2023-08-11T15:29:30.482 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/update-bootloader --remove 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:30.482 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-08-11T15:29:30.483 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2023-08-11T15:29:30.573 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2023-08-11T15:29:30.573 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2023-08-11T15:29:30.573 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.06 2023-08-11T15:29:30.574 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 53.el9 2023-08-11T15:29:30.574 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2023-08-11T15:29:30.574 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Mon 30 Jan 2023 10:38:40 AM UTC 2023-08-11T15:29:30.574 INFO:teuthology.orchestra.run.smithi119.stdout:Group : Unspecified 2023-08-11T15:29:30.575 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 8551923 2023-08-11T15:29:30.575 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2023-08-11T15:29:30.575 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Wed 18 Jan 2023 04:40:07 PM UTC, Key ID 05b555b38483c65d 2023-08-11T15:29:30.576 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.06-53.el9.src.rpm 2023-08-11T15:29:30.576 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Fri 13 Jan 2023 09:07:28 PM UTC 2023-08-11T15:29:30.576 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-08-11T15:29:30.577 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : builder@centos.org 2023-08-11T15:29:30.577 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2023-08-11T15:29:30.577 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-11T15:29:30.577 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2023-08-11T15:29:30.578 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2023-08-11T15:29:30.578 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:29:30.578 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-11T15:29:30.579 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-11T15:29:30.579 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-11T15:29:30.579 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2023-08-11T15:29:30.580 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:29:30.580 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2023-08-11T15:29:30.581 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-11T15:29:30.581 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:30.581 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-11T15:29:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2023-08-11T15:29:32.636 INFO:teuthology.orchestra.run.smithi119.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-08-11T15:29:32.668 INFO:teuthology.orchestra.run.smithi119.stderr:done 2023-08-11T15:29:32.671 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2023-08-11T15:29:32.729 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.33SLmOH1X3 2023-08-11T15:29:32.729 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.33SLmOH1X3 2023-08-11T15:29:32.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.33SLmOH1X3 2023-08-11T15:29:32.956 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.33SLmOH1X3 is 6KB 2023-08-11T15:29:33.004 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.33SLmOH1X3 2023-08-11T15:29:33.018 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2023-08-11T15:29:33.083 INFO:teuthology.orchestra.run.smithi119.stdout:e13cba1d25f14c51a1d04da8db823a4e-0-rescue.conf 2023-08-11T15:29:33.083 INFO:teuthology.orchestra.run.smithi119.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-239.el9.x86_64.conf 2023-08-11T15:29:33.083 INFO:teuthology.orchestra.run.smithi119.stdout:e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc4-ga7fb1265323d.conf 2023-08-11T15:29:33.085 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:29:33.269 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2023-08-11T15:29:52.541 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-08-11T15:29:52.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-08-11T15:29:52.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:30:03.284 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-08-11T15:30:03.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-08-11T15:30:03.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:30:21.680 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-08-11T15:30:27.060 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-08-11T15:30:30.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-08-11T15:30:30.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:30:33.748 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-08-11T15:30:36.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-08-11T15:30:36.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:30:39.120 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-08-11T15:30:45.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-08-11T15:30:45.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:30:51.122 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-08-11T15:30:51.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:30:54.192 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-08-11T15:31:04.176 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-08-11T15:31:09.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-08-11T15:31:09.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:31:19.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-08-11T15:31:19.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:31:19.678 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-08-11T15:31:20.331 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-08-11T15:31:20.331 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc4-ga7fb1265323d"... 2023-08-11T15:31:20.331 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-08-11T15:31:20.372 INFO:teuthology.orchestra.run.smithi119.stdout:6.5.0-rc4-ga7fb1265323d 2023-08-11T15:31:20.373 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc4-ga7fb1265323d vs 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:31:20.373 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-11T15:31:20.373 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-08-11T15:31:21.374 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-08-11T15:31:21.374 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-08-11T15:31:21.500 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2023-08-11T15:31:21.543 DEBUG:teuthology.parallel:result is None 2023-08-11T15:31:41.228 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-08-11T15:31:41.894 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-08-11T15:31:41.894 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc4-ga7fb1265323d"... 2023-08-11T15:31:41.895 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2023-08-11T15:31:41.936 INFO:teuthology.orchestra.run.smithi027.stdout:6.5.0-rc4-ga7fb1265323d 2023-08-11T15:31:41.937 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc4-ga7fb1265323d vs 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:31:41.937 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-11T15:31:41.937 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-08-11T15:31:42.938 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-08-11T15:31:42.939 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-08-11T15:31:43.077 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2023-08-11T15:31:43.144 DEBUG:teuthology.parallel:result is None 2023-08-11T15:31:43.145 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-11T15:31:43.151 INFO:teuthology.task.internal:Creating test directory... 2023-08-11T15:31:43.152 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-11T15:31:43.158 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-11T15:31:43.218 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-11T15:31:43.225 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-11T15:31:43.236 INFO:teuthology.task.internal:Creating archive directory... 2023-08-11T15:31:43.236 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-11T15:31:43.242 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-11T15:31:43.312 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-11T15:31:43.324 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-11T15:31:43.325 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-11T15:31:43.332 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-11T15:31:43.440 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T15:31:43.450 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T15:31:43.484 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T15:31:43.496 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T15:31:43.501 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-11T15:31:43.509 INFO:teuthology.task.internal:Configuring sudo... 2023-08-11T15:31:43.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-11T15:31:43.514 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-11T15:31:43.618 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-11T15:31:43.627 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-11T15:31:43.628 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-11T15:31:43.633 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-11T15:31:43.692 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T15:31:43.789 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T15:31:43.887 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T15:31:43.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T15:31:44.052 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:31:44.052 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-11T15:31:44.130 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T15:31:44.224 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T15:31:44.324 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T15:31:44.399 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T15:31:44.495 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:31:44.495 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-11T15:31:44.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2023-08-11T15:31:44.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2023-08-11T15:31:44.715 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T15:31:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T15:31:45.185 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-11T15:31:45.193 INFO:teuthology.task.internal:Starting timer... 2023-08-11T15:31:45.194 INFO:teuthology.run_tasks:Running task pcp... 2023-08-11T15:31:45.207 INFO:teuthology.run_tasks:Running task selinux... 2023-08-11T15:31:45.217 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2023-08-11T15:31:45.341 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2023-08-11T15:31:45.345 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2023-08-11T15:31:45.485 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2023-08-11T15:31:45.490 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-11T15:31:45.490 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2023-08-11T15:31:45.544 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2023-08-11T15:31:45.545 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2023-08-11T15:31:45.610 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2023-08-11T15:31:45.611 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi119.front.sepia.ceph.com': 'permissive'} 2023-08-11T15:31:45.611 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T15:31:45.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:31:45.695 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T15:31:45.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:31:45.800 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-11T15:31:45.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2023-08-11T15:31:45.895 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2023-08-11T15:31:45.991 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-11T15:31:46.001 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-08-11T15:31:46.001 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-11T15:31:46.021 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-11T15:31:46.022 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi119.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-11T15:43:17.270 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com')] 2023-08-11T15:43:17.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-08-11T15:43:17.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:43:17.460 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-08-11T15:43:17.628 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-08-11T15:43:17.628 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-08-11T15:43:17.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:43:17.769 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-08-11T15:43:17.928 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-08-11T15:43:17.928 INFO:teuthology.run_tasks:Running task clock... 2023-08-11T15:43:17.951 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-11T15:43:17.952 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-11T15:43:17.952 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T15:43:17.962 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-11T15:43:17.963 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T15:43:18.065 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-11T15:43:18.074 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-11T15:43:18.117 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-11T15:43:18.139 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-11T15:43:18.209 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2023-08-11T15:43:18.251 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2023-08-11T15:43:18.252 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2023-08-11T15:43:18.301 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2023-08-11T15:43:18.305 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-11T15:43:18.357 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-11T15:43:18.366 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-11T15:43:18.432 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-11T15:43:18.488 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2023-08-11T15:43:18.495 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T15:43:18.495 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2023-08-11T15:43:18.496 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:43:18.496 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:43:18.496 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:43:18.496 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:43:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2023-08-11T15:43:18.591 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T15:43:18.591 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2023-08-11T15:43:18.591 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:43:18.592 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:43:18.592 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:43:18.592 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:43:18.592 INFO:teuthology.run_tasks:Running task install... 2023-08-11T15:43:18.601 DEBUG:teuthology.task.install:project ceph 2023-08-11T15:43:18.601 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc'}} 2023-08-11T15:43:18.602 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc'} 2023-08-11T15:43:18.602 INFO:teuthology.task.install:Using flavor: default 2023-08-11T15:43:18.607 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-11T15:43:18.608 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-08-11T15:43:18.608 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'tag': None, 'wait_for_package': False} 2023-08-11T15:43:18.609 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:43:18.611 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'tag': None, 'wait_for_package': False} 2023-08-11T15:43:18.611 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:43:18.755 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/9/flavors/default/ 2023-08-11T15:43:18.755 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5437.g92684807 2023-08-11T15:43:18.841 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/9/flavors/default/ 2023-08-11T15:43:18.841 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5437.g92684807 2023-08-11T15:43:18.941 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-11T15:43:18.942 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:43:18.942 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-11T15:43:19.013 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-11T15:43:19.013 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:43:19.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-11T15:43:19.046 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-11T15:43:19.046 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92684807f016ce045ac66e37fdde6f8c47857bcc/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-11T15:43:19.135 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-11T15:43:19.135 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92684807f016ce045ac66e37fdde6f8c47857bcc/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-11T15:43:19.190 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-11T15:43:19.289 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-11T15:43:19.322 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-11T15:43:19.418 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2023-08-11T15:43:19.423 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2023-08-11T15:43:19.449 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-11T15:43:19.557 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2023-08-11T15:43:19.562 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2023-08-11T15:43:19.792 INFO:teuthology.orchestra.run.smithi119.stdout:72 files removed 2023-08-11T15:43:19.830 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-11T15:43:19.963 INFO:teuthology.orchestra.run.smithi027.stdout:72 files removed 2023-08-11T15:43:20.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-11T15:43:20.716 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 175 kB/s | 78 kB 00:00 2023-08-11T15:43:20.883 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 189 kB/s | 78 kB 00:00 2023-08-11T15:43:21.276 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-08-11T15:43:21.458 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 19 kB/s | 9.4 kB 00:00 2023-08-11T15:43:21.768 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2023-08-11T15:43:21.958 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2023-08-11T15:43:22.357 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-08-11T15:43:22.612 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-08-11T15:43:23.863 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2023-08-11T15:43:24.171 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2023-08-11T15:43:24.926 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 7.8 MB 00:00 2023-08-11T15:43:25.142 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 7.8 MB 00:00 2023-08-11T15:43:28.307 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 18 MB 00:01 2023-08-11T15:43:28.333 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 18 MB 00:02 2023-08-11T15:43:32.328 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 14 MB/s | 5.6 MB 00:00 2023-08-11T15:43:32.353 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 10 MB/s | 5.6 MB 00:00 2023-08-11T15:43:33.829 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 14 kB 00:00 2023-08-11T15:43:33.837 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 14 kB 00:00 2023-08-11T15:43:55.870 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 861 kB/s | 18 MB 00:21 2023-08-11T15:44:03.373 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-11T15:44:03.375 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-11T15:44:03.393 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.41.8-1.el9.x86_64 is already installed. 2023-08-11T15:44:03.394 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.0.0-2.el9.x86_64 is already installed. 2023-08-11T15:44:03.395 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-9.el9.x86_64 is already installed. 2023-08-11T15:44:03.397 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-08-11T15:44:03.399 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-08-11T15:44:03.402 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.10-1.el9.x86_64 is already installed. 2023-08-11T15:44:03.403 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-5.14.2-1.el9.x86_64 is already installed. 2023-08-11T15:44:03.491 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-08-11T15:44:03.519 INFO:teuthology.orchestra.run.smithi027.stdout:============================================================================================ 2023-08-11T15:44:03.519 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2023-08-11T15:44:03.520 INFO:teuthology.orchestra.run.smithi027.stdout:============================================================================================ 2023-08-11T15:44:03.520 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2023-08-11T15:44:03.520 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-08-11T15:44:03.520 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:18.0.0-5437.g92684807.el9 ceph 6.5 k 2023-08-11T15:44:03.520 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:18.0.0-5437.g92684807.el9 ceph 5.2 M 2023-08-11T15:44:03.520 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:18.0.0-5437.g92684807.el9 ceph 848 k 2023-08-11T15:44:03.521 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5437.g92684807.el9 ceph 141 k 2023-08-11T15:44:03.521 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:18.0.0-5437.g92684807.el9 ceph 1.5 M 2023-08-11T15:44:03.521 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 131 k 2023-08-11T15:44:03.521 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 1.7 M 2023-08-11T15:44:03.521 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 7.4 M 2023-08-11T15:44:03.521 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 47 k 2023-08-11T15:44:03.521 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:18.0.0-5437.g92684807.el9 ceph 8.1 M 2023-08-11T15:44:03.521 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:18.0.0-5437.g92684807.el9 ceph 40 M 2023-08-11T15:44:03.522 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 261 k 2023-08-11T15:44:03.522 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 224 k 2023-08-11T15:44:03.522 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:18.0.0-5437.g92684807.el9 ceph 218 k 2023-08-11T15:44:03.522 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 18 k 2023-08-11T15:44:03.522 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 CentOS-AppStream 21 k 2023-08-11T15:44:03.522 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-08-11T15:44:03.522 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-08-11T15:44:03.523 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:18.0.0-5437.g92684807.el9 ceph 31 k 2023-08-11T15:44:03.523 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:18.0.0-5437.g92684807.el9 ceph 665 k 2023-08-11T15:44:03.523 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:18.0.0-5437.g92684807.el9 ceph 126 k 2023-08-11T15:44:03.523 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-08-11T15:44:03.523 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-13.el9 CentOS-AppStream 25 k 2023-08-11T15:44:03.523 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.0.7-24.el9 CentOS-AppStream 3.0 M 2023-08-11T15:44:03.523 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-08-11T15:44:03.523 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-08-11T15:44:03.524 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-08-11T15:44:03.524 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:18.0.0-5437.g92684807.el9 ceph 162 k 2023-08-11T15:44:03.524 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:18.0.0-5437.g92684807.el9 ceph 321 k 2023-08-11T15:44:03.524 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:18.0.0-5437.g92684807.el9 ceph 297 k 2023-08-11T15:44:03.524 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:18.0.0-5437.g92684807.el9 ceph 99 k 2023-08-11T15:44:03.524 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:18.0.0-5437.g92684807.el9 ceph 85 k 2023-08-11T15:44:03.524 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:18.0.0-5437.g92684807.el9 ceph 3.0 M 2023-08-11T15:44:03.524 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:18.0.0-5437.g92684807.el9 ceph 168 k 2023-08-11T15:44:03.525 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-08-11T15:44:03.525 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2023-08-11T15:44:03.525 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager x86_64 1:1.43.90-1.el9 CentOS-BaseOS 2.2 M 2023-08-11T15:44:03.525 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm x86_64 1:1.43.90-1.el9 CentOS-BaseOS 1.8 M 2023-08-11T15:44:03.525 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team x86_64 1:1.43.90-1.el9 CentOS-BaseOS 41 k 2023-08-11T15:44:03.525 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui x86_64 1:1.43.90-1.el9 CentOS-BaseOS 245 k 2023-08-11T15:44:03.525 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client x86_64 0.189-3.el9 CentOS-BaseOS 36 k 2023-08-11T15:44:03.526 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf x86_64 0.189-3.el9 CentOS-BaseOS 192 k 2023-08-11T15:44:03.526 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs x86_64 0.189-3.el9 CentOS-BaseOS 254 k 2023-08-11T15:44:03.526 INFO:teuthology.orchestra.run.smithi027.stdout: iproute x86_64 6.2.0-5.el9 CentOS-BaseOS 785 k 2023-08-11T15:44:03.526 INFO:teuthology.orchestra.run.smithi027.stdout: iproute-tc x86_64 6.2.0-5.el9 CentOS-BaseOS 448 k 2023-08-11T15:44:03.526 INFO:teuthology.orchestra.run.smithi027.stdout: libblkid x86_64 2.37.4-13.el9 CentOS-BaseOS 107 k 2023-08-11T15:44:03.526 INFO:teuthology.orchestra.run.smithi027.stdout: libfdisk x86_64 2.37.4-13.el9 CentOS-BaseOS 154 k 2023-08-11T15:44:03.526 INFO:teuthology.orchestra.run.smithi027.stdout: libmount x86_64 2.37.4-13.el9 CentOS-BaseOS 133 k 2023-08-11T15:44:03.526 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:18.0.0-5437.g92684807.el9 ceph 3.3 M 2023-08-11T15:44:03.527 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:18.0.0-5437.g92684807.el9 ceph 3.0 M 2023-08-11T15:44:03.527 INFO:teuthology.orchestra.run.smithi027.stdout: libsmartcols x86_64 2.37.4-13.el9 CentOS-BaseOS 63 k 2023-08-11T15:44:03.527 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid x86_64 2.37.4-13.el9 CentOS-BaseOS 28 k 2023-08-11T15:44:03.527 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.0.7-24.el9 CentOS-BaseOS 1.2 M 2023-08-11T15:44:03.527 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.0.7-24.el9 CentOS-BaseOS 2.1 M 2023-08-11T15:44:03.527 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux x86_64 2.37.4-13.el9 CentOS-BaseOS 2.2 M 2023-08-11T15:44:03.527 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-core x86_64 2.37.4-13.el9 CentOS-BaseOS 433 k 2023-08-11T15:44:03.528 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-user x86_64 2.37.4-13.el9 CentOS-BaseOS 30 k 2023-08-11T15:44:03.528 INFO:teuthology.orchestra.run.smithi027.stdout: xfsdump x86_64 3.1.12-3.el9 CentOS-BaseOS 321 k 2023-08-11T15:44:03.528 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs x86_64 5.19.0-4.el9 CentOS-BaseOS 1.0 M 2023-08-11T15:44:03.528 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2023-08-11T15:44:03.528 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-08-11T15:44:03.528 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-08-11T15:44:03.528 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:18.0.0-5437.g92684807.el9 ceph 19 M 2023-08-11T15:44:03.528 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 23 k 2023-08-11T15:44:03.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:18.0.0-5437.g92684807.el9 ceph 2.1 M 2023-08-11T15:44:03.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 242 k 2023-08-11T15:44:03.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:18.0.0-5437.g92684807.el9 ceph 4.4 M 2023-08-11T15:44:03.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:18.0.0-5437.g92684807.el9 ceph 18 M 2023-08-11T15:44:03.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 15 k 2023-08-11T15:44:03.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:18.0.0-5437.g92684807.el9 ceph 24 k 2023-08-11T15:44:03.529 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-08-11T15:44:03.530 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-08-11T15:44:03.530 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-08-11T15:44:03.530 INFO:teuthology.orchestra.run.smithi027.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-08-11T15:44:03.530 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-08-11T15:44:03.530 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-08-11T15:44:03.530 INFO:teuthology.orchestra.run.smithi027.stdout: jq x86_64 1.6-15.el9 CentOS-AppStream 186 k 2023-08-11T15:44:03.530 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-08-11T15:44:03.530 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-08-11T15:44:03.531 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:18.0.0-5437.g92684807.el9 ceph 163 k 2023-08-11T15:44:03.531 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.4.1-2.1.el9 CentOS-BaseOS 807 k 2023-08-11T15:44:03.531 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-08-11T15:44:03.531 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-08-11T15:44:03.531 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.4.1-2.1.el9 CentOS-BaseOS 197 k 2023-08-11T15:44:03.531 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-08-11T15:44:03.531 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:18.0.0-5437.g92684807.el9 ceph 472 k 2023-08-11T15:44:03.531 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-08-11T15:44:03.532 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:18.0.0-5437.g92684807.el9 ceph 4.7 M 2023-08-11T15:44:03.532 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-08-11T15:44:03.532 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-08-11T15:44:03.532 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-08-11T15:44:03.532 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-08-11T15:44:03.532 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-08-11T15:44:03.532 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-08-11T15:44:03.533 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-08-11T15:44:03.533 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-08-11T15:44:03.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-08-11T15:44:03.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-08-11T15:44:03.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-08-11T15:44:03.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-08-11T15:44:03.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:18.0.0-5437.g92684807.el9 ceph 45 k 2023-08-11T15:44:03.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:18.0.0-5437.g92684807.el9 ceph 127 k 2023-08-11T15:44:03.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-08-11T15:44:03.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-08-11T15:44:03.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-08-11T15:44:03.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-08-11T15:44:03.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-08-11T15:44:03.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.16-1.el9 appstream 247 k 2023-08-11T15:44:03.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-08-11T15:44:03.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-08-11T15:44:03.535 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-08-11T15:44:03.535 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-08-11T15:44:03.535 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-08-11T15:44:03.535 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-08-11T15:44:03.535 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-08-11T15:44:03.535 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-08-11T15:44:03.535 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-08-11T15:44:03.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-08-11T15:44:03.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-08-11T15:44:03.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-08-11T15:44:03.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-08-11T15:44:03.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-08-11T15:44:03.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-08-11T15:44:03.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-08-11T15:44:03.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-08-11T15:44:03.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-08-11T15:44:03.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-08-11T15:44:03.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-08-11T15:44:03.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-08-11T15:44:03.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-08-11T15:44:03.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-08-11T15:44:03.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-08-11T15:44:03.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-08-11T15:44:03.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-08-11T15:44:03.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-08-11T15:44:03.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-08-11T15:44:03.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-08-11T15:44:03.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-08-11T15:44:03.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-08-11T15:44:03.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-08-11T15:44:03.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-08-11T15:44:03.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-08-11T15:44:03.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-08-11T15:44:03.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-08-11T15:44:03.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-08-11T15:44:03.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-08-11T15:44:03.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-08-11T15:44:03.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-08-11T15:44:03.540 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-2.el9 epel 192 k 2023-08-11T15:44:03.540 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-08-11T15:44:03.540 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-08-11T15:44:03.540 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-08-11T15:44:03.540 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-08-11T15:44:03.540 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-36.el9 appstream 45 k 2023-08-11T15:44:03.540 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2023-08-11T15:44:03.540 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-08-11T15:44:03.541 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:44:03.541 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-08-11T15:44:03.541 INFO:teuthology.orchestra.run.smithi027.stdout:============================================================================================ 2023-08-11T15:44:03.541 INFO:teuthology.orchestra.run.smithi027.stdout:Install 131 Packages 2023-08-11T15:44:03.541 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 23 Packages 2023-08-11T15:44:03.541 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:44:03.542 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 205 M 2023-08-11T15:44:03.542 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2023-08-11T15:44:04.308 INFO:teuthology.orchestra.run.smithi027.stdout:(1/154): ceph-18.0.0-5437.g92684807.el9.x86_64. 20 kB/s | 6.5 kB 00:00 2023-08-11T15:44:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:(2/154): ceph-fuse-18.0.0-5437.g92684807.el9.x8 2.0 MB/s | 848 kB 00:00 2023-08-11T15:44:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:(3/154): ceph-immutable-object-cache-18.0.0-543 1.2 MB/s | 141 kB 00:00 2023-08-11T15:44:05.033 INFO:teuthology.orchestra.run.smithi027.stdout:(4/154): ceph-base-18.0.0-5437.g92684807.el9.x8 4.9 MB/s | 5.2 MB 00:01 2023-08-11T15:44:05.217 INFO:teuthology.orchestra.run.smithi027.stdout:(5/154): ceph-mgr-18.0.0-5437.g92684807.el9.x86 8.0 MB/s | 1.5 MB 00:00 2023-08-11T15:44:05.626 INFO:teuthology.orchestra.run.smithi027.stdout:(6/154): ceph-mds-18.0.0-5437.g92684807.el9.x86 2.7 MB/s | 2.1 MB 00:00 2023-08-11T15:44:05.759 INFO:teuthology.orchestra.run.smithi027.stdout:(7/154): ceph-mon-18.0.0-5437.g92684807.el9.x86 8.1 MB/s | 4.4 MB 00:00 2023-08-11T15:44:06.152 INFO:teuthology.orchestra.run.smithi027.stdout:(8/154): ceph-common-18.0.0-5437.g92684807.el9. 8.6 MB/s | 19 MB 00:02 2023-08-11T15:44:06.201 INFO:teuthology.orchestra.run.smithi027.stdout:(9/154): ceph-selinux-18.0.0-5437.g92684807.el9 493 kB/s | 24 kB 00:00 2023-08-11T15:44:06.552 INFO:teuthology.orchestra.run.smithi027.stdout:(10/154): ceph-radosgw-18.0.0-5437.g92684807.el 10 MB/s | 8.1 MB 00:00 2023-08-11T15:44:06.644 INFO:teuthology.orchestra.run.smithi027.stdout:(11/154): cephfs-mirror-18.0.0-5437.g92684807.e 2.4 MB/s | 218 kB 00:00 2023-08-11T15:44:06.711 INFO:teuthology.orchestra.run.smithi027.stdout:(12/154): libcephfs-devel-18.0.0-5437.g92684807 476 kB/s | 31 kB 00:00 2023-08-11T15:44:06.828 INFO:teuthology.orchestra.run.smithi027.stdout:(13/154): libcephfs2-18.0.0-5437.g92684807.el9. 5.6 MB/s | 665 kB 00:00 2023-08-11T15:44:06.911 INFO:teuthology.orchestra.run.smithi027.stdout:(14/154): libcephsqlite-18.0.0-5437.g92684807.e 1.9 MB/s | 163 kB 00:00 2023-08-11T15:44:06.995 INFO:teuthology.orchestra.run.smithi027.stdout:(15/154): librados-devel-18.0.0-5437.g92684807. 1.5 MB/s | 126 kB 00:00 2023-08-11T15:44:07.112 INFO:teuthology.orchestra.run.smithi027.stdout:(16/154): libradosstriper1-18.0.0-5437.g9268480 4.0 MB/s | 472 kB 00:00 2023-08-11T15:44:07.754 INFO:teuthology.orchestra.run.smithi027.stdout:(17/154): librgw2-18.0.0-5437.g92684807.el9.x86 7.3 MB/s | 4.7 MB 00:00 2023-08-11T15:44:07.838 INFO:teuthology.orchestra.run.smithi027.stdout:(18/154): python3-ceph-argparse-18.0.0-5437.g92 547 kB/s | 45 kB 00:00 2023-08-11T15:44:07.930 INFO:teuthology.orchestra.run.smithi027.stdout:(19/154): python3-ceph-common-18.0.0-5437.g9268 1.4 MB/s | 127 kB 00:00 2023-08-11T15:44:08.030 INFO:teuthology.orchestra.run.smithi027.stdout:(20/154): python3-cephfs-18.0.0-5437.g92684807. 1.6 MB/s | 162 kB 00:00 2023-08-11T15:44:08.130 INFO:teuthology.orchestra.run.smithi027.stdout:(21/154): python3-rados-18.0.0-5437.g92684807.e 3.2 MB/s | 321 kB 00:00 2023-08-11T15:44:08.231 INFO:teuthology.orchestra.run.smithi027.stdout:(22/154): python3-rbd-18.0.0-5437.g92684807.el9 2.9 MB/s | 297 kB 00:00 2023-08-11T15:44:08.322 INFO:teuthology.orchestra.run.smithi027.stdout:(23/154): python3-rgw-18.0.0-5437.g92684807.el9 1.1 MB/s | 99 kB 00:00 2023-08-11T15:44:08.406 INFO:teuthology.orchestra.run.smithi027.stdout:(24/154): rbd-fuse-18.0.0-5437.g92684807.el9.x8 1.0 MB/s | 85 kB 00:00 2023-08-11T15:44:08.790 INFO:teuthology.orchestra.run.smithi027.stdout:(25/154): ceph-osd-18.0.0-5437.g92684807.el9.x8 5.6 MB/s | 18 MB 00:03 2023-08-11T15:44:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:(26/154): ceph-test-18.0.0-5437.g92684807.el9.x 13 MB/s | 40 MB 00:03 2023-08-11T15:44:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:(27/154): rbd-nbd-18.0.0-5437.g92684807.el9.x86 321 kB/s | 168 kB 00:00 2023-08-11T15:44:09.332 INFO:teuthology.orchestra.run.smithi027.stdout:(28/154): ceph-grafana-dashboards-18.0.0-5437.g 399 kB/s | 23 kB 00:00 2023-08-11T15:44:09.424 INFO:teuthology.orchestra.run.smithi027.stdout:(29/154): rbd-mirror-18.0.0-5437.g92684807.el9. 3.0 MB/s | 3.0 MB 00:01 2023-08-11T15:44:09.475 INFO:teuthology.orchestra.run.smithi027.stdout:(30/154): ceph-mgr-cephadm-18.0.0-5437.g9268480 826 kB/s | 131 kB 00:00 2023-08-11T15:44:09.583 INFO:teuthology.orchestra.run.smithi027.stdout:(31/154): ceph-mgr-dashboard-18.0.0-5437.g92684 7.0 MB/s | 1.7 MB 00:00 2023-08-11T15:44:09.625 INFO:teuthology.orchestra.run.smithi027.stdout:(32/154): ceph-mgr-modules-core-18.0.0-5437.g92 1.6 MB/s | 242 kB 00:00 2023-08-11T15:44:09.651 INFO:teuthology.orchestra.run.smithi027.stdout:(33/154): ceph-mgr-rook-18.0.0-5437.g92684807.e 721 kB/s | 47 kB 00:00 2023-08-11T15:44:09.692 INFO:teuthology.orchestra.run.smithi027.stdout:(34/154): ceph-prometheus-alerts-18.0.0-5437.g9 223 kB/s | 15 kB 00:00 2023-08-11T15:44:09.751 INFO:teuthology.orchestra.run.smithi027.stdout:(35/154): ceph-volume-18.0.0-5437.g92684807.el9 2.6 MB/s | 261 kB 00:00 2023-08-11T15:44:09.801 INFO:teuthology.orchestra.run.smithi027.stdout:(36/154): cephadm-18.0.0-5437.g92684807.el9.noa 2.0 MB/s | 224 kB 00:00 2023-08-11T15:44:09.826 INFO:teuthology.orchestra.run.smithi027.stdout:(37/154): cephfs-top-18.0.0-5437.g92684807.el9. 244 kB/s | 18 kB 00:00 2023-08-11T15:44:10.127 INFO:teuthology.orchestra.run.smithi027.stdout:(38/154): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-08-11T15:44:10.177 INFO:teuthology.orchestra.run.smithi027.stdout:(39/154): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 662 kB 00:00 2023-08-11T15:44:10.252 INFO:teuthology.orchestra.run.smithi027.stdout:(40/154): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 920 kB 00:00 2023-08-11T15:44:10.286 INFO:teuthology.orchestra.run.smithi027.stdout:(41/154): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 106 kB 00:00 2023-08-11T15:44:10.311 INFO:teuthology.orchestra.run.smithi027.stdout:(42/154): elfutils-libelf-devel-0.189-3.el9.x86 170 kB/s | 21 kB 00:00 2023-08-11T15:44:10.356 INFO:teuthology.orchestra.run.smithi027.stdout:(43/154): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 307 kB 00:00 2023-08-11T15:44:10.395 INFO:teuthology.orchestra.run.smithi027.stdout:(44/154): flexiblas-3.0.4-8.el9.x86_64.rpm 401 kB/s | 32 kB 00:00 2023-08-11T15:44:10.529 INFO:teuthology.orchestra.run.smithi027.stdout:(45/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2023-08-11T15:44:10.554 INFO:teuthology.orchestra.run.smithi027.stdout:(46/154): flexiblas-openblas-openmp-3.0.4-8.el9 91 kB/s | 17 kB 00:00 2023-08-11T15:44:10.604 INFO:teuthology.orchestra.run.smithi027.stdout:(47/154): jq-1.6-15.el9.x86_64.rpm 965 kB/s | 186 kB 00:00 2023-08-11T15:44:10.630 INFO:teuthology.orchestra.run.smithi027.stdout:(48/154): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 10 kB 00:00 2023-08-11T15:44:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:(49/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2023-08-11T15:44:10.680 INFO:teuthology.orchestra.run.smithi027.stdout:(50/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2023-08-11T15:44:10.747 INFO:teuthology.orchestra.run.smithi027.stdout:(51/154): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 661 kB 00:00 2023-08-11T15:44:10.801 INFO:teuthology.orchestra.run.smithi027.stdout:(52/154): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 578 kB 00:00 2023-08-11T15:44:10.823 INFO:teuthology.orchestra.run.smithi027.stdout:(53/154): libuuid-devel-2.37.4-13.el9.x86_64.rp 193 kB/s | 25 kB 00:00 2023-08-11T15:44:10.849 INFO:teuthology.orchestra.run.smithi027.stdout:(54/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 474 kB/s | 46 kB 00:00 2023-08-11T15:44:10.915 INFO:teuthology.orchestra.run.smithi027.stdout:(55/154): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.4 MB/s | 217 kB 00:00 2023-08-11T15:44:10.973 INFO:teuthology.orchestra.run.smithi027.stdout:(56/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 282 kB 00:00 2023-08-11T15:44:10.998 INFO:teuthology.orchestra.run.smithi027.stdout:(57/154): openblas-0.3.21-2.el9.x86_64.rpm 228 kB/s | 34 kB 00:00 2023-08-11T15:44:11.224 INFO:teuthology.orchestra.run.smithi027.stdout:(58/154): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2023-08-11T15:44:11.316 INFO:teuthology.orchestra.run.smithi027.stdout:(59/154): postgresql-13.11-1.el9.x86_64.rpm 5.1 MB/s | 1.5 MB 00:00 2023-08-11T15:44:11.425 INFO:teuthology.orchestra.run.smithi027.stdout:(60/154): openssl-devel-3.0.7-24.el9.x86_64.rpm 7.3 MB/s | 3.0 MB 00:00 2023-08-11T15:44:11.475 INFO:teuthology.orchestra.run.smithi027.stdout:(61/154): postgresql-contrib-13.11-1.el9.x86_64 3.2 MB/s | 811 kB 00:00 2023-08-11T15:44:11.516 INFO:teuthology.orchestra.run.smithi027.stdout:(62/154): postgresql-private-libs-13.11-1.el9.x 738 kB/s | 134 kB 00:00 2023-08-11T15:44:11.817 INFO:teuthology.orchestra.run.smithi027.stdout:(63/154): python3-jinja2-2.11.3-4.el9.noarch.rp 781 kB/s | 228 kB 00:00 2023-08-11T15:44:11.950 INFO:teuthology.orchestra.run.smithi027.stdout:(64/154): postgresql-server-13.11-1.el9.x86_64. 12 MB/s | 5.7 MB 00:00 2023-08-11T15:44:12.084 INFO:teuthology.orchestra.run.smithi027.stdout:(65/154): python3-babel-2.9.1-2.el9.noarch.rpm 9.5 MB/s | 5.8 MB 00:00 2023-08-11T15:44:12.109 INFO:teuthology.orchestra.run.smithi027.stdout:(66/154): python3-jsonpatch-1.21-16.el9.noarch. 87 kB/s | 24 kB 00:00 2023-08-11T15:44:12.134 INFO:teuthology.orchestra.run.smithi027.stdout:(67/154): python3-jsonpointer-2.0-4.el9.noarch. 372 kB/s | 18 kB 00:00 2023-08-11T15:44:12.168 INFO:teuthology.orchestra.run.smithi027.stdout:(68/154): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2023-08-11T15:44:12.193 INFO:teuthology.orchestra.run.smithi027.stdout:(69/154): python3-markupsafe-1.1.1-12.el9.x86_6 382 kB/s | 32 kB 00:00 2023-08-11T15:44:12.252 INFO:teuthology.orchestra.run.smithi027.stdout:(70/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.6 MB/s | 217 kB 00:00 2023-08-11T15:44:12.410 INFO:teuthology.orchestra.run.smithi027.stdout:(71/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2023-08-11T15:44:12.444 INFO:teuthology.orchestra.run.smithi027.stdout:(72/154): python3-oauthlib-3.1.1-5.el9.noarch.r 699 kB/s | 174 kB 00:00 2023-08-11T15:44:12.469 INFO:teuthology.orchestra.run.smithi027.stdout:(73/154): python3-prettytable-0.7.2-27.el9.noar 208 kB/s | 41 kB 00:00 2023-08-11T15:44:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:(74/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 132 kB 00:00 2023-08-11T15:44:12.545 INFO:teuthology.orchestra.run.smithi027.stdout:(75/154): python3-pytz-2021.1-5.el9.noarch.rpm 646 kB/s | 47 kB 00:00 2023-08-11T15:44:12.587 INFO:teuthology.orchestra.run.smithi027.stdout:(76/154): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 211 kB 00:00 2023-08-11T15:44:12.612 INFO:teuthology.orchestra.run.smithi027.stdout:(77/154): python3-requests-oauthlib-1.3.0-12.el 473 kB/s | 43 kB 00:00 2023-08-11T15:44:13.087 INFO:teuthology.orchestra.run.smithi027.stdout:(78/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2023-08-11T15:44:13.121 INFO:teuthology.orchestra.run.smithi027.stdout:(79/154): python3-toml-0.10.2-6.el9.noarch.rpm 79 kB/s | 39 kB 00:00 2023-08-11T15:44:13.163 INFO:teuthology.orchestra.run.smithi027.stdout:(80/154): socat-1.7.4.1-5.el9.x86_64.rpm 565 kB/s | 300 kB 00:00 2023-08-11T15:44:13.188 INFO:teuthology.orchestra.run.smithi027.stdout:(81/154): uuid-1.6.2-55.el9.x86_64.rpm 588 kB/s | 56 kB 00:00 2023-08-11T15:44:13.222 INFO:teuthology.orchestra.run.smithi027.stdout:(82/154): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.5 MB/s | 85 kB 00:00 2023-08-11T15:44:13.247 INFO:teuthology.orchestra.run.smithi027.stdout:(83/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 767 kB/s | 63 kB 00:00 2023-08-11T15:44:13.289 INFO:teuthology.orchestra.run.smithi027.stdout:(84/154): fuse-2.9.9-15.el9.x86_64.rpm 791 kB/s | 79 kB 00:00 2023-08-11T15:44:13.330 INFO:teuthology.orchestra.run.smithi027.stdout:(85/154): libquadmath-11.4.1-2.1.el9.x86_64.rpm 2.4 MB/s | 197 kB 00:00 2023-08-11T15:44:13.397 INFO:teuthology.orchestra.run.smithi027.stdout:(86/154): libgfortran-11.4.1-2.1.el9.x86_64.rpm 5.3 MB/s | 807 kB 00:00 2023-08-11T15:44:13.431 INFO:teuthology.orchestra.run.smithi027.stdout:(87/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 241 kB 00:00 2023-08-11T15:44:13.498 INFO:teuthology.orchestra.run.smithi027.stdout:(88/154): python3-cryptography-36.0.1-4.el9.x86 7.4 MB/s | 1.1 MB 00:00 2023-08-11T15:44:13.549 INFO:teuthology.orchestra.run.smithi027.stdout:(89/154): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2023-08-11T15:44:13.582 INFO:teuthology.orchestra.run.smithi027.stdout:(90/154): python3-ply-3.11-14.el9.noarch.rpm 756 kB/s | 103 kB 00:00 2023-08-11T15:44:13.707 INFO:teuthology.orchestra.run.smithi027.stdout:(91/154): zlib-devel-1.2.11-36.el9.x86_64.rpm 288 kB/s | 45 kB 00:00 2023-08-11T15:44:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:(92/154): python3-devel-3.9.16-1.el9.x86_64.rpm 824 kB/s | 247 kB 00:00 2023-08-11T15:44:13.833 INFO:teuthology.orchestra.run.smithi027.stdout:(93/154): fmt-8.1.1-5.el9.x86_64.rpm 444 kB/s | 111 kB 00:00 2023-08-11T15:44:13.908 INFO:teuthology.orchestra.run.smithi027.stdout:(94/154): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.5 MB/s | 309 kB 00:00 2023-08-11T15:44:13.942 INFO:teuthology.orchestra.run.smithi027.stdout:(95/154): libarrow-doc-9.0.0-9.el9.noarch.rpm 228 kB/s | 24 kB 00:00 2023-08-11T15:44:14.017 INFO:teuthology.orchestra.run.smithi027.stdout:(96/154): liboath-2.6.7-2.el9.x86_64.rpm 502 kB/s | 49 kB 00:00 2023-08-11T15:44:14.134 INFO:teuthology.orchestra.run.smithi027.stdout:(97/154): libarrow-9.0.0-9.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2023-08-11T15:44:14.176 INFO:teuthology.orchestra.run.smithi027.stdout:(98/154): libunwind-1.6.2-1.el9.x86_64.rpm 315 kB/s | 67 kB 00:00 2023-08-11T15:44:14.243 INFO:teuthology.orchestra.run.smithi027.stdout:(99/154): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.7 MB/s | 837 kB 00:00 2023-08-11T15:44:14.301 INFO:teuthology.orchestra.run.smithi027.stdout:(100/154): python3-asyncssh-2.13.2-3.el9.noarch 4.7 MB/s | 548 kB 00:00 2023-08-11T15:44:14.326 INFO:teuthology.orchestra.run.smithi027.stdout:(101/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 312 kB/s | 43 kB 00:00 2023-08-11T15:44:14.352 INFO:teuthology.orchestra.run.smithi027.stdout:(102/154): python3-cachetools-4.2.4-1.el9.noarc 319 kB/s | 32 kB 00:00 2023-08-11T15:44:14.377 INFO:teuthology.orchestra.run.smithi027.stdout:(103/154): python3-certifi-2021.10.8-2.el9.noar 308 kB/s | 15 kB 00:00 2023-08-11T15:44:14.411 INFO:teuthology.orchestra.run.smithi027.stdout:(104/154): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2023-08-11T15:44:14.461 INFO:teuthology.orchestra.run.smithi027.stdout:(105/154): python3-cherrypy-18.6.1-2.el9.noarch 3.3 MB/s | 358 kB 00:00 2023-08-11T15:44:14.511 INFO:teuthology.orchestra.run.smithi027.stdout:(106/154): python3-google-auth-2.22.0-1.el9.noa 2.0 MB/s | 197 kB 00:00 2023-08-11T15:44:14.536 INFO:teuthology.orchestra.run.smithi027.stdout:(107/154): python3-jaraco-8.2.1-3.el9.noarch.rp 89 kB/s | 11 kB 00:00 2023-08-11T15:44:14.566 INFO:teuthology.orchestra.run.smithi027.stdout:(108/154): python3-jaraco-classes-3.2.1-5.el9.n 182 kB/s | 18 kB 00:00 2023-08-11T15:44:14.587 INFO:teuthology.orchestra.run.smithi027.stdout:(109/154): python3-jaraco-collections-3.0.0-8.e 474 kB/s | 23 kB 00:00 2023-08-11T15:44:14.612 INFO:teuthology.orchestra.run.smithi027.stdout:(110/154): python3-jaraco-functools-3.5.0-2.el9 264 kB/s | 19 kB 00:00 2023-08-11T15:44:14.637 INFO:teuthology.orchestra.run.smithi027.stdout:(111/154): python3-jaraco-text-3.2.0-6.el9.noar 266 kB/s | 20 kB 00:00 2023-08-11T15:44:14.671 INFO:teuthology.orchestra.run.smithi027.stdout:(112/154): python3-jwt+crypto-2.4.0-1.el9.noarc 157 kB/s | 9.0 kB 00:00 2023-08-11T15:44:14.717 INFO:teuthology.orchestra.run.smithi027.stdout:(113/154): python3-jwt-2.4.0-1.el9.noarch.rpm 399 kB/s | 41 kB 00:00 2023-08-11T15:44:14.781 INFO:teuthology.orchestra.run.smithi027.stdout:(114/154): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2023-08-11T15:44:14.805 INFO:teuthology.orchestra.run.smithi027.stdout:(115/154): python3-logutils-0.3.5-21.el9.noarch 614 kB/s | 46 kB 00:00 2023-08-11T15:44:14.839 INFO:teuthology.orchestra.run.smithi027.stdout:(116/154): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2023-08-11T15:44:14.872 INFO:teuthology.orchestra.run.smithi027.stdout:(117/154): python3-natsort-7.1.1-5.el9.noarch.r 643 kB/s | 58 kB 00:00 2023-08-11T15:44:14.922 INFO:teuthology.orchestra.run.smithi027.stdout:(118/154): python3-pecan-1.4.2-3.el9.noarch.rpm 3.3 MB/s | 272 kB 00:00 2023-08-11T15:44:14.948 INFO:teuthology.orchestra.run.smithi027.stdout:(119/154): python3-portend-3.1.0-2.el9.noarch.r 154 kB/s | 16 kB 00:00 2023-08-11T15:44:14.990 INFO:teuthology.orchestra.run.smithi027.stdout:(120/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 797 kB/s | 90 kB 00:00 2023-08-11T15:44:15.015 INFO:teuthology.orchestra.run.smithi027.stdout:(121/154): python3-repoze-lru-0.7-16.el9.noarch 475 kB/s | 31 kB 00:00 2023-08-11T15:44:15.049 INFO:teuthology.orchestra.run.smithi027.stdout:(122/154): python3-routes-2.5.1-5.el9.noarch.rp 1.9 MB/s | 188 kB 00:00 2023-08-11T15:44:15.074 INFO:teuthology.orchestra.run.smithi027.stdout:(123/154): python3-rsa-4.9-2.el9.noarch.rpm 738 kB/s | 59 kB 00:00 2023-08-11T15:44:15.099 INFO:teuthology.orchestra.run.smithi027.stdout:(124/154): python3-tempora-5.0.0-2.el9.noarch.r 740 kB/s | 36 kB 00:00 2023-08-11T15:44:15.124 INFO:teuthology.orchestra.run.smithi027.stdout:(125/154): python3-typing-extensions-4.4.0-2.el 695 kB/s | 51 kB 00:00 2023-08-11T15:44:15.175 INFO:teuthology.orchestra.run.smithi027.stdout:(126/154): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2023-08-11T15:44:15.217 INFO:teuthology.orchestra.run.smithi027.stdout:(127/154): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2023-08-11T15:44:15.258 INFO:teuthology.orchestra.run.smithi027.stdout:(128/154): python3-werkzeug-2.0.3-3.el9.1.noarc 3.2 MB/s | 427 kB 00:00 2023-08-11T15:44:15.284 INFO:teuthology.orchestra.run.smithi027.stdout:(129/154): python3-zc-lockfile-2.0-10.el9.noarc 196 kB/s | 20 kB 00:00 2023-08-11T15:44:15.342 INFO:teuthology.orchestra.run.smithi027.stdout:(130/154): re2-20211101-2.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2023-08-11T15:44:15.419 INFO:teuthology.orchestra.run.smithi027.stdout:(131/154): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2023-08-11T15:44:15.543 INFO:teuthology.orchestra.run.smithi027.stdout:(132/154): NetworkManager-1.43.90-1.el9.x86_64. 18 MB/s | 2.2 MB 00:00 2023-08-11T15:44:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:(133/154): NetworkManager-libnm-1.43.90-1.el9.x 17 MB/s | 1.8 MB 00:00 2023-08-11T15:44:15.693 INFO:teuthology.orchestra.run.smithi027.stdout:(134/154): NetworkManager-team-1.43.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2023-08-11T15:44:15.744 INFO:teuthology.orchestra.run.smithi027.stdout:(135/154): NetworkManager-tui-1.43.90-1.el9.x86 4.9 MB/s | 245 kB 00:00 2023-08-11T15:44:15.778 INFO:teuthology.orchestra.run.smithi027.stdout:(136/154): elfutils-debuginfod-client-0.189-3.e 1.1 MB/s | 36 kB 00:00 2023-08-11T15:44:15.828 INFO:teuthology.orchestra.run.smithi027.stdout:(137/154): elfutils-libelf-0.189-3.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2023-08-11T15:44:15.920 INFO:teuthology.orchestra.run.smithi027.stdout:(138/154): librbd1-18.0.0-5437.g92684807.el9.x8 5.4 MB/s | 3.0 MB 00:00 2023-08-11T15:44:15.961 INFO:teuthology.orchestra.run.smithi027.stdout:(139/154): elfutils-libs-0.189-3.el9.x86_64.rpm 1.9 MB/s | 254 kB 00:00 2023-08-11T15:44:16.012 INFO:teuthology.orchestra.run.smithi027.stdout:(140/154): iproute-6.2.0-5.el9.x86_64.rpm 8.7 MB/s | 785 kB 00:00 2023-08-11T15:44:16.054 INFO:teuthology.orchestra.run.smithi027.stdout:(141/154): iproute-tc-6.2.0-5.el9.x86_64.rpm 5.4 MB/s | 448 kB 00:00 2023-08-11T15:44:16.104 INFO:teuthology.orchestra.run.smithi027.stdout:(142/154): libblkid-2.37.4-13.el9.x86_64.rpm 1.2 MB/s | 107 kB 00:00 2023-08-11T15:44:16.146 INFO:teuthology.orchestra.run.smithi027.stdout:(143/154): libfdisk-2.37.4-13.el9.x86_64.rpm 1.7 MB/s | 154 kB 00:00 2023-08-11T15:44:16.188 INFO:teuthology.orchestra.run.smithi027.stdout:(144/154): libmount-2.37.4-13.el9.x86_64.rpm 1.6 MB/s | 133 kB 00:00 2023-08-11T15:44:16.214 INFO:teuthology.orchestra.run.smithi027.stdout:(145/154): libsmartcols-2.37.4-13.el9.x86_64.rp 972 kB/s | 63 kB 00:00 2023-08-11T15:44:16.239 INFO:teuthology.orchestra.run.smithi027.stdout:(146/154): libuuid-2.37.4-13.el9.x86_64.rpm 576 kB/s | 28 kB 00:00 2023-08-11T15:44:16.306 INFO:teuthology.orchestra.run.smithi027.stdout:(147/154): openssl-3.0.7-24.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2023-08-11T15:44:16.406 INFO:teuthology.orchestra.run.smithi027.stdout:(148/154): openssl-libs-3.0.7-24.el9.x86_64.rpm 14 MB/s | 2.1 MB 00:00 2023-08-11T15:44:16.506 INFO:teuthology.orchestra.run.smithi027.stdout:(149/154): util-linux-2.37.4-13.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2023-08-11T15:44:16.557 INFO:teuthology.orchestra.run.smithi027.stdout:(150/154): util-linux-core-2.37.4-13.el9.x86_64 3.2 MB/s | 433 kB 00:00 2023-08-11T15:44:16.582 INFO:teuthology.orchestra.run.smithi027.stdout:(151/154): util-linux-user-2.37.4-13.el9.x86_64 431 kB/s | 30 kB 00:00 2023-08-11T15:44:16.657 INFO:teuthology.orchestra.run.smithi027.stdout:(152/154): xfsdump-3.1.12-3.el9.x86_64.rpm 3.2 MB/s | 321 kB 00:00 2023-08-11T15:44:16.740 INFO:teuthology.orchestra.run.smithi027.stdout:(153/154): xfsprogs-5.19.0-4.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2023-08-11T15:44:16.849 INFO:teuthology.orchestra.run.smithi027.stdout:(154/154): librados2-18.0.0-5437.g92684807.el9. 2.1 MB/s | 3.3 MB 00:01 2023-08-11T15:44:16.861 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:44:16.862 INFO:teuthology.orchestra.run.smithi027.stdout:Total 15 MB/s | 205 MB 00:13 2023-08-11T15:44:16.963 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2023-08-11T15:44:17.303 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2023-08-11T15:44:17.305 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2023-08-11T15:44:19.614 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2023-08-11T15:44:19.615 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2023-08-11T15:44:26.248 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2023-08-11T15:44:26.364 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.0.7-24.el9.x86_64 1/177 2023-08-11T15:44:26.412 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libuuid-2.37.4-13.el9.x86_64 2/177 2023-08-11T15:44:26.451 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libblkid-2.37.4-13.el9.x86_64 3/177 2023-08-11T15:44:26.575 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libblkid-2.37.4-13.el9.x86_64 3/177 2023-08-11T15:44:26.643 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/177 2023-08-11T15:44:26.710 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libelf-0.189-3.el9.x86_64 5/177 2023-08-11T15:44:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/177 2023-08-11T15:44:26.981 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 7/177 2023-08-11T15:44:27.027 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/177 2023-08-11T15:44:27.133 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/177 2023-08-11T15:44:27.590 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/177 2023-08-11T15:44:27.632 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:18.0.0-5437.g92684807.el9.x86_64 11/177 2023-08-11T15:44:27.786 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:18.0.0-5437.g92684807.el9.x86_64 11/177 2023-08-11T15:44:27.941 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:18.0.0-5437.g92684807.el9.x86_64 12/177 2023-08-11T15:44:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:18.0.0-5437.g92684807.el9.x86_64 12/177 2023-08-11T15:44:28.117 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:18.0.0-5437.g92684807.el9.x86_64 13/177 2023-08-11T15:44:28.401 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/177 2023-08-11T15:44:28.453 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/177 2023-08-11T15:44:28.487 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:18.0.0-5437.g92684807.el9.x86_64 16/177 2023-08-11T15:44:28.646 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5437.g92684807.el9.x86_64 16/177 2023-08-11T15:44:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:18.0.0-5437.g92684807.el9.x86 17/177 2023-08-11T15:44:28.806 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5437.g92684807.el9.x86 17/177 2023-08-11T15:44:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libmount-2.37.4-13.el9.x86_64 18/177 2023-08-11T15:44:29.242 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libsmartcols-2.37.4-13.el9.x86_64 19/177 2023-08-11T15:44:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-libnm-1:1.43.90-1.el9.x86_64 20/177 2023-08-11T15:44:29.457 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.43.90-1.el9.x86_64 20/177 2023-08-11T15:44:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-11T15:44:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-11T15:44:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-11T15:44:30.318 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/177 2023-08-11T15:44:30.358 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/177 2023-08-11T15:44:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 24/177 2023-08-11T15:44:30.486 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 25/177 2023-08-11T15:44:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/177 2023-08-11T15:44:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/177 2023-08-11T15:44:30.677 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/177 2023-08-11T15:44:30.789 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/177 2023-08-11T15:44:30.827 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:18.0.0-5437.g92684807.el9. 30/177 2023-08-11T15:44:30.878 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 31/177 2023-08-11T15:44:30.998 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:18.0.0-5437.g92684807.el9.x86_6 32/177 2023-08-11T15:44:31.219 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/177 2023-08-11T15:44:31.279 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/177 2023-08-11T15:44:32.029 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/177 2023-08-11T15:44:32.132 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/177 2023-08-11T15:44:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 37/177 2023-08-11T15:44:32.309 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : util-linux-core-2.37.4-13.el9.x86_64 38/177 2023-08-11T15:44:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: util-linux-core-2.37.4-13.el9.x86_64 38/177 2023-08-11T15:44:32.394 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:18.0.0-5437.g92684807.el9.x86_6 39/177 2023-08-11T15:44:32.440 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 40/177 2023-08-11T15:44:32.606 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libs-0.189-3.el9.x86_64 41/177 2023-08-11T15:44:32.688 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : iproute-6.2.0-5.el9.x86_64 42/177 2023-08-11T15:44:33.045 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libfdisk-2.37.4-13.el9.x86_64 43/177 2023-08-11T15:44:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : util-linux-2.37.4-13.el9.x86_64 44/177 2023-08-11T15:44:33.372 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : xfsprogs-5.19.0-4.el9.x86_64 45/177 2023-08-11T15:44:33.423 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-13.el9.x86_64 46/177 2023-08-11T15:44:33.632 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/177 2023-08-11T15:44:33.683 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.0.7-24.el9.x86_64 48/177 2023-08-11T15:44:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-2.el9.x86_64 49/177 2023-08-11T15:44:33.886 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 50/177 2023-08-11T15:44:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/177 2023-08-11T15:44:34.080 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/177 2023-08-11T15:44:34.120 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/177 2023-08-11T15:44:34.151 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/177 2023-08-11T15:44:34.220 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/177 2023-08-11T15:44:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/177 2023-08-11T15:44:34.347 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/177 2023-08-11T15:44:34.547 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/177 2023-08-11T15:44:34.594 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/177 2023-08-11T15:44:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/177 2023-08-11T15:44:34.780 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/177 2023-08-11T15:44:34.826 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 62/177 2023-08-11T15:44:34.866 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/177 2023-08-11T15:44:34.913 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/177 2023-08-11T15:44:34.946 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 65/177 2023-08-11T15:44:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fmt-8.1.1-5.el9.x86_64 66/177 2023-08-11T15:44:35.166 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-36.el9.x86_64 67/177 2023-08-11T15:44:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.16-1.el9.x86_64 68/177 2023-08-11T15:44:35.299 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 69/177 2023-08-11T15:44:35.388 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 70/177 2023-08-11T15:44:35.657 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 71/177 2023-08-11T15:44:35.719 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 72/177 2023-08-11T15:44:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 73/177 2023-08-11T15:44:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 74/177 2023-08-11T15:44:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 75/177 2023-08-11T15:44:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 76/177 2023-08-11T15:44:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 77/177 2023-08-11T15:44:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-15.el9.x86_64 78/177 2023-08-11T15:44:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/177 2023-08-11T15:44:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/177 2023-08-11T15:44:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/177 2023-08-11T15:44:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/177 2023-08-11T15:44:36.493 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/177 2023-08-11T15:44:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/177 2023-08-11T15:44:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/177 2023-08-11T15:44:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/177 2023-08-11T15:44:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/177 2023-08-11T15:44:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/177 2023-08-11T15:44:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/177 2023-08-11T15:44:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 90/177 2023-08-11T15:44:38.119 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/177 2023-08-11T15:44:38.177 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 92/177 2023-08-11T15:44:38.209 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 93/177 2023-08-11T15:44:38.244 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 94/177 2023-08-11T15:44:38.522 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/177 2023-08-11T15:44:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/177 2023-08-11T15:44:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/177 2023-08-11T15:44:39.517 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/177 2023-08-11T15:44:39.670 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/177 2023-08-11T15:44:41.347 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/177 2023-08-11T15:44:41.452 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/177 2023-08-11T15:44:41.489 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 102/177 2023-08-11T15:44:41.520 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : jq-1.6-15.el9.x86_64 103/177 2023-08-11T15:44:41.562 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 104/177 2023-08-11T15:44:41.980 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/177 2023-08-11T15:44:42.016 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:18.0.0-5437.g92684807.el9.x86_64 106/177 2023-08-11T15:44:42.121 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:18.0.0-5437.g92684807.el9.x86_64 106/177 2023-08-11T15:44:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:18.0.0-5437.g92684807.el9.x86_64 107/177 2023-08-11T15:44:42.212 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/177 2023-08-11T15:44:42.722 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 109/177 2023-08-11T15:44:42.752 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:18.0.0-5437.g92684807.el9.x86_64 110/177 2023-08-11T15:44:42.853 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:18.0.0-5437.g92684807.el9.x86_64 110/177 2023-08-11T15:44:42.880 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:18.0.0-5437.g92684807.el9.x86_64 111/177 2023-08-11T15:44:45.523 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 112/177 2023-08-11T15:44:45.750 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 112/177 2023-08-11T15:44:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 112/177 2023-08-11T15:44:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:18.0.0-5437.g92684807.el9.x86_64 113/177 2023-08-11T15:44:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:18.0.0-5437.g92684807.el9.x86_64 113/177 2023-08-11T15:44:46.707 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-08-11T15:44:46.707 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-08-11T15:44:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:44:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:18.0.0-5437.g92684807.el9.x86_64 114/177 2023-08-11T15:44:59.897 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5437.g92684807.el9.x86_64 114/177 2023-08-11T15:44:59.898 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2023-08-11T15:44:59.899 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2023-08-11T15:44:59.899 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2023-08-11T15:44:59.899 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2023-08-11T15:44:59.899 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2023-08-11T15:44:59.899 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2023-08-11T15:44:59.900 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2023-08-11T15:44:59.900 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:00.133 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:18.0.0-5437.g92684807.el9.noarch 115/177 2023-08-11T15:45:00.190 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:18.0.0-5437.g92684807.el9.noarch 115/177 2023-08-11T15:45:00.191 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:45:00.191 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-08-11T15:45:00.191 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:18.0.0-5437.g92684807.el9.x86_64 116/177 2023-08-11T15:45:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:18.0.0-5437.g92684807.el9.x86_64 116/177 2023-08-11T15:45:02.098 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:45:02.098 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-08-11T15:45:02.099 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-11T15:45:02.099 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-11T15:45:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:02.342 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:18.0.0-5437.g92684807.el9.x86_64 117/177 2023-08-11T15:45:02.425 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:18.0.0-5437.g92684807.el9.x86_64 117/177 2023-08-11T15:45:02.426 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:45:02.426 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-08-11T15:45:02.426 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-11T15:45:02.426 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-11T15:45:02.426 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:02.980 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:18.0.0-5437.g92684807.el9.x86_64 118/177 2023-08-11T15:45:03.069 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:18.0.0-5437.g92684807.el9.x86_64 118/177 2023-08-11T15:45:03.070 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:45:03.070 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-08-11T15:45:03.070 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-11T15:45:03.070 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-11T15:45:03.070 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:03.194 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 119/177 2023-08-11T15:45:03.832 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:18.0.0-5437.g92684807.el9.noarch 120/177 2023-08-11T15:45:03.873 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:18.0.0-5437.g92684807.el9.noarch 120/177 2023-08-11T15:45:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 121/177 2023-08-11T15:45:04.054 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 122/177 2023-08-11T15:45:04.083 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el9.noa 123/177 2023-08-11T15:45:04.423 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el9.noa 123/177 2023-08-11T15:45:04.467 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el9.n 124/177 2023-08-11T15:45:05.702 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el9.n 124/177 2023-08-11T15:45:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 125/177 2023-08-11T15:45:05.886 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 125/177 2023-08-11T15:45:06.069 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 126/177 2023-08-11T15:45:06.103 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:18.0.0-5437.g92684807.el9.x86_64 127/177 2023-08-11T15:45:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5437.g92684807.el9.x86_64 127/177 2023-08-11T15:45:06.197 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:45:06.197 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-08-11T15:45:06.197 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-11T15:45:06.197 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-11T15:45:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:18.0.0-5437.g92684807.el9.noarch 128/177 2023-08-11T15:45:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5437.g92684807.el9.noarch 128/177 2023-08-11T15:45:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:18.0.0-5437.g92684807.el9.x86_64 129/177 2023-08-11T15:45:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/177 2023-08-11T15:45:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:18.0.0-5437.g92684807.el9.x86_64 131/177 2023-08-11T15:45:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5437.g92684807.el9.x86_64 131/177 2023-08-11T15:45:07.402 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:45:07.402 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-08-11T15:45:07.402 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-11T15:45:07.402 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-11T15:45:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5437.g92684 132/177 2023-08-11T15:45:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5437.g92684 132/177 2023-08-11T15:45:07.545 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:45:07.545 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-08-11T15:45:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:18.0.0-5437.g92684807.el9.x86_64 133/177 2023-08-11T15:45:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5437.g92684807.el9.x86_64 133/177 2023-08-11T15:45:07.714 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:45:07.714 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-08-11T15:45:07.714 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-11T15:45:07.714 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-11T15:45:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:18.0.0-5437.g92684807.el9.x86_64 134/177 2023-08-11T15:45:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5437.g92684807.el9.x86_64 134/177 2023-08-11T15:45:08.227 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:45:08.228 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-08-11T15:45:08.228 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-11T15:45:08.228 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-11T15:45:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:12.707 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:18.0.0-5437.g92684807.el9.x86_64 135/177 2023-08-11T15:45:12.764 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:18.0.0-5437.g92684807.el9.x86_64 136/177 2023-08-11T15:45:12.803 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:18.0.0-5437.g92684807.el9.x86_64 137/177 2023-08-11T15:45:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 138/177 2023-08-11T15:45:13.224 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 139/177 2023-08-11T15:45:13.270 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:18.0.0-5437.g92684807.el9.x86_64 140/177 2023-08-11T15:45:13.329 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 141/177 2023-08-11T15:45:13.393 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : xfsdump-3.1.12-3.el9.x86_64 142/177 2023-08-11T15:45:13.502 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : util-linux-user-2.37.4-13.el9.x86_64 143/177 2023-08-11T15:45:13.579 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : iproute-tc-6.2.0-5.el9.x86_64 144/177 2023-08-11T15:45:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el9.x86_64 145/177 2023-08-11T15:45:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:18.0.0-5437.g92684807.el9.x86_ 146/177 2023-08-11T15:45:13.716 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-team-1:1.43.90-1.el9.x86_64 147/177 2023-08-11T15:45:13.785 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-tui-1:1.43.90-1.el9.x86_64 148/177 2023-08-11T15:45:14.737 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-11T15:45:14.781 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-11T15:45:14.843 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-11T15:45:16.290 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:18.0.0-5437.g92684807.el9.noarch 150/177 2023-08-11T15:45:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 151/177 2023-08-11T15:45:16.434 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/177 2023-08-11T15:45:16.572 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/177 2023-08-11T15:45:16.615 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/177 2023-08-11T15:45:16.655 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-tui-1:1.41.8-1.el9.x86_64 155/177 2023-08-11T15:45:16.692 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-debuginfod-client-0.188-3.el9.x86_64 156/177 2023-08-11T15:45:16.734 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libs-0.188-3.el9.x86_64 157/177 2023-08-11T15:45:16.734 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-11T15:45:16.897 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-11T15:45:16.945 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.0.7-2.el9.x86_64 159/177 2023-08-11T15:45:16.945 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-11T15:45:17.084 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-11T15:45:17.129 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : xfsdump-3.1.10-1.el9.x86_64 161/177 2023-08-11T15:45:17.176 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : xfsprogs-5.14.2-1.el9.x86_64 162/177 2023-08-11T15:45:17.223 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : iproute-tc-6.0.0-2.el9.x86_64 163/177 2023-08-11T15:45:17.273 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : iproute-6.0.0-2.el9.x86_64 164/177 2023-08-11T15:45:17.348 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : util-linux-user-2.37.4-9.el9.x86_64 165/177 2023-08-11T15:45:17.437 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : util-linux-2.37.4-9.el9.x86_64 166/177 2023-08-11T15:45:17.486 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : util-linux-core-2.37.4-9.el9.x86_64 167/177 2023-08-11T15:45:17.519 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libfdisk-2.37.4-9.el9.x86_64 168/177 2023-08-11T15:45:17.552 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libmount-2.37.4-9.el9.x86_64 169/177 2023-08-11T15:45:17.586 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libblkid-2.37.4-9.el9.x86_64 170/177 2023-08-11T15:45:17.620 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-team-1:1.41.8-1.el9.x86_64 171/177 2023-08-11T15:45:17.660 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-11T15:45:17.661 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-11T15:45:18.003 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-11T15:45:18.004 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-11T15:45:18.176 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-11T15:45:18.214 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libuuid-2.37.4-9.el9.x86_64 174/177 2023-08-11T15:45:18.248 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libsmartcols-2.37.4-9.el9.x86_64 175/177 2023-08-11T15:45:18.284 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libelf-0.188-3.el9.x86_64 176/177 2023-08-11T15:45:18.450 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-11T15:45:18.976 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-11T15:45:18.977 INFO:teuthology.orchestra.run.smithi027.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-08-11T15:45:18.977 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:18.0.0-5437.g92684807.el9.x86_64 1/177 2023-08-11T15:45:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:18.0.0-5437.g92684807.el9.x86_64 2/177 2023-08-11T15:45:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 3/177 2023-08-11T15:45:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:18.0.0-5437.g92684807.el9.x86_64 4/177 2023-08-11T15:45:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5437.g92684 5/177 2023-08-11T15:45:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:18.0.0-5437.g92684807.el9.x86_64 6/177 2023-08-11T15:45:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:18.0.0-5437.g92684807.el9.x86_64 7/177 2023-08-11T15:45:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:18.0.0-5437.g92684807.el9.x86_64 8/177 2023-08-11T15:45:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:18.0.0-5437.g92684807.el9.x86_64 9/177 2023-08-11T15:45:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:18.0.0-5437.g92684807.el9.x86_64 10/177 2023-08-11T15:45:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:18.0.0-5437.g92684807.el9.x86_64 11/177 2023-08-11T15:45:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:18.0.0-5437.g92684807.el9.x86_64 12/177 2023-08-11T15:45:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:18.0.0-5437.g92684807.el9.x86_64 13/177 2023-08-11T15:45:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:18.0.0-5437.g92684807.el9.x86_ 14/177 2023-08-11T15:45:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:18.0.0-5437.g92684807.el9.x86_64 15/177 2023-08-11T15:45:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:18.0.0-5437.g92684807.el9.x86_64 16/177 2023-08-11T15:45:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:18.0.0-5437.g92684807.el9.x86_6 17/177 2023-08-11T15:45:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:18.0.0-5437.g92684807.el9.x86 18/177 2023-08-11T15:45:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:18.0.0-5437.g92684807.el9.x86_64 19/177 2023-08-11T15:45:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 20/177 2023-08-11T15:45:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:18.0.0-5437.g92684807.el9. 21/177 2023-08-11T15:45:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:18.0.0-5437.g92684807.el9.x86_6 22/177 2023-08-11T15:45:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:18.0.0-5437.g92684807.el9.x86_64 23/177 2023-08-11T15:45:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:18.0.0-5437.g92684807.el9.x86_64 24/177 2023-08-11T15:45:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:18.0.0-5437.g92684807.el9.x86_64 25/177 2023-08-11T15:45:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:18.0.0-5437.g92684807.el9.x86_64 26/177 2023-08-11T15:45:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:18.0.0-5437.g92684807.el9.x86_64 27/177 2023-08-11T15:45:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:18.0.0-5437.g92684807.el9.x86_64 28/177 2023-08-11T15:45:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 29/177 2023-08-11T15:45:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el9.noa 30/177 2023-08-11T15:45:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el9.n 31/177 2023-08-11T15:45:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 32/177 2023-08-11T15:45:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 33/177 2023-08-11T15:45:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5437.g92684807.el9.noarch 34/177 2023-08-11T15:45:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 35/177 2023-08-11T15:45:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:18.0.0-5437.g92684807.el9.noarch 36/177 2023-08-11T15:45:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:18.0.0-5437.g92684807.el9.noarch 37/177 2023-08-11T15:45:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:18.0.0-5437.g92684807.el9.noarch 38/177 2023-08-11T15:45:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/177 2023-08-11T15:45:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/177 2023-08-11T15:45:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/177 2023-08-11T15:45:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 42/177 2023-08-11T15:45:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/177 2023-08-11T15:45:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/177 2023-08-11T15:45:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/177 2023-08-11T15:45:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/177 2023-08-11T15:45:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : jq-1.6-15.el9.x86_64 47/177 2023-08-11T15:45:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/177 2023-08-11T15:45:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/177 2023-08-11T15:45:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/177 2023-08-11T15:45:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/177 2023-08-11T15:45:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/177 2023-08-11T15:45:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-13.el9.x86_64 53/177 2023-08-11T15:45:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/177 2023-08-11T15:45:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/177 2023-08-11T15:45:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/177 2023-08-11T15:45:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/177 2023-08-11T15:45:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/177 2023-08-11T15:45:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 59/177 2023-08-11T15:45:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/177 2023-08-11T15:45:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/177 2023-08-11T15:45:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/177 2023-08-11T15:45:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/177 2023-08-11T15:45:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/177 2023-08-11T15:45:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/177 2023-08-11T15:45:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/177 2023-08-11T15:45:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/177 2023-08-11T15:45:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/177 2023-08-11T15:45:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/177 2023-08-11T15:45:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/177 2023-08-11T15:45:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/177 2023-08-11T15:45:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/177 2023-08-11T15:45:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/177 2023-08-11T15:45:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/177 2023-08-11T15:45:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/177 2023-08-11T15:45:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/177 2023-08-11T15:45:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/177 2023-08-11T15:45:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/177 2023-08-11T15:45:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/177 2023-08-11T15:45:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/177 2023-08-11T15:45:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/177 2023-08-11T15:45:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/177 2023-08-11T15:45:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/177 2023-08-11T15:45:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/177 2023-08-11T15:45:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 85/177 2023-08-11T15:45:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 86/177 2023-08-11T15:45:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/177 2023-08-11T15:45:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/177 2023-08-11T15:45:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/177 2023-08-11T15:45:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/177 2023-08-11T15:45:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.16-1.el9.x86_64 91/177 2023-08-11T15:45:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-36.el9.x86_64 92/177 2023-08-11T15:45:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 93/177 2023-08-11T15:45:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 94/177 2023-08-11T15:45:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 95/177 2023-08-11T15:45:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 96/177 2023-08-11T15:45:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 97/177 2023-08-11T15:45:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 98/177 2023-08-11T15:45:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 99/177 2023-08-11T15:45:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 100/177 2023-08-11T15:45:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 101/177 2023-08-11T15:45:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 102/177 2023-08-11T15:45:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 103/177 2023-08-11T15:45:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 104/177 2023-08-11T15:45:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 105/177 2023-08-11T15:45:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 106/177 2023-08-11T15:45:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 107/177 2023-08-11T15:45:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 108/177 2023-08-11T15:45:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 109/177 2023-08-11T15:45:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 110/177 2023-08-11T15:45:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 111/177 2023-08-11T15:45:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 112/177 2023-08-11T15:45:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 113/177 2023-08-11T15:45:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 114/177 2023-08-11T15:45:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 115/177 2023-08-11T15:45:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 116/177 2023-08-11T15:45:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 117/177 2023-08-11T15:45:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 118/177 2023-08-11T15:45:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 119/177 2023-08-11T15:45:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/177 2023-08-11T15:45:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 121/177 2023-08-11T15:45:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 122/177 2023-08-11T15:45:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 123/177 2023-08-11T15:45:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 124/177 2023-08-11T15:45:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 125/177 2023-08-11T15:45:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 126/177 2023-08-11T15:45:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 127/177 2023-08-11T15:45:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 128/177 2023-08-11T15:45:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 129/177 2023-08-11T15:45:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-2.el9.x86_64 130/177 2023-08-11T15:45:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 131/177 2023-08-11T15:45:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:18.0.0-5437.g92684807.el9.x86_64 132/177 2023-08-11T15:45:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 133/177 2023-08-11T15:45:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:18.0.0-5437.g92684807.el9.x86_64 134/177 2023-08-11T15:45:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 135/177 2023-08-11T15:45:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.43.90-1.el9.x86_64 136/177 2023-08-11T15:45:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.41.8-1.el9.x86_64 137/177 2023-08-11T15:45:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.43.90-1.el9.x86_64 138/177 2023-08-11T15:45:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 139/177 2023-08-11T15:45:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.43.90-1.el9.x86_64 140/177 2023-08-11T15:45:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.41.8-1.el9.x86_64 141/177 2023-08-11T15:45:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.43.90-1.el9.x86_64 142/177 2023-08-11T15:45:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.41.8-1.el9.x86_64 143/177 2023-08-11T15:45:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 144/177 2023-08-11T15:45:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el9.x86_64 145/177 2023-08-11T15:45:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 146/177 2023-08-11T15:45:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.188-3.el9.x86_64 147/177 2023-08-11T15:45:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 148/177 2023-08-11T15:45:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.188-3.el9.x86_64 149/177 2023-08-11T15:45:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 150/177 2023-08-11T15:45:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : iproute-6.0.0-2.el9.x86_64 151/177 2023-08-11T15:45:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 152/177 2023-08-11T15:45:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : iproute-tc-6.0.0-2.el9.x86_64 153/177 2023-08-11T15:45:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libblkid-2.37.4-13.el9.x86_64 154/177 2023-08-11T15:45:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libblkid-2.37.4-9.el9.x86_64 155/177 2023-08-11T15:45:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libfdisk-2.37.4-13.el9.x86_64 156/177 2023-08-11T15:45:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libfdisk-2.37.4-9.el9.x86_64 157/177 2023-08-11T15:45:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libmount-2.37.4-13.el9.x86_64 158/177 2023-08-11T15:45:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libmount-2.37.4-9.el9.x86_64 159/177 2023-08-11T15:45:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsmartcols-2.37.4-13.el9.x86_64 160/177 2023-08-11T15:45:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsmartcols-2.37.4-9.el9.x86_64 161/177 2023-08-11T15:45:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-2.37.4-13.el9.x86_64 162/177 2023-08-11T15:45:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-2.37.4-9.el9.x86_64 163/177 2023-08-11T15:45:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 164/177 2023-08-11T15:45:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.0.7-2.el9.x86_64 165/177 2023-08-11T15:45:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 166/177 2023-08-11T15:45:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.0.7-2.el9.x86_64 167/177 2023-08-11T15:45:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-2.37.4-13.el9.x86_64 168/177 2023-08-11T15:45:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-2.37.4-9.el9.x86_64 169/177 2023-08-11T15:45:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-core-2.37.4-13.el9.x86_64 170/177 2023-08-11T15:45:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-core-2.37.4-9.el9.x86_64 171/177 2023-08-11T15:45:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-user-2.37.4-13.el9.x86_64 172/177 2023-08-11T15:45:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-user-2.37.4-9.el9.x86_64 173/177 2023-08-11T15:45:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 174/177 2023-08-11T15:45:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsdump-3.1.10-1.el9.x86_64 175/177 2023-08-11T15:45:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 176/177 2023-08-11T15:45:25.971 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-5.14.2-1.el9.x86_64 177/177 2023-08-11T15:45:25.971 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:25.971 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2023-08-11T15:45:25.972 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-1:1.43.90-1.el9.x86_64 2023-08-11T15:45:25.972 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm-1:1.43.90-1.el9.x86_64 2023-08-11T15:45:25.972 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team-1:1.43.90-1.el9.x86_64 2023-08-11T15:45:25.972 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui-1:1.43.90-1.el9.x86_64 2023-08-11T15:45:25.972 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client-0.189-3.el9.x86_64 2023-08-11T15:45:25.972 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-0.189-3.el9.x86_64 2023-08-11T15:45:25.972 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs-0.189-3.el9.x86_64 2023-08-11T15:45:25.973 INFO:teuthology.orchestra.run.smithi027.stdout: iproute-6.2.0-5.el9.x86_64 2023-08-11T15:45:25.973 INFO:teuthology.orchestra.run.smithi027.stdout: iproute-tc-6.2.0-5.el9.x86_64 2023-08-11T15:45:25.973 INFO:teuthology.orchestra.run.smithi027.stdout: libblkid-2.37.4-13.el9.x86_64 2023-08-11T15:45:25.973 INFO:teuthology.orchestra.run.smithi027.stdout: libfdisk-2.37.4-13.el9.x86_64 2023-08-11T15:45:25.973 INFO:teuthology.orchestra.run.smithi027.stdout: libmount-2.37.4-13.el9.x86_64 2023-08-11T15:45:25.973 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.973 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.974 INFO:teuthology.orchestra.run.smithi027.stdout: libsmartcols-2.37.4-13.el9.x86_64 2023-08-11T15:45:25.974 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-2.37.4-13.el9.x86_64 2023-08-11T15:45:25.974 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.0.7-24.el9.x86_64 2023-08-11T15:45:25.974 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.0.7-24.el9.x86_64 2023-08-11T15:45:25.974 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-2.37.4-13.el9.x86_64 2023-08-11T15:45:25.974 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-core-2.37.4-13.el9.x86_64 2023-08-11T15:45:25.974 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-user-2.37.4-13.el9.x86_64 2023-08-11T15:45:25.976 INFO:teuthology.orchestra.run.smithi027.stdout: xfsdump-3.1.12-3.el9.x86_64 2023-08-11T15:45:25.976 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-5.19.0-4.el9.x86_64 2023-08-11T15:45:25.976 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2023-08-11T15:45:25.977 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2023-08-11T15:45:25.977 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2023-08-11T15:45:25.977 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-08-11T15:45:25.977 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.977 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.978 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.978 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.978 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:45:25.978 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.978 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.978 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.979 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:45:25.979 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:45:25.980 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:45:25.980 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:45:25.980 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:45:25.980 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.980 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.981 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:45:25.981 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.981 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.981 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.981 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:45:25.982 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:45:25.982 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.982 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:45:25.982 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-08-11T15:45:25.983 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2023-08-11T15:45:25.983 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-08-11T15:45:25.983 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-08-11T15:45:25.983 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-08-11T15:45:25.984 INFO:teuthology.orchestra.run.smithi027.stdout: fmt-8.1.1-5.el9.x86_64 2023-08-11T15:45:25.984 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-15.el9.x86_64 2023-08-11T15:45:25.984 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-08-11T15:45:25.984 INFO:teuthology.orchestra.run.smithi027.stdout: jq-1.6-15.el9.x86_64 2023-08-11T15:45:25.984 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-08-11T15:45:25.984 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-9.el9.x86_64 2023-08-11T15:45:25.985 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-08-11T15:45:25.985 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.985 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.985 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.985 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-08-11T15:45:25.985 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2023-08-11T15:45:25.986 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-08-11T15:45:25.986 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-08-11T15:45:25.986 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-08-11T15:45:25.986 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.986 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.986 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-08-11T15:45:25.987 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.987 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-45.el9.x86_64 2023-08-11T15:45:25.987 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2023-08-11T15:45:25.987 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-13.el9.x86_64 2023-08-11T15:45:25.988 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-08-11T15:45:25.988 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-08-11T15:45:25.988 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-08-11T15:45:25.988 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.21-2.el9.x86_64 2023-08-11T15:45:25.989 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-08-11T15:45:25.989 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-08-11T15:45:25.989 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-08-11T15:45:25.989 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.11-1.el9.x86_64 2023-08-11T15:45:25.990 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-08-11T15:45:25.991 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-08-11T15:45:25.991 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.11-1.el9.x86_64 2023-08-11T15:45:25.991 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-08-11T15:45:25.991 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2023-08-11T15:45:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-08-11T15:45:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-08-11T15:45:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.993 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:25.993 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-08-11T15:45:25.993 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-08-11T15:45:25.993 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-08-11T15:45:25.994 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-08-11T15:45:25.994 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-08-11T15:45:25.994 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.16-1.el9.x86_64 2023-08-11T15:45:25.994 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-08-11T15:45:25.995 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-08-11T15:45:25.995 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-08-11T15:45:25.995 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-08-11T15:45:25.995 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-08-11T15:45:25.996 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-08-11T15:45:25.996 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-08-11T15:45:25.996 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-08-11T15:45:25.996 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-08-11T15:45:25.997 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-08-11T15:45:25.997 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-08-11T15:45:25.998 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-08-11T15:45:25.998 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-08-11T15:45:25.998 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2023-08-11T15:45:25.998 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-08-11T15:45:25.998 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-08-11T15:45:25.998 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-08-11T15:45:25.999 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-08-11T15:45:25.999 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-08-11T15:45:25.999 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-08-11T15:45:25.999 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-08-11T15:45:25.999 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2023-08-11T15:45:25.999 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2023-08-11T15:45:25.999 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-08-11T15:45:26.000 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-08-11T15:45:26.000 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-08-11T15:45:26.000 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-08-11T15:45:26.000 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2023-08-11T15:45:26.000 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2023-08-11T15:45:26.000 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:26.000 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:26.000 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-08-11T15:45:26.001 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-08-11T15:45:26.001 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:26.001 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2023-08-11T15:45:26.001 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2023-08-11T15:45:26.001 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-08-11T15:45:26.001 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-08-11T15:45:26.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2023-08-11T15:45:26.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-08-11T15:45:26.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.7-6.el9.noarch 2023-08-11T15:45:26.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-08-11T15:45:26.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-08-11T15:45:26.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-08-11T15:45:26.002 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:26.002 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:26.003 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:45:26.003 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-2.el9.x86_64 2023-08-11T15:45:26.003 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-5.el9.x86_64 2023-08-11T15:45:26.003 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2023-08-11T15:45:26.003 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2023-08-11T15:45:26.003 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-08-11T15:45:26.003 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-08-11T15:45:26.004 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-36.el9.x86_64 2023-08-11T15:45:26.004 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:45:26.004 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-08-11T15:45:26.661 DEBUG:teuthology.parallel:result is None 2023-08-11T15:48:05.623 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 69 kB/s | 18 MB 04:31 2023-08-11T15:48:12.798 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-11T15:48:12.800 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-11T15:48:12.819 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.41.8-1.el9.x86_64 is already installed. 2023-08-11T15:48:12.820 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-6.0.0-2.el9.x86_64 is already installed. 2023-08-11T15:48:12.822 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.37.4-9.el9.x86_64 is already installed. 2023-08-11T15:48:12.823 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-08-11T15:48:12.825 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-08-11T15:48:12.829 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.10-1.el9.x86_64 is already installed. 2023-08-11T15:48:12.831 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-5.14.2-1.el9.x86_64 is already installed. 2023-08-11T15:48:12.912 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-08-11T15:48:12.937 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2023-08-11T15:48:12.937 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-08-11T15:48:12.938 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2023-08-11T15:48:12.938 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-08-11T15:48:12.938 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-08-11T15:48:12.938 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:18.0.0-5437.g92684807.el9 ceph 6.5 k 2023-08-11T15:48:12.939 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:18.0.0-5437.g92684807.el9 ceph 5.2 M 2023-08-11T15:48:12.939 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:18.0.0-5437.g92684807.el9 ceph 848 k 2023-08-11T15:48:12.939 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5437.g92684807.el9 ceph 141 k 2023-08-11T15:48:12.940 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:18.0.0-5437.g92684807.el9 ceph 1.5 M 2023-08-11T15:48:12.940 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 131 k 2023-08-11T15:48:12.940 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 1.7 M 2023-08-11T15:48:12.940 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 7.4 M 2023-08-11T15:48:12.940 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 47 k 2023-08-11T15:48:12.940 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:18.0.0-5437.g92684807.el9 ceph 8.1 M 2023-08-11T15:48:12.941 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:18.0.0-5437.g92684807.el9 ceph 40 M 2023-08-11T15:48:12.941 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 261 k 2023-08-11T15:48:12.941 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 224 k 2023-08-11T15:48:12.941 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:18.0.0-5437.g92684807.el9 ceph 218 k 2023-08-11T15:48:12.941 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 18 k 2023-08-11T15:48:12.941 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 CentOS-AppStream 21 k 2023-08-11T15:48:12.941 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-08-11T15:48:12.942 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-08-11T15:48:12.942 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:18.0.0-5437.g92684807.el9 ceph 31 k 2023-08-11T15:48:12.942 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:18.0.0-5437.g92684807.el9 ceph 665 k 2023-08-11T15:48:12.942 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:18.0.0-5437.g92684807.el9 ceph 126 k 2023-08-11T15:48:12.942 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-08-11T15:48:12.942 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.37.4-13.el9 CentOS-AppStream 25 k 2023-08-11T15:48:12.942 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:3.0.7-24.el9 CentOS-AppStream 3.0 M 2023-08-11T15:48:12.943 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-08-11T15:48:12.943 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-08-11T15:48:12.943 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-08-11T15:48:12.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:18.0.0-5437.g92684807.el9 ceph 162 k 2023-08-11T15:48:12.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:18.0.0-5437.g92684807.el9 ceph 321 k 2023-08-11T15:48:12.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:18.0.0-5437.g92684807.el9 ceph 297 k 2023-08-11T15:48:12.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:18.0.0-5437.g92684807.el9 ceph 99 k 2023-08-11T15:48:12.944 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:18.0.0-5437.g92684807.el9 ceph 85 k 2023-08-11T15:48:12.944 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:18.0.0-5437.g92684807.el9 ceph 3.0 M 2023-08-11T15:48:12.944 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:18.0.0-5437.g92684807.el9 ceph 168 k 2023-08-11T15:48:12.944 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-08-11T15:48:12.944 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2023-08-11T15:48:12.944 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager x86_64 1:1.43.90-1.el9 CentOS-BaseOS 2.2 M 2023-08-11T15:48:12.944 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm x86_64 1:1.43.90-1.el9 CentOS-BaseOS 1.8 M 2023-08-11T15:48:12.944 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team x86_64 1:1.43.90-1.el9 CentOS-BaseOS 41 k 2023-08-11T15:48:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui x86_64 1:1.43.90-1.el9 CentOS-BaseOS 245 k 2023-08-11T15:48:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-debuginfod-client x86_64 0.189-3.el9 CentOS-BaseOS 36 k 2023-08-11T15:48:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf x86_64 0.189-3.el9 CentOS-BaseOS 192 k 2023-08-11T15:48:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libs x86_64 0.189-3.el9 CentOS-BaseOS 254 k 2023-08-11T15:48:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: iproute x86_64 6.2.0-5.el9 CentOS-BaseOS 785 k 2023-08-11T15:48:12.945 INFO:teuthology.orchestra.run.smithi119.stdout: iproute-tc x86_64 6.2.0-5.el9 CentOS-BaseOS 448 k 2023-08-11T15:48:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: libblkid x86_64 2.37.4-13.el9 CentOS-BaseOS 107 k 2023-08-11T15:48:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: libfdisk x86_64 2.37.4-13.el9 CentOS-BaseOS 154 k 2023-08-11T15:48:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: libmount x86_64 2.37.4-13.el9 CentOS-BaseOS 133 k 2023-08-11T15:48:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:18.0.0-5437.g92684807.el9 ceph 3.3 M 2023-08-11T15:48:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:18.0.0-5437.g92684807.el9 ceph 3.0 M 2023-08-11T15:48:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: libsmartcols x86_64 2.37.4-13.el9 CentOS-BaseOS 63 k 2023-08-11T15:48:12.946 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid x86_64 2.37.4-13.el9 CentOS-BaseOS 28 k 2023-08-11T15:48:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: openssl x86_64 1:3.0.7-24.el9 CentOS-BaseOS 1.2 M 2023-08-11T15:48:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs x86_64 1:3.0.7-24.el9 CentOS-BaseOS 2.1 M 2023-08-11T15:48:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux x86_64 2.37.4-13.el9 CentOS-BaseOS 2.2 M 2023-08-11T15:48:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-core x86_64 2.37.4-13.el9 CentOS-BaseOS 433 k 2023-08-11T15:48:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-user x86_64 2.37.4-13.el9 CentOS-BaseOS 30 k 2023-08-11T15:48:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: xfsdump x86_64 3.1.12-3.el9 CentOS-BaseOS 321 k 2023-08-11T15:48:12.947 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs x86_64 5.19.0-4.el9 CentOS-BaseOS 1.0 M 2023-08-11T15:48:12.947 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-08-11T15:48:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-08-11T15:48:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-08-11T15:48:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:18.0.0-5437.g92684807.el9 ceph 19 M 2023-08-11T15:48:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 23 k 2023-08-11T15:48:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:18.0.0-5437.g92684807.el9 ceph 2.1 M 2023-08-11T15:48:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 242 k 2023-08-11T15:48:12.948 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:18.0.0-5437.g92684807.el9 ceph 4.4 M 2023-08-11T15:48:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:18.0.0-5437.g92684807.el9 ceph 18 M 2023-08-11T15:48:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5437.g92684807.el9 ceph-noarch 15 k 2023-08-11T15:48:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:18.0.0-5437.g92684807.el9 ceph 24 k 2023-08-11T15:48:12.949 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-08-11T15:48:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-08-11T15:48:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-08-11T15:48:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-08-11T15:48:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-08-11T15:48:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-08-11T15:48:12.950 INFO:teuthology.orchestra.run.smithi119.stdout: jq x86_64 1.6-15.el9 CentOS-AppStream 186 k 2023-08-11T15:48:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-08-11T15:48:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-08-11T15:48:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:18.0.0-5437.g92684807.el9 ceph 163 k 2023-08-11T15:48:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.4.1-2.1.el9 CentOS-BaseOS 807 k 2023-08-11T15:48:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-08-11T15:48:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-08-11T15:48:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.4.1-2.1.el9 CentOS-BaseOS 197 k 2023-08-11T15:48:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-08-11T15:48:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:18.0.0-5437.g92684807.el9 ceph 472 k 2023-08-11T15:48:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-08-11T15:48:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:18.0.0-5437.g92684807.el9 ceph 4.7 M 2023-08-11T15:48:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-08-11T15:48:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-08-11T15:48:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-08-11T15:48:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-08-11T15:48:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-08-11T15:48:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-08-11T15:48:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-08-11T15:48:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-08-11T15:48:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-08-11T15:48:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-08-11T15:48:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-08-11T15:48:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-08-11T15:48:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:18.0.0-5437.g92684807.el9 ceph 45 k 2023-08-11T15:48:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:18.0.0-5437.g92684807.el9 ceph 127 k 2023-08-11T15:48:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-08-11T15:48:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-08-11T15:48:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-08-11T15:48:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-08-11T15:48:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-08-11T15:48:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.16-1.el9 appstream 247 k 2023-08-11T15:48:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-08-11T15:48:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-08-11T15:48:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-08-11T15:48:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-08-11T15:48:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-08-11T15:48:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-08-11T15:48:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-08-11T15:48:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-08-11T15:48:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-08-11T15:48:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-08-11T15:48:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-08-11T15:48:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-08-11T15:48:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-08-11T15:48:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-08-11T15:48:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-08-11T15:48:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-08-11T15:48:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-08-11T15:48:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-08-11T15:48:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-08-11T15:48:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-08-11T15:48:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-08-11T15:48:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-08-11T15:48:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-08-11T15:48:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-08-11T15:48:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-08-11T15:48:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-08-11T15:48:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-08-11T15:48:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-08-11T15:48:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-08-11T15:48:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-08-11T15:48:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-08-11T15:48:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-08-11T15:48:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-08-11T15:48:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-08-11T15:48:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-08-11T15:48:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-08-11T15:48:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-08-11T15:48:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-08-11T15:48:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-08-11T15:48:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-08-11T15:48:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-2.el9 epel 192 k 2023-08-11T15:48:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-08-11T15:48:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-08-11T15:48:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-08-11T15:48:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-08-11T15:48:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-36.el9 appstream 45 k 2023-08-11T15:48:12.961 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2023-08-11T15:48:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-08-11T15:48:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:48:12.962 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-08-11T15:48:12.962 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2023-08-11T15:48:12.962 INFO:teuthology.orchestra.run.smithi119.stdout:Install 131 Packages 2023-08-11T15:48:12.962 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 23 Packages 2023-08-11T15:48:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:48:12.963 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 205 M 2023-08-11T15:48:12.963 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-08-11T15:48:13.465 INFO:teuthology.orchestra.run.smithi119.stdout:(1/154): ceph-18.0.0-5437.g92684807.el9.x86_64. 27 kB/s | 6.5 kB 00:00 2023-08-11T15:48:13.682 INFO:teuthology.orchestra.run.smithi119.stdout:(2/154): ceph-fuse-18.0.0-5437.g92684807.el9.x8 3.8 MB/s | 848 kB 00:00 2023-08-11T15:48:13.774 INFO:teuthology.orchestra.run.smithi119.stdout:(3/154): ceph-immutable-object-cache-18.0.0-543 1.5 MB/s | 141 kB 00:00 2023-08-11T15:48:14.024 INFO:teuthology.orchestra.run.smithi119.stdout:(4/154): ceph-mds-18.0.0-5437.g92684807.el9.x86 8.5 MB/s | 2.1 MB 00:00 2023-08-11T15:48:14.250 INFO:teuthology.orchestra.run.smithi119.stdout:(5/154): ceph-mgr-18.0.0-5437.g92684807.el9.x86 6.5 MB/s | 1.5 MB 00:00 2023-08-11T15:48:14.767 INFO:teuthology.orchestra.run.smithi119.stdout:(6/154): ceph-mon-18.0.0-5437.g92684807.el9.x86 8.5 MB/s | 4.4 MB 00:00 2023-08-11T15:48:15.076 INFO:teuthology.orchestra.run.smithi119.stdout:(7/154): ceph-common-18.0.0-5437.g92684807.el9. 10 MB/s | 19 MB 00:01 2023-08-11T15:48:15.301 INFO:teuthology.orchestra.run.smithi119.stdout:(8/154): ceph-base-18.0.0-5437.g92684807.el9.x8 2.5 MB/s | 5.2 MB 00:02 2023-08-11T15:48:15.384 INFO:teuthology.orchestra.run.smithi119.stdout:(9/154): ceph-selinux-18.0.0-5437.g92684807.el9 292 kB/s | 24 kB 00:00 2023-08-11T15:48:15.826 INFO:teuthology.orchestra.run.smithi119.stdout:(10/154): ceph-radosgw-18.0.0-5437.g92684807.el 11 MB/s | 8.1 MB 00:00 2023-08-11T15:48:15.902 INFO:teuthology.orchestra.run.smithi119.stdout:(11/154): cephfs-mirror-18.0.0-5437.g92684807.e 2.9 MB/s | 218 kB 00:00 2023-08-11T15:48:15.952 INFO:teuthology.orchestra.run.smithi119.stdout:(12/154): libcephfs-devel-18.0.0-5437.g92684807 635 kB/s | 31 kB 00:00 2023-08-11T15:48:16.295 INFO:teuthology.orchestra.run.smithi119.stdout:(13/154): ceph-osd-18.0.0-5437.g92684807.el9.x8 12 MB/s | 18 MB 00:01 2023-08-11T15:48:16.352 INFO:teuthology.orchestra.run.smithi119.stdout:(14/154): libcephfs2-18.0.0-5437.g92684807.el9. 1.6 MB/s | 665 kB 00:00 2023-08-11T15:48:16.403 INFO:teuthology.orchestra.run.smithi119.stdout:(15/154): libcephsqlite-18.0.0-5437.g92684807.e 1.5 MB/s | 163 kB 00:00 2023-08-11T15:48:16.445 INFO:teuthology.orchestra.run.smithi119.stdout:(16/154): librados-devel-18.0.0-5437.g92684807. 1.4 MB/s | 126 kB 00:00 2023-08-11T15:48:16.496 INFO:teuthology.orchestra.run.smithi119.stdout:(17/154): libradosstriper1-18.0.0-5437.g9268480 5.1 MB/s | 472 kB 00:00 2023-08-11T15:48:16.570 INFO:teuthology.orchestra.run.smithi119.stdout:(18/154): python3-ceph-argparse-18.0.0-5437.g92 611 kB/s | 45 kB 00:00 2023-08-11T15:48:16.645 INFO:teuthology.orchestra.run.smithi119.stdout:(19/154): python3-ceph-common-18.0.0-5437.g9268 1.7 MB/s | 127 kB 00:00 2023-08-11T15:48:16.730 INFO:teuthology.orchestra.run.smithi119.stdout:(20/154): python3-cephfs-18.0.0-5437.g92684807. 1.9 MB/s | 162 kB 00:00 2023-08-11T15:48:16.863 INFO:teuthology.orchestra.run.smithi119.stdout:(21/154): librgw2-18.0.0-5437.g92684807.el9.x86 11 MB/s | 4.7 MB 00:00 2023-08-11T15:48:16.945 INFO:teuthology.orchestra.run.smithi119.stdout:(22/154): python3-rados-18.0.0-5437.g92684807.e 1.5 MB/s | 321 kB 00:00 2023-08-11T15:48:16.989 INFO:teuthology.orchestra.run.smithi119.stdout:(23/154): python3-rbd-18.0.0-5437.g92684807.el9 2.3 MB/s | 297 kB 00:00 2023-08-11T15:48:17.039 INFO:teuthology.orchestra.run.smithi119.stdout:(24/154): python3-rgw-18.0.0-5437.g92684807.el9 1.0 MB/s | 99 kB 00:00 2023-08-11T15:48:17.081 INFO:teuthology.orchestra.run.smithi119.stdout:(25/154): rbd-fuse-18.0.0-5437.g92684807.el9.x8 930 kB/s | 85 kB 00:00 2023-08-11T15:48:17.173 INFO:teuthology.orchestra.run.smithi119.stdout:(26/154): rbd-nbd-18.0.0-5437.g92684807.el9.x86 1.8 MB/s | 168 kB 00:00 2023-08-11T15:48:17.290 INFO:teuthology.orchestra.run.smithi119.stdout:(27/154): rbd-mirror-18.0.0-5437.g92684807.el9. 12 MB/s | 3.0 MB 00:00 2023-08-11T15:48:17.349 INFO:teuthology.orchestra.run.smithi119.stdout:(28/154): ceph-grafana-dashboards-18.0.0-5437.g 133 kB/s | 23 kB 00:00 2023-08-11T15:48:17.391 INFO:teuthology.orchestra.run.smithi119.stdout:(29/154): ceph-mgr-cephadm-18.0.0-5437.g9268480 1.3 MB/s | 131 kB 00:00 2023-08-11T15:48:17.541 INFO:teuthology.orchestra.run.smithi119.stdout:(30/154): ceph-mgr-dashboard-18.0.0-5437.g92684 9.1 MB/s | 1.7 MB 00:00 2023-08-11T15:48:17.642 INFO:teuthology.orchestra.run.smithi119.stdout:(31/154): ceph-mgr-modules-core-18.0.0-5437.g92 2.4 MB/s | 242 kB 00:00 2023-08-11T15:48:17.708 INFO:teuthology.orchestra.run.smithi119.stdout:(32/154): ceph-mgr-rook-18.0.0-5437.g92684807.e 724 kB/s | 47 kB 00:00 2023-08-11T15:48:17.792 INFO:teuthology.orchestra.run.smithi119.stdout:(33/154): ceph-prometheus-alerts-18.0.0-5437.g9 178 kB/s | 15 kB 00:00 2023-08-11T15:48:17.959 INFO:teuthology.orchestra.run.smithi119.stdout:(34/154): ceph-mgr-diskprediction-local-18.0.0- 13 MB/s | 7.4 MB 00:00 2023-08-11T15:48:18.009 INFO:teuthology.orchestra.run.smithi119.stdout:(35/154): ceph-volume-18.0.0-5437.g92684807.el9 1.2 MB/s | 261 kB 00:00 2023-08-11T15:48:18.060 INFO:teuthology.orchestra.run.smithi119.stdout:(36/154): cephadm-18.0.0-5437.g92684807.el9.noa 2.2 MB/s | 224 kB 00:00 2023-08-11T15:48:18.085 INFO:teuthology.orchestra.run.smithi119.stdout:(37/154): cephfs-top-18.0.0-5437.g92684807.el9. 242 kB/s | 18 kB 00:00 2023-08-11T15:48:18.536 INFO:teuthology.orchestra.run.smithi119.stdout:(38/154): automake-1.16.2-8.el9.noarch.rpm 1.4 MB/s | 662 kB 00:00 2023-08-11T15:48:18.594 INFO:teuthology.orchestra.run.smithi119.stdout:(39/154): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 920 kB 00:00 2023-08-11T15:48:18.628 INFO:teuthology.orchestra.run.smithi119.stdout:(40/154): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 106 kB 00:00 2023-08-11T15:48:18.653 INFO:teuthology.orchestra.run.smithi119.stdout:(41/154): elfutils-libelf-devel-0.189-3.el9.x86 375 kB/s | 21 kB 00:00 2023-08-11T15:48:18.720 INFO:teuthology.orchestra.run.smithi119.stdout:(42/154): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2023-08-11T15:48:18.745 INFO:teuthology.orchestra.run.smithi119.stdout:(43/154): flexiblas-3.0.4-8.el9.x86_64.rpm 346 kB/s | 32 kB 00:00 2023-08-11T15:48:18.837 INFO:teuthology.orchestra.run.smithi119.stdout:(44/154): flexiblas-openblas-openmp-3.0.4-8.el9 188 kB/s | 17 kB 00:00 2023-08-11T15:48:18.988 INFO:teuthology.orchestra.run.smithi119.stdout:(45/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2023-08-11T15:48:19.038 INFO:teuthology.orchestra.run.smithi119.stdout:(46/154): jq-1.6-15.el9.x86_64.rpm 995 kB/s | 186 kB 00:00 2023-08-11T15:48:19.072 INFO:teuthology.orchestra.run.smithi119.stdout:(47/154): libaio-devel-0.3.111-13.el9.x86_64.rp 124 kB/s | 10 kB 00:00 2023-08-11T15:48:19.114 INFO:teuthology.orchestra.run.smithi119.stdout:(48/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2023-08-11T15:48:19.139 INFO:teuthology.orchestra.run.smithi119.stdout:(49/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 686 kB/s | 44 kB 00:00 2023-08-11T15:48:19.214 INFO:teuthology.orchestra.run.smithi119.stdout:(50/154): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 661 kB 00:00 2023-08-11T15:48:19.273 INFO:teuthology.orchestra.run.smithi119.stdout:(51/154): libtool-2.4.6-45.el9.x86_64.rpm 4.2 MB/s | 578 kB 00:00 2023-08-11T15:48:19.290 INFO:teuthology.orchestra.run.smithi119.stdout:(52/154): libuuid-devel-2.37.4-13.el9.x86_64.rp 360 kB/s | 25 kB 00:00 2023-08-11T15:48:19.349 INFO:teuthology.orchestra.run.smithi119.stdout:(53/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 625 kB/s | 46 kB 00:00 2023-08-11T15:48:19.391 INFO:teuthology.orchestra.run.smithi119.stdout:(54/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 282 kB 00:00 2023-08-11T15:48:19.432 INFO:teuthology.orchestra.run.smithi119.stdout:(55/154): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.7 MB/s | 217 kB 00:00 2023-08-11T15:48:19.458 INFO:teuthology.orchestra.run.smithi119.stdout:(56/154): openblas-0.3.21-2.el9.x86_64.rpm 536 kB/s | 34 kB 00:00 2023-08-11T15:48:19.775 INFO:teuthology.orchestra.run.smithi119.stdout:(57/154): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2023-08-11T15:48:19.876 INFO:teuthology.orchestra.run.smithi119.stdout:(58/154): openssl-devel-3.0.7-24.el9.x86_64.rpm 7.2 MB/s | 3.0 MB 00:00 2023-08-11T15:48:19.959 INFO:teuthology.orchestra.run.smithi119.stdout:(59/154): postgresql-13.11-1.el9.x86_64.rpm 8.4 MB/s | 1.5 MB 00:00 2023-08-11T15:48:20.026 INFO:teuthology.orchestra.run.smithi119.stdout:(60/154): postgresql-contrib-13.11-1.el9.x86_64 5.9 MB/s | 811 kB 00:00 2023-08-11T15:48:20.069 INFO:teuthology.orchestra.run.smithi119.stdout:(61/154): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2023-08-11T15:48:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:(62/154): postgresql-server-13.11-1.el9.x86_64. 25 MB/s | 5.7 MB 00:00 2023-08-11T15:48:20.419 INFO:teuthology.orchestra.run.smithi119.stdout:(63/154): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2023-08-11T15:48:20.452 INFO:teuthology.orchestra.run.smithi119.stdout:(64/154): python3-jinja2-2.11.3-4.el9.noarch.rp 1.2 MB/s | 228 kB 00:00 2023-08-11T15:48:20.528 INFO:teuthology.orchestra.run.smithi119.stdout:(65/154): python3-jsonpointer-2.0-4.el9.noarch. 246 kB/s | 18 kB 00:00 2023-08-11T15:48:20.553 INFO:teuthology.orchestra.run.smithi119.stdout:(66/154): python3-jsonpatch-1.21-16.el9.noarch. 246 kB/s | 24 kB 00:00 2023-08-11T15:48:20.595 INFO:teuthology.orchestra.run.smithi119.stdout:(67/154): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2023-08-11T15:48:20.620 INFO:teuthology.orchestra.run.smithi119.stdout:(68/154): python3-markupsafe-1.1.1-12.el9.x86_6 489 kB/s | 32 kB 00:00 2023-08-11T15:48:20.787 INFO:teuthology.orchestra.run.smithi119.stdout:(69/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2023-08-11T15:48:20.834 INFO:teuthology.orchestra.run.smithi119.stdout:(70/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2023-08-11T15:48:20.905 INFO:teuthology.orchestra.run.smithi119.stdout:(71/154): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 174 kB 00:00 2023-08-11T15:48:20.930 INFO:teuthology.orchestra.run.smithi119.stdout:(72/154): python3-prettytable-0.7.2-27.el9.noar 443 kB/s | 41 kB 00:00 2023-08-11T15:48:20.972 INFO:teuthology.orchestra.run.smithi119.stdout:(73/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2023-08-11T15:48:21.014 INFO:teuthology.orchestra.run.smithi119.stdout:(74/154): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2023-08-11T15:48:21.039 INFO:teuthology.orchestra.run.smithi119.stdout:(75/154): python3-pytz-2021.1-5.el9.noarch.rpm 743 kB/s | 47 kB 00:00 2023-08-11T15:48:21.065 INFO:teuthology.orchestra.run.smithi119.stdout:(76/154): python3-requests-oauthlib-1.3.0-12.el 872 kB/s | 43 kB 00:00 2023-08-11T15:48:21.292 INFO:teuthology.orchestra.run.smithi119.stdout:(77/154): python3-toml-0.10.2-6.el9.noarch.rpm 192 kB/s | 39 kB 00:00 2023-08-11T15:48:21.540 INFO:teuthology.orchestra.run.smithi119.stdout:(78/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2023-08-11T15:48:21.582 INFO:teuthology.orchestra.run.smithi119.stdout:(79/154): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2023-08-11T15:48:21.632 INFO:teuthology.orchestra.run.smithi119.stdout:(80/154): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.7 MB/s | 85 kB 00:00 2023-08-11T15:48:21.657 INFO:teuthology.orchestra.run.smithi119.stdout:(81/154): uuid-1.6.2-55.el9.x86_64.rpm 761 kB/s | 56 kB 00:00 2023-08-11T15:48:21.708 INFO:teuthology.orchestra.run.smithi119.stdout:(82/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 850 kB/s | 63 kB 00:00 2023-08-11T15:48:21.741 INFO:teuthology.orchestra.run.smithi119.stdout:(83/154): fuse-2.9.9-15.el9.x86_64.rpm 959 kB/s | 79 kB 00:00 2023-08-11T15:48:21.800 INFO:teuthology.orchestra.run.smithi119.stdout:(84/154): libgfortran-11.4.1-2.1.el9.x86_64.rpm 8.7 MB/s | 807 kB 00:00 2023-08-11T15:48:21.859 INFO:teuthology.orchestra.run.smithi119.stdout:(85/154): libquadmath-11.4.1-2.1.el9.x86_64.rpm 1.8 MB/s | 197 kB 00:00 2023-08-11T15:48:21.909 INFO:teuthology.orchestra.run.smithi119.stdout:(86/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.2 MB/s | 241 kB 00:00 2023-08-11T15:48:21.976 INFO:teuthology.orchestra.run.smithi119.stdout:(87/154): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.1 MB 00:00 2023-08-11T15:48:22.010 INFO:teuthology.orchestra.run.smithi119.stdout:(88/154): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2023-08-11T15:48:22.043 INFO:teuthology.orchestra.run.smithi119.stdout:(89/154): python3-pycparser-2.20-6.el9.noarch.r 1.9 MB/s | 124 kB 00:00 2023-08-11T15:48:22.544 INFO:teuthology.orchestra.run.smithi119.stdout:(90/154): zlib-devel-1.2.11-36.el9.x86_64.rpm 90 kB/s | 45 kB 00:00 2023-08-11T15:48:22.736 INFO:teuthology.orchestra.run.smithi119.stdout:(91/154): python3-devel-3.9.16-1.el9.x86_64.rpm 342 kB/s | 247 kB 00:00 2023-08-11T15:48:23.545 INFO:teuthology.orchestra.run.smithi119.stdout:(92/154): ceph-test-18.0.0-5437.g92684807.el9.x 4.9 MB/s | 40 MB 00:08 2023-08-11T15:48:23.695 INFO:teuthology.orchestra.run.smithi119.stdout:(93/154): fmt-8.1.1-5.el9.x86_64.rpm 96 kB/s | 111 kB 00:01 2023-08-11T15:48:23.879 INFO:teuthology.orchestra.run.smithi119.stdout:(94/154): libarrow-doc-9.0.0-9.el9.noarch.rpm 217 kB/s | 24 kB 00:00 2023-08-11T15:48:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:(95/154): liboath-2.6.7-2.el9.x86_64.rpm 376 kB/s | 49 kB 00:00 2023-08-11T15:48:24.213 INFO:teuthology.orchestra.run.smithi119.stdout:(96/154): gperftools-libs-2.9.1-2.el9.x86_64.rp 209 kB/s | 309 kB 00:01 2023-08-11T15:48:24.246 INFO:teuthology.orchestra.run.smithi119.stdout:(97/154): libunwind-1.6.2-1.el9.x86_64.rpm 289 kB/s | 67 kB 00:00 2023-08-11T15:48:25.097 INFO:teuthology.orchestra.run.smithi119.stdout:(98/154): python3-asyncssh-2.13.2-3.el9.noarch. 645 kB/s | 548 kB 00:00 2023-08-11T15:48:25.255 INFO:teuthology.orchestra.run.smithi119.stdout:(99/154): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 276 kB/s | 43 kB 00:00 2023-08-11T15:48:25.380 INFO:teuthology.orchestra.run.smithi119.stdout:(100/154): python3-cachetools-4.2.4-1.el9.noarc 259 kB/s | 32 kB 00:00 2023-08-11T15:48:25.523 INFO:teuthology.orchestra.run.smithi119.stdout:(101/154): python3-certifi-2021.10.8-2.el9.noar 106 kB/s | 15 kB 00:00 2023-08-11T15:48:25.573 INFO:teuthology.orchestra.run.smithi119.stdout:(102/154): parquet-libs-9.0.0-9.el9.x86_64.rpm 616 kB/s | 837 kB 00:01 2023-08-11T15:48:25.834 INFO:teuthology.orchestra.run.smithi119.stdout:(103/154): python3-cheroot-8.6.0-4.el9.noarch.r 554 kB/s | 172 kB 00:00 2023-08-11T15:48:26.140 INFO:teuthology.orchestra.run.smithi119.stdout:(104/154): python3-google-auth-2.22.0-1.el9.noa 642 kB/s | 197 kB 00:00 2023-08-11T15:48:26.191 INFO:teuthology.orchestra.run.smithi119.stdout:(105/154): python3-cherrypy-18.6.1-2.el9.noarch 581 kB/s | 358 kB 00:00 2023-08-11T15:48:26.258 INFO:teuthology.orchestra.run.smithi119.stdout:(106/154): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2023-08-11T15:48:26.317 INFO:teuthology.orchestra.run.smithi119.stdout:(107/154): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2023-08-11T15:48:26.375 INFO:teuthology.orchestra.run.smithi119.stdout:(108/154): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2023-08-11T15:48:26.433 INFO:teuthology.orchestra.run.smithi119.stdout:(109/154): python3-jaraco-functools-3.5.0-2.el9 168 kB/s | 19 kB 00:00 2023-08-11T15:48:26.484 INFO:teuthology.orchestra.run.smithi119.stdout:(110/154): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2023-08-11T15:48:26.551 INFO:teuthology.orchestra.run.smithi119.stdout:(111/154): python3-jwt+crypto-2.4.0-1.el9.noarc 78 kB/s | 9.0 kB 00:00 2023-08-11T15:48:26.601 INFO:teuthology.orchestra.run.smithi119.stdout:(112/154): python3-jwt-2.4.0-1.el9.noarch.rpm 351 kB/s | 41 kB 00:00 2023-08-11T15:48:26.718 INFO:teuthology.orchestra.run.smithi119.stdout:(113/154): python3-logutils-0.3.5-21.el9.noarch 399 kB/s | 46 kB 00:00 2023-08-11T15:48:26.852 INFO:teuthology.orchestra.run.smithi119.stdout:(114/154): python3-more-itertools-8.12.0-2.el9. 594 kB/s | 79 kB 00:00 2023-08-11T15:48:26.960 INFO:teuthology.orchestra.run.smithi119.stdout:(115/154): python3-natsort-7.1.1-5.el9.noarch.r 536 kB/s | 58 kB 00:00 2023-08-11T15:48:27.219 INFO:teuthology.orchestra.run.smithi119.stdout:(116/154): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2023-08-11T15:48:27.336 INFO:teuthology.orchestra.run.smithi119.stdout:(117/154): python3-portend-3.1.0-2.el9.noarch.r 142 kB/s | 16 kB 00:00 2023-08-11T15:48:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:(118/154): python3-pecan-1.4.2-3.el9.noarch.rpm 605 kB/s | 272 kB 00:00 2023-08-11T15:48:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:(119/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 676 kB/s | 90 kB 00:00 2023-08-11T15:48:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:(120/154): python3-repoze-lru-0.7-16.el9.noarch 264 kB/s | 31 kB 00:00 2023-08-11T15:48:27.629 INFO:teuthology.orchestra.run.smithi119.stdout:(121/154): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2023-08-11T15:48:27.671 INFO:teuthology.orchestra.run.smithi119.stdout:(122/154): python3-rsa-4.9-2.el9.noarch.rpm 418 kB/s | 59 kB 00:00 2023-08-11T15:48:27.746 INFO:teuthology.orchestra.run.smithi119.stdout:(123/154): python3-tempora-5.0.0-2.el9.noarch.r 308 kB/s | 36 kB 00:00 2023-08-11T15:48:27.788 INFO:teuthology.orchestra.run.smithi119.stdout:(124/154): python3-typing-extensions-4.4.0-2.el 438 kB/s | 51 kB 00:00 2023-08-11T15:48:27.880 INFO:teuthology.orchestra.run.smithi119.stdout:(125/154): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2023-08-11T15:48:28.014 INFO:teuthology.orchestra.run.smithi119.stdout:(126/154): python3-websocket-client-1.2.3-2.el9 399 kB/s | 90 kB 00:00 2023-08-11T15:48:28.073 INFO:teuthology.orchestra.run.smithi119.stdout:(127/154): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2023-08-11T15:48:28.131 INFO:teuthology.orchestra.run.smithi119.stdout:(128/154): python3-zc-lockfile-2.0-10.el9.noarc 172 kB/s | 20 kB 00:00 2023-08-11T15:48:28.206 INFO:teuthology.orchestra.run.smithi119.stdout:(129/154): re2-20211101-2.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2023-08-11T15:48:29.848 INFO:teuthology.orchestra.run.smithi119.stdout:(130/154): libarrow-9.0.0-9.el9.x86_64.rpm 718 kB/s | 4.4 MB 00:06 2023-08-11T15:48:30.449 INFO:teuthology.orchestra.run.smithi119.stdout:(131/154): thrift-0.15.0-2.el9.x86_64.rpm 699 kB/s | 1.6 MB 00:02 2023-08-11T15:48:30.591 INFO:teuthology.orchestra.run.smithi119.stdout:(132/154): NetworkManager-1.43.90-1.el9.x86_64. 16 MB/s | 2.2 MB 00:00 2023-08-11T15:48:30.736 INFO:teuthology.orchestra.run.smithi119.stdout:(133/154): NetworkManager-libnm-1.43.90-1.el9.x 13 MB/s | 1.8 MB 00:00 2023-08-11T15:48:30.783 INFO:teuthology.orchestra.run.smithi119.stdout:(134/154): NetworkManager-team-1.43.90-1.el9.x8 885 kB/s | 41 kB 00:00 2023-08-11T15:48:30.850 INFO:teuthology.orchestra.run.smithi119.stdout:(135/154): NetworkManager-tui-1.43.90-1.el9.x86 3.6 MB/s | 245 kB 00:00 2023-08-11T15:48:30.883 INFO:teuthology.orchestra.run.smithi119.stdout:(136/154): elfutils-debuginfod-client-0.189-3.e 1.1 MB/s | 36 kB 00:00 2023-08-11T15:48:30.942 INFO:teuthology.orchestra.run.smithi119.stdout:(137/154): elfutils-libelf-0.189-3.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2023-08-11T15:48:30.993 INFO:teuthology.orchestra.run.smithi119.stdout:(138/154): elfutils-libs-0.189-3.el9.x86_64.rpm 5.1 MB/s | 254 kB 00:00 2023-08-11T15:48:31.060 INFO:teuthology.orchestra.run.smithi119.stdout:(139/154): iproute-6.2.0-5.el9.x86_64.rpm 12 MB/s | 785 kB 00:00 2023-08-11T15:48:31.143 INFO:teuthology.orchestra.run.smithi119.stdout:(140/154): iproute-tc-6.2.0-5.el9.x86_64.rpm 5.3 MB/s | 448 kB 00:00 2023-08-11T15:48:31.194 INFO:teuthology.orchestra.run.smithi119.stdout:(141/154): libblkid-2.37.4-13.el9.x86_64.rpm 2.1 MB/s | 107 kB 00:00 2023-08-11T15:48:31.244 INFO:teuthology.orchestra.run.smithi119.stdout:(142/154): libfdisk-2.37.4-13.el9.x86_64.rpm 3.1 MB/s | 154 kB 00:00 2023-08-11T15:48:31.302 INFO:teuthology.orchestra.run.smithi119.stdout:(143/154): libmount-2.37.4-13.el9.x86_64.rpm 2.3 MB/s | 133 kB 00:00 2023-08-11T15:48:31.336 INFO:teuthology.orchestra.run.smithi119.stdout:(144/154): libsmartcols-2.37.4-13.el9.x86_64.rp 1.9 MB/s | 63 kB 00:00 2023-08-11T15:48:31.362 INFO:teuthology.orchestra.run.smithi119.stdout:(145/154): libuuid-2.37.4-13.el9.x86_64.rpm 1.1 MB/s | 28 kB 00:00 2023-08-11T15:48:31.454 INFO:teuthology.orchestra.run.smithi119.stdout:(146/154): librados2-18.0.0-5437.g92684807.el9. 1.0 MB/s | 3.3 MB 00:03 2023-08-11T15:48:31.550 INFO:teuthology.orchestra.run.smithi119.stdout:(147/154): openssl-3.0.7-24.el9.x86_64.rpm 6.2 MB/s | 1.2 MB 00:00 2023-08-11T15:48:31.646 INFO:teuthology.orchestra.run.smithi119.stdout:(148/154): openssl-libs-3.0.7-24.el9.x86_64.rpm 12 MB/s | 2.1 MB 00:00 2023-08-11T15:48:31.746 INFO:teuthology.orchestra.run.smithi119.stdout:(149/154): util-linux-2.37.4-13.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2023-08-11T15:48:31.797 INFO:teuthology.orchestra.run.smithi119.stdout:(150/154): util-linux-core-2.37.4-13.el9.x86_64 3.1 MB/s | 433 kB 00:00 2023-08-11T15:48:31.822 INFO:teuthology.orchestra.run.smithi119.stdout:(151/154): util-linux-user-2.37.4-13.el9.x86_64 426 kB/s | 30 kB 00:00 2023-08-11T15:48:31.872 INFO:teuthology.orchestra.run.smithi119.stdout:(152/154): xfsdump-3.1.12-3.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2023-08-11T15:48:31.963 INFO:teuthology.orchestra.run.smithi119.stdout:(153/154): xfsprogs-5.19.0-4.el9.x86_64.rpm 7.7 MB/s | 1.0 MB 00:00 2023-08-11T15:48:32.081 INFO:teuthology.orchestra.run.smithi119.stdout:(154/154): librbd1-18.0.0-5437.g92684807.el9.x8 1.4 MB/s | 3.0 MB 00:02 2023-08-11T15:48:32.099 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:48:32.100 INFO:teuthology.orchestra.run.smithi119.stdout:Total 11 MB/s | 205 MB 00:19 2023-08-11T15:48:32.198 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-08-11T15:48:32.514 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-08-11T15:48:32.514 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-08-11T15:48:34.708 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-08-11T15:48:34.709 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-08-11T15:48:41.363 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-08-11T15:48:41.522 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-libs-1:3.0.7-24.el9.x86_64 1/177 2023-08-11T15:48:41.587 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libuuid-2.37.4-13.el9.x86_64 2/177 2023-08-11T15:48:41.650 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libblkid-2.37.4-13.el9.x86_64 3/177 2023-08-11T15:48:41.786 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libblkid-2.37.4-13.el9.x86_64 3/177 2023-08-11T15:48:41.924 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/177 2023-08-11T15:48:41.994 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-libelf-0.189-3.el9.x86_64 5/177 2023-08-11T15:48:42.093 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/177 2023-08-11T15:48:42.162 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 7/177 2023-08-11T15:48:42.217 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/177 2023-08-11T15:48:42.353 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/177 2023-08-11T15:48:42.777 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/177 2023-08-11T15:48:42.858 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:18.0.0-5437.g92684807.el9.x86_64 11/177 2023-08-11T15:48:42.998 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:18.0.0-5437.g92684807.el9.x86_64 11/177 2023-08-11T15:48:43.064 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:18.0.0-5437.g92684807.el9.x86_64 12/177 2023-08-11T15:48:43.194 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:18.0.0-5437.g92684807.el9.x86_64 12/177 2023-08-11T15:48:43.244 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:18.0.0-5437.g92684807.el9.x86_64 13/177 2023-08-11T15:48:43.520 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/177 2023-08-11T15:48:43.587 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/177 2023-08-11T15:48:43.629 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:18.0.0-5437.g92684807.el9.x86_64 16/177 2023-08-11T15:48:43.780 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5437.g92684807.el9.x86_64 16/177 2023-08-11T15:48:43.821 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:18.0.0-5437.g92684807.el9.x86 17/177 2023-08-11T15:48:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5437.g92684807.el9.x86 17/177 2023-08-11T15:48:44.002 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libmount-2.37.4-13.el9.x86_64 18/177 2023-08-11T15:48:44.381 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libsmartcols-2.37.4-13.el9.x86_64 19/177 2023-08-11T15:48:44.560 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-libnm-1:1.43.90-1.el9.x86_64 20/177 2023-08-11T15:48:44.690 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.43.90-1.el9.x86_64 20/177 2023-08-11T15:48:44.896 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-11T15:48:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-11T15:48:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-11T15:48:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/177 2023-08-11T15:48:45.573 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/177 2023-08-11T15:48:45.664 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 24/177 2023-08-11T15:48:45.738 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 25/177 2023-08-11T15:48:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/177 2023-08-11T15:48:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/177 2023-08-11T15:48:46.311 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/177 2023-08-11T15:48:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/177 2023-08-11T15:48:46.523 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:18.0.0-5437.g92684807.el9. 30/177 2023-08-11T15:48:46.621 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 31/177 2023-08-11T15:48:46.798 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:18.0.0-5437.g92684807.el9.x86_6 32/177 2023-08-11T15:48:47.001 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/177 2023-08-11T15:48:47.080 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/177 2023-08-11T15:48:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/177 2023-08-11T15:48:47.944 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/177 2023-08-11T15:48:48.077 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 37/177 2023-08-11T15:48:48.146 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-core-2.37.4-13.el9.x86_64 38/177 2023-08-11T15:48:48.187 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: util-linux-core-2.37.4-13.el9.x86_64 38/177 2023-08-11T15:48:48.246 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:18.0.0-5437.g92684807.el9.x86_6 39/177 2023-08-11T15:48:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 40/177 2023-08-11T15:48:48.514 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-libs-0.189-3.el9.x86_64 41/177 2023-08-11T15:48:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : iproute-6.2.0-5.el9.x86_64 42/177 2023-08-11T15:48:48.964 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libfdisk-2.37.4-13.el9.x86_64 43/177 2023-08-11T15:48:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-2.37.4-13.el9.x86_64 44/177 2023-08-11T15:48:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : xfsprogs-5.19.0-4.el9.x86_64 45/177 2023-08-11T15:48:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.37.4-13.el9.x86_64 46/177 2023-08-11T15:48:49.598 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/177 2023-08-11T15:48:49.652 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-1:3.0.7-24.el9.x86_64 48/177 2023-08-11T15:48:49.828 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-2.el9.x86_64 49/177 2023-08-11T15:48:49.880 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 50/177 2023-08-11T15:48:49.997 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/177 2023-08-11T15:48:50.106 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/177 2023-08-11T15:48:50.156 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/177 2023-08-11T15:48:50.205 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/177 2023-08-11T15:48:50.288 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/177 2023-08-11T15:48:50.380 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/177 2023-08-11T15:48:50.456 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/177 2023-08-11T15:48:50.668 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/177 2023-08-11T15:48:50.763 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/177 2023-08-11T15:48:50.823 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/177 2023-08-11T15:48:50.973 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/177 2023-08-11T15:48:51.020 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 62/177 2023-08-11T15:48:51.088 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/177 2023-08-11T15:48:51.162 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/177 2023-08-11T15:48:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 65/177 2023-08-11T15:48:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fmt-8.1.1-5.el9.x86_64 66/177 2023-08-11T15:48:51.461 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-36.el9.x86_64 67/177 2023-08-11T15:48:51.527 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.16-1.el9.x86_64 68/177 2023-08-11T15:48:51.624 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 69/177 2023-08-11T15:48:51.747 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 70/177 2023-08-11T15:48:52.038 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 71/177 2023-08-11T15:48:52.113 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 72/177 2023-08-11T15:48:52.306 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 73/177 2023-08-11T15:48:52.390 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 74/177 2023-08-11T15:48:52.423 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 75/177 2023-08-11T15:48:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 76/177 2023-08-11T15:48:52.534 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 77/177 2023-08-11T15:48:52.602 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-15.el9.x86_64 78/177 2023-08-11T15:48:52.683 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/177 2023-08-11T15:48:52.794 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/177 2023-08-11T15:48:52.863 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/177 2023-08-11T15:48:53.022 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/177 2023-08-11T15:48:53.073 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/177 2023-08-11T15:48:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/177 2023-08-11T15:48:53.214 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/177 2023-08-11T15:48:53.285 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/177 2023-08-11T15:48:53.454 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/177 2023-08-11T15:48:53.508 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/177 2023-08-11T15:48:53.698 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/177 2023-08-11T15:48:53.771 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 90/177 2023-08-11T15:48:54.769 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/177 2023-08-11T15:48:54.838 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 92/177 2023-08-11T15:48:54.879 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 93/177 2023-08-11T15:48:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 94/177 2023-08-11T15:48:55.224 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/177 2023-08-11T15:48:55.276 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/177 2023-08-11T15:48:55.405 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/177 2023-08-11T15:48:56.219 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/177 2023-08-11T15:48:56.389 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/177 2023-08-11T15:48:57.954 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/177 2023-08-11T15:48:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/177 2023-08-11T15:48:58.101 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 102/177 2023-08-11T15:48:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : jq-1.6-15.el9.x86_64 103/177 2023-08-11T15:48:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 104/177 2023-08-11T15:48:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/177 2023-08-11T15:48:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:18.0.0-5437.g92684807.el9.x86_64 106/177 2023-08-11T15:48:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:18.0.0-5437.g92684807.el9.x86_64 106/177 2023-08-11T15:48:58.802 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:18.0.0-5437.g92684807.el9.x86_64 107/177 2023-08-11T15:48:58.879 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/177 2023-08-11T15:48:59.380 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 109/177 2023-08-11T15:48:59.416 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:18.0.0-5437.g92684807.el9.x86_64 110/177 2023-08-11T15:48:59.548 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:18.0.0-5437.g92684807.el9.x86_64 110/177 2023-08-11T15:48:59.584 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:18.0.0-5437.g92684807.el9.x86_64 111/177 2023-08-11T15:49:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 112/177 2023-08-11T15:49:02.130 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 112/177 2023-08-11T15:49:02.754 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 112/177 2023-08-11T15:49:02.809 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:18.0.0-5437.g92684807.el9.x86_64 113/177 2023-08-11T15:49:02.961 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:18.0.0-5437.g92684807.el9.x86_64 113/177 2023-08-11T15:49:02.961 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-08-11T15:49:02.962 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-08-11T15:49:02.962 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:18.0.0-5437.g92684807.el9.x86_64 114/177 2023-08-11T15:49:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5437.g92684807.el9.x86_64 114/177 2023-08-11T15:49:15.743 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2023-08-11T15:49:15.743 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2023-08-11T15:49:15.743 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2023-08-11T15:49:15.743 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2023-08-11T15:49:15.744 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2023-08-11T15:49:15.744 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2023-08-11T15:49:15.744 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2023-08-11T15:49:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:18.0.0-5437.g92684807.el9.noarch 115/177 2023-08-11T15:49:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:18.0.0-5437.g92684807.el9.noarch 115/177 2023-08-11T15:49:16.034 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:49:16.034 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-08-11T15:49:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:17.741 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:18.0.0-5437.g92684807.el9.x86_64 116/177 2023-08-11T15:49:17.827 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:18.0.0-5437.g92684807.el9.x86_64 116/177 2023-08-11T15:49:17.828 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:49:17.828 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-08-11T15:49:17.828 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-11T15:49:17.828 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-11T15:49:17.828 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:18.075 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:18.0.0-5437.g92684807.el9.x86_64 117/177 2023-08-11T15:49:18.154 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:18.0.0-5437.g92684807.el9.x86_64 117/177 2023-08-11T15:49:18.154 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:49:18.154 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-08-11T15:49:18.154 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-11T15:49:18.154 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-11T15:49:18.154 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:18.0.0-5437.g92684807.el9.x86_64 118/177 2023-08-11T15:49:18.754 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:18.0.0-5437.g92684807.el9.x86_64 118/177 2023-08-11T15:49:18.755 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:49:18.755 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-08-11T15:49:18.755 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-11T15:49:18.755 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-11T15:49:18.756 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.2-8.el9.noarch 119/177 2023-08-11T15:49:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:18.0.0-5437.g92684807.el9.noarch 120/177 2023-08-11T15:49:19.577 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:18.0.0-5437.g92684807.el9.noarch 120/177 2023-08-11T15:49:19.627 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 121/177 2023-08-11T15:49:19.760 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 122/177 2023-08-11T15:49:19.807 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el9.noa 123/177 2023-08-11T15:49:20.141 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el9.noa 123/177 2023-08-11T15:49:20.192 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el9.n 124/177 2023-08-11T15:49:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el9.n 124/177 2023-08-11T15:49:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 125/177 2023-08-11T15:49:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 125/177 2023-08-11T15:49:21.759 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 126/177 2023-08-11T15:49:21.810 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:18.0.0-5437.g92684807.el9.x86_64 127/177 2023-08-11T15:49:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5437.g92684807.el9.x86_64 127/177 2023-08-11T15:49:21.887 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:49:21.887 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-08-11T15:49:21.887 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-11T15:49:21.887 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-11T15:49:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:18.0.0-5437.g92684807.el9.noarch 128/177 2023-08-11T15:49:22.011 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5437.g92684807.el9.noarch 128/177 2023-08-11T15:49:22.128 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:18.0.0-5437.g92684807.el9.x86_64 129/177 2023-08-11T15:49:23.063 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/177 2023-08-11T15:49:23.139 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:18.0.0-5437.g92684807.el9.x86_64 131/177 2023-08-11T15:49:23.213 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5437.g92684807.el9.x86_64 131/177 2023-08-11T15:49:23.213 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:49:23.214 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-08-11T15:49:23.214 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-11T15:49:23.214 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-11T15:49:23.214 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:23.272 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5437.g92684 132/177 2023-08-11T15:49:23.346 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5437.g92684 132/177 2023-08-11T15:49:23.347 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:49:23.347 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-08-11T15:49:23.347 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:23.431 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:18.0.0-5437.g92684807.el9.x86_64 133/177 2023-08-11T15:49:23.512 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5437.g92684807.el9.x86_64 133/177 2023-08-11T15:49:23.513 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:49:23.513 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-08-11T15:49:23.513 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-11T15:49:23.513 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-11T15:49:23.513 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:23.890 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:18.0.0-5437.g92684807.el9.x86_64 134/177 2023-08-11T15:49:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5437.g92684807.el9.x86_64 134/177 2023-08-11T15:49:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-11T15:49:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-08-11T15:49:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-11T15:49:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-11T15:49:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:28.146 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:18.0.0-5437.g92684807.el9.x86_64 135/177 2023-08-11T15:49:28.219 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:18.0.0-5437.g92684807.el9.x86_64 136/177 2023-08-11T15:49:28.286 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:18.0.0-5437.g92684807.el9.x86_64 137/177 2023-08-11T15:49:28.590 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 138/177 2023-08-11T15:49:28.744 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 139/177 2023-08-11T15:49:28.802 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:18.0.0-5437.g92684807.el9.x86_64 140/177 2023-08-11T15:49:28.871 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 141/177 2023-08-11T15:49:28.944 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : xfsdump-3.1.12-3.el9.x86_64 142/177 2023-08-11T15:49:29.072 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-user-2.37.4-13.el9.x86_64 143/177 2023-08-11T15:49:29.136 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : iproute-tc-6.2.0-5.el9.x86_64 144/177 2023-08-11T15:49:29.191 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el9.x86_64 145/177 2023-08-11T15:49:29.243 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:18.0.0-5437.g92684807.el9.x86_ 146/177 2023-08-11T15:49:29.313 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-team-1:1.43.90-1.el9.x86_64 147/177 2023-08-11T15:49:29.384 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-tui-1:1.43.90-1.el9.x86_64 148/177 2023-08-11T15:49:30.349 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-11T15:49:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-11T15:49:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-11T15:49:31.809 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:18.0.0-5437.g92684807.el9.noarch 150/177 2023-08-11T15:49:31.900 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 151/177 2023-08-11T15:49:31.964 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/177 2023-08-11T15:49:32.123 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/177 2023-08-11T15:49:32.178 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/177 2023-08-11T15:49:32.251 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-tui-1:1.41.8-1.el9.x86_64 155/177 2023-08-11T15:49:32.297 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-debuginfod-client-0.188-3.el9.x86_64 156/177 2023-08-11T15:49:32.348 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-libs-0.188-3.el9.x86_64 157/177 2023-08-11T15:49:32.348 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-11T15:49:32.524 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-11T15:49:32.608 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-1:3.0.7-2.el9.x86_64 159/177 2023-08-11T15:49:32.608 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-11T15:49:32.762 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-11T15:49:32.835 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : xfsdump-3.1.10-1.el9.x86_64 161/177 2023-08-11T15:49:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : xfsprogs-5.14.2-1.el9.x86_64 162/177 2023-08-11T15:49:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : iproute-tc-6.0.0-2.el9.x86_64 163/177 2023-08-11T15:49:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : iproute-6.0.0-2.el9.x86_64 164/177 2023-08-11T15:49:33.121 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-user-2.37.4-9.el9.x86_64 165/177 2023-08-11T15:49:33.217 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-2.37.4-9.el9.x86_64 166/177 2023-08-11T15:49:33.283 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-core-2.37.4-9.el9.x86_64 167/177 2023-08-11T15:49:33.332 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libfdisk-2.37.4-9.el9.x86_64 168/177 2023-08-11T15:49:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libmount-2.37.4-9.el9.x86_64 169/177 2023-08-11T15:49:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libblkid-2.37.4-9.el9.x86_64 170/177 2023-08-11T15:49:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-team-1:1.41.8-1.el9.x86_64 171/177 2023-08-11T15:49:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-11T15:49:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-11T15:49:33.877 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-11T15:49:33.878 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-11T15:49:34.030 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-11T15:49:34.077 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libuuid-2.37.4-9.el9.x86_64 174/177 2023-08-11T15:49:34.128 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libsmartcols-2.37.4-9.el9.x86_64 175/177 2023-08-11T15:49:34.214 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-libelf-0.188-3.el9.x86_64 176/177 2023-08-11T15:49:34.359 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-11T15:49:34.851 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-11T15:49:34.851 INFO:teuthology.orchestra.run.smithi119.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-08-11T15:49:34.851 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:39.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:18.0.0-5437.g92684807.el9.x86_64 1/177 2023-08-11T15:49:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:18.0.0-5437.g92684807.el9.x86_64 2/177 2023-08-11T15:49:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 3/177 2023-08-11T15:49:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:18.0.0-5437.g92684807.el9.x86_64 4/177 2023-08-11T15:49:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5437.g92684 5/177 2023-08-11T15:49:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:18.0.0-5437.g92684807.el9.x86_64 6/177 2023-08-11T15:49:39.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:18.0.0-5437.g92684807.el9.x86_64 7/177 2023-08-11T15:49:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:18.0.0-5437.g92684807.el9.x86_64 8/177 2023-08-11T15:49:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:18.0.0-5437.g92684807.el9.x86_64 9/177 2023-08-11T15:49:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:18.0.0-5437.g92684807.el9.x86_64 10/177 2023-08-11T15:49:39.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:18.0.0-5437.g92684807.el9.x86_64 11/177 2023-08-11T15:49:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:18.0.0-5437.g92684807.el9.x86_64 12/177 2023-08-11T15:49:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:18.0.0-5437.g92684807.el9.x86_64 13/177 2023-08-11T15:49:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:18.0.0-5437.g92684807.el9.x86_ 14/177 2023-08-11T15:49:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:18.0.0-5437.g92684807.el9.x86_64 15/177 2023-08-11T15:49:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:18.0.0-5437.g92684807.el9.x86_64 16/177 2023-08-11T15:49:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:18.0.0-5437.g92684807.el9.x86_6 17/177 2023-08-11T15:49:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:18.0.0-5437.g92684807.el9.x86 18/177 2023-08-11T15:49:39.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:18.0.0-5437.g92684807.el9.x86_64 19/177 2023-08-11T15:49:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 20/177 2023-08-11T15:49:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:18.0.0-5437.g92684807.el9. 21/177 2023-08-11T15:49:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:18.0.0-5437.g92684807.el9.x86_6 22/177 2023-08-11T15:49:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:18.0.0-5437.g92684807.el9.x86_64 23/177 2023-08-11T15:49:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:18.0.0-5437.g92684807.el9.x86_64 24/177 2023-08-11T15:49:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:18.0.0-5437.g92684807.el9.x86_64 25/177 2023-08-11T15:49:39.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:18.0.0-5437.g92684807.el9.x86_64 26/177 2023-08-11T15:49:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:18.0.0-5437.g92684807.el9.x86_64 27/177 2023-08-11T15:49:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:18.0.0-5437.g92684807.el9.x86_64 28/177 2023-08-11T15:49:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 29/177 2023-08-11T15:49:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el9.noa 30/177 2023-08-11T15:49:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el9.n 31/177 2023-08-11T15:49:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 32/177 2023-08-11T15:49:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 33/177 2023-08-11T15:49:39.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5437.g92684807.el9.noarch 34/177 2023-08-11T15:49:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 35/177 2023-08-11T15:49:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:18.0.0-5437.g92684807.el9.noarch 36/177 2023-08-11T15:49:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:18.0.0-5437.g92684807.el9.noarch 37/177 2023-08-11T15:49:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:18.0.0-5437.g92684807.el9.noarch 38/177 2023-08-11T15:49:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/177 2023-08-11T15:49:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/177 2023-08-11T15:49:39.974 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/177 2023-08-11T15:49:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 42/177 2023-08-11T15:49:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/177 2023-08-11T15:49:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/177 2023-08-11T15:49:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/177 2023-08-11T15:49:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/177 2023-08-11T15:49:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : jq-1.6-15.el9.x86_64 47/177 2023-08-11T15:49:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/177 2023-08-11T15:49:39.975 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/177 2023-08-11T15:49:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/177 2023-08-11T15:49:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/177 2023-08-11T15:49:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/177 2023-08-11T15:49:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.37.4-13.el9.x86_64 53/177 2023-08-11T15:49:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/177 2023-08-11T15:49:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/177 2023-08-11T15:49:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/177 2023-08-11T15:49:39.976 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/177 2023-08-11T15:49:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/177 2023-08-11T15:49:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 59/177 2023-08-11T15:49:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/177 2023-08-11T15:49:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/177 2023-08-11T15:49:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/177 2023-08-11T15:49:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/177 2023-08-11T15:49:39.977 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/177 2023-08-11T15:49:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/177 2023-08-11T15:49:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/177 2023-08-11T15:49:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/177 2023-08-11T15:49:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/177 2023-08-11T15:49:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/177 2023-08-11T15:49:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/177 2023-08-11T15:49:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/177 2023-08-11T15:49:39.978 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/177 2023-08-11T15:49:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/177 2023-08-11T15:49:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/177 2023-08-11T15:49:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/177 2023-08-11T15:49:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/177 2023-08-11T15:49:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/177 2023-08-11T15:49:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/177 2023-08-11T15:49:39.979 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/177 2023-08-11T15:49:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/177 2023-08-11T15:49:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/177 2023-08-11T15:49:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/177 2023-08-11T15:49:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/177 2023-08-11T15:49:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/177 2023-08-11T15:49:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 85/177 2023-08-11T15:49:39.980 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 86/177 2023-08-11T15:49:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/177 2023-08-11T15:49:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/177 2023-08-11T15:49:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/177 2023-08-11T15:49:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/177 2023-08-11T15:49:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.16-1.el9.x86_64 91/177 2023-08-11T15:49:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-36.el9.x86_64 92/177 2023-08-11T15:49:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 93/177 2023-08-11T15:49:39.981 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 94/177 2023-08-11T15:49:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 95/177 2023-08-11T15:49:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 96/177 2023-08-11T15:49:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 97/177 2023-08-11T15:49:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 98/177 2023-08-11T15:49:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 99/177 2023-08-11T15:49:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 100/177 2023-08-11T15:49:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 101/177 2023-08-11T15:49:39.982 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 102/177 2023-08-11T15:49:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 103/177 2023-08-11T15:49:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 104/177 2023-08-11T15:49:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 105/177 2023-08-11T15:49:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 106/177 2023-08-11T15:49:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 107/177 2023-08-11T15:49:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 108/177 2023-08-11T15:49:39.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 109/177 2023-08-11T15:49:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 110/177 2023-08-11T15:49:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 111/177 2023-08-11T15:49:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 112/177 2023-08-11T15:49:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 113/177 2023-08-11T15:49:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 114/177 2023-08-11T15:49:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 115/177 2023-08-11T15:49:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 116/177 2023-08-11T15:49:39.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 117/177 2023-08-11T15:49:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 118/177 2023-08-11T15:49:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 119/177 2023-08-11T15:49:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/177 2023-08-11T15:49:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 121/177 2023-08-11T15:49:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 122/177 2023-08-11T15:49:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 123/177 2023-08-11T15:49:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 124/177 2023-08-11T15:49:39.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 125/177 2023-08-11T15:49:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 126/177 2023-08-11T15:49:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 127/177 2023-08-11T15:49:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 128/177 2023-08-11T15:49:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 129/177 2023-08-11T15:49:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-2.el9.x86_64 130/177 2023-08-11T15:49:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 131/177 2023-08-11T15:49:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:18.0.0-5437.g92684807.el9.x86_64 132/177 2023-08-11T15:49:39.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 133/177 2023-08-11T15:49:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:18.0.0-5437.g92684807.el9.x86_64 134/177 2023-08-11T15:49:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 135/177 2023-08-11T15:49:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.43.90-1.el9.x86_64 136/177 2023-08-11T15:49:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.41.8-1.el9.x86_64 137/177 2023-08-11T15:49:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.43.90-1.el9.x86_64 138/177 2023-08-11T15:49:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 139/177 2023-08-11T15:49:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.43.90-1.el9.x86_64 140/177 2023-08-11T15:49:39.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.41.8-1.el9.x86_64 141/177 2023-08-11T15:49:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.43.90-1.el9.x86_64 142/177 2023-08-11T15:49:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.41.8-1.el9.x86_64 143/177 2023-08-11T15:49:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 144/177 2023-08-11T15:49:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el9.x86_64 145/177 2023-08-11T15:49:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 146/177 2023-08-11T15:49:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-0.188-3.el9.x86_64 147/177 2023-08-11T15:49:39.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 148/177 2023-08-11T15:49:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libs-0.188-3.el9.x86_64 149/177 2023-08-11T15:49:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 150/177 2023-08-11T15:49:39.989 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : iproute-6.0.0-2.el9.x86_64 151/177 2023-08-11T15:49:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 152/177 2023-08-11T15:49:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : iproute-tc-6.0.0-2.el9.x86_64 153/177 2023-08-11T15:49:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libblkid-2.37.4-13.el9.x86_64 154/177 2023-08-11T15:49:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libblkid-2.37.4-9.el9.x86_64 155/177 2023-08-11T15:49:39.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libfdisk-2.37.4-13.el9.x86_64 156/177 2023-08-11T15:49:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libfdisk-2.37.4-9.el9.x86_64 157/177 2023-08-11T15:49:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libmount-2.37.4-13.el9.x86_64 158/177 2023-08-11T15:49:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libmount-2.37.4-9.el9.x86_64 159/177 2023-08-11T15:49:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsmartcols-2.37.4-13.el9.x86_64 160/177 2023-08-11T15:49:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsmartcols-2.37.4-9.el9.x86_64 161/177 2023-08-11T15:49:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-2.37.4-13.el9.x86_64 162/177 2023-08-11T15:49:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-2.37.4-9.el9.x86_64 163/177 2023-08-11T15:49:39.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 164/177 2023-08-11T15:49:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-2.el9.x86_64 165/177 2023-08-11T15:49:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 166/177 2023-08-11T15:49:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-2.el9.x86_64 167/177 2023-08-11T15:49:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-2.37.4-13.el9.x86_64 168/177 2023-08-11T15:49:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-2.37.4-9.el9.x86_64 169/177 2023-08-11T15:49:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-core-2.37.4-13.el9.x86_64 170/177 2023-08-11T15:49:39.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-core-2.37.4-9.el9.x86_64 171/177 2023-08-11T15:49:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-user-2.37.4-13.el9.x86_64 172/177 2023-08-11T15:49:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-user-2.37.4-9.el9.x86_64 173/177 2023-08-11T15:49:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 174/177 2023-08-11T15:49:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsdump-3.1.10-1.el9.x86_64 175/177 2023-08-11T15:49:39.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 176/177 2023-08-11T15:49:42.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-5.14.2-1.el9.x86_64 177/177 2023-08-11T15:49:42.569 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2023-08-11T15:49:42.569 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-1:1.43.90-1.el9.x86_64 2023-08-11T15:49:42.569 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm-1:1.43.90-1.el9.x86_64 2023-08-11T15:49:42.569 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team-1:1.43.90-1.el9.x86_64 2023-08-11T15:49:42.569 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui-1:1.43.90-1.el9.x86_64 2023-08-11T15:49:42.570 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-debuginfod-client-0.189-3.el9.x86_64 2023-08-11T15:49:42.570 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-0.189-3.el9.x86_64 2023-08-11T15:49:42.570 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libs-0.189-3.el9.x86_64 2023-08-11T15:49:42.570 INFO:teuthology.orchestra.run.smithi119.stdout: iproute-6.2.0-5.el9.x86_64 2023-08-11T15:49:42.570 INFO:teuthology.orchestra.run.smithi119.stdout: iproute-tc-6.2.0-5.el9.x86_64 2023-08-11T15:49:42.570 INFO:teuthology.orchestra.run.smithi119.stdout: libblkid-2.37.4-13.el9.x86_64 2023-08-11T15:49:42.570 INFO:teuthology.orchestra.run.smithi119.stdout: libfdisk-2.37.4-13.el9.x86_64 2023-08-11T15:49:42.571 INFO:teuthology.orchestra.run.smithi119.stdout: libmount-2.37.4-13.el9.x86_64 2023-08-11T15:49:42.571 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.571 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.571 INFO:teuthology.orchestra.run.smithi119.stdout: libsmartcols-2.37.4-13.el9.x86_64 2023-08-11T15:49:42.571 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-2.37.4-13.el9.x86_64 2023-08-11T15:49:42.571 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-1:3.0.7-24.el9.x86_64 2023-08-11T15:49:42.571 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs-1:3.0.7-24.el9.x86_64 2023-08-11T15:49:42.571 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-2.37.4-13.el9.x86_64 2023-08-11T15:49:42.572 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-core-2.37.4-13.el9.x86_64 2023-08-11T15:49:42.572 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-user-2.37.4-13.el9.x86_64 2023-08-11T15:49:42.572 INFO:teuthology.orchestra.run.smithi119.stdout: xfsdump-3.1.12-3.el9.x86_64 2023-08-11T15:49:42.572 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-5.19.0-4.el9.x86_64 2023-08-11T15:49:42.572 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-08-11T15:49:42.572 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.2-8.el9.noarch 2023-08-11T15:49:42.572 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.7.4-5.el9.x86_64 2023-08-11T15:49:42.573 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-08-11T15:49:42.573 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.573 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.573 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.574 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.574 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:49:42.574 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.574 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.574 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.574 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:49:42.574 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:49:42.575 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:49:42.575 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:49:42.575 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:49:42.575 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.575 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.575 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:49:42.575 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.576 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.576 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.576 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:49:42.576 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:49:42.576 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.576 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:18.0.0-5437.g92684807.el9.noarch 2023-08-11T15:49:42.576 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-08-11T15:49:42.577 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.4-9.el9.x86_64 2023-08-11T15:49:42.577 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-08-11T15:49:42.577 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-08-11T15:49:42.577 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-08-11T15:49:42.577 INFO:teuthology.orchestra.run.smithi119.stdout: fmt-8.1.1-5.el9.x86_64 2023-08-11T15:49:42.577 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-15.el9.x86_64 2023-08-11T15:49:42.577 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-08-11T15:49:42.578 INFO:teuthology.orchestra.run.smithi119.stdout: jq-1.6-15.el9.x86_64 2023-08-11T15:49:42.578 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-08-11T15:49:42.578 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-9.el9.x86_64 2023-08-11T15:49:42.578 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-08-11T15:49:42.578 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.578 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.578 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.578 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-08-11T15:49:42.579 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2023-08-11T15:49:42.579 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-08-11T15:49:42.579 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-08-11T15:49:42.579 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-08-11T15:49:42.579 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.579 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.579 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-08-11T15:49:42.580 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.580 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-45.el9.x86_64 2023-08-11T15:49:42.580 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2023-08-11T15:49:42.580 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.37.4-13.el9.x86_64 2023-08-11T15:49:42.580 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-08-11T15:49:42.580 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-08-11T15:49:42.580 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-08-11T15:49:42.581 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.21-2.el9.x86_64 2023-08-11T15:49:42.581 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-08-11T15:49:42.581 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-08-11T15:49:42.581 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-08-11T15:49:42.581 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-13.11-1.el9.x86_64 2023-08-11T15:49:42.581 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-08-11T15:49:42.581 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-08-11T15:49:42.581 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-13.11-1.el9.x86_64 2023-08-11T15:49:42.582 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-08-11T15:49:42.582 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2023-08-11T15:49:42.582 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-08-11T15:49:42.582 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-08-11T15:49:42.582 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.582 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.582 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.582 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-08-11T15:49:42.583 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-08-11T15:49:42.583 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-08-11T15:49:42.583 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-08-11T15:49:42.583 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-08-11T15:49:42.583 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.16-1.el9.x86_64 2023-08-11T15:49:42.584 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-08-11T15:49:42.584 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-08-11T15:49:42.584 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-08-11T15:49:42.584 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-08-11T15:49:42.584 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-08-11T15:49:42.584 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-08-11T15:49:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-08-11T15:49:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-08-11T15:49:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-08-11T15:49:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-08-11T15:49:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-08-11T15:49:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-08-11T15:49:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-08-11T15:49:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2023-08-11T15:49:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-08-11T15:49:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-08-11T15:49:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-08-11T15:49:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-08-11T15:49:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-08-11T15:49:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-08-11T15:49:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-08-11T15:49:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2023-08-11T15:49:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2023-08-11T15:49:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-08-11T15:49:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-08-11T15:49:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-08-11T15:49:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-08-11T15:49:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2023-08-11T15:49:42.588 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2023-08-11T15:49:42.588 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.588 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.588 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-08-11T15:49:42.588 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-08-11T15:49:42.589 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.589 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2023-08-11T15:49:42.589 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2023-08-11T15:49:42.590 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-08-11T15:49:42.590 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-08-11T15:49:42.590 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2023-08-11T15:49:42.590 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-08-11T15:49:42.591 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2023-08-11T15:49:42.591 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-08-11T15:49:42.591 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-08-11T15:49:42.592 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-08-11T15:49:42.592 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.592 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.592 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:18.0.0-5437.g92684807.el9.x86_64 2023-08-11T15:49:42.593 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-2.el9.x86_64 2023-08-11T15:49:42.593 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-5.el9.x86_64 2023-08-11T15:49:42.593 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2023-08-11T15:49:42.593 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-55.el9.x86_64 2023-08-11T15:49:42.594 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-08-11T15:49:42.594 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-08-11T15:49:42.595 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-36.el9.x86_64 2023-08-11T15:49:42.595 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:42.595 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-08-11T15:49:43.332 DEBUG:teuthology.parallel:result is None 2023-08-11T15:49:43.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:49:43.471 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-11T15:49:43.551 INFO:teuthology.orchestra.run.smithi027.stdout:18.0.0-5437.g92684807.el9 2023-08-11T15:49:43.552 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5437.g92684807.el9 2023-08-11T15:49:43.552 INFO:teuthology.task.install:The correct ceph version 18.0.0-5437.g92684807 is installed. 2023-08-11T15:49:43.553 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:49:43.687 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-11T15:49:43.743 INFO:teuthology.orchestra.run.smithi119.stdout:18.0.0-5437.g92684807.el9 2023-08-11T15:49:43.743 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5437.g92684807.el9 2023-08-11T15:49:43.744 INFO:teuthology.task.install:The correct ceph version 18.0.0-5437.g92684807 is installed. 2023-08-11T15:49:43.744 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-11T15:49:43.744 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:43.745 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-11T15:49:43.868 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:49:43.868 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-11T15:49:43.951 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-11T15:49:43.952 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:43.952 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2023-08-11T15:49:44.074 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-11T15:49:44.181 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:49:44.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2023-08-11T15:49:44.255 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-11T15:49:44.383 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-11T15:49:44.383 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:44.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-11T15:49:44.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-11T15:49:44.646 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:49:44.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-11T15:49:44.722 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-11T15:49:44.838 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-11T15:49:44.838 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:44.839 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2023-08-11T15:49:44.956 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-11T15:49:45.063 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:49:45.063 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2023-08-11T15:49:45.142 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-11T15:49:45.255 INFO:teuthology.run_tasks:Running task ceph... 2023-08-11T15:49:45.331 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-08-11T15:49:45.332 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /var/log/ceph 2023-08-11T15:49:45.341 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2023-08-11T15:49:45.451 INFO:tasks.ceph:Disabling ceph logrotate... 2023-08-11T15:49:45.451 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-11T15:49:45.460 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-11T15:49:45.559 INFO:tasks.ceph:Creating extra log directories... 2023-08-11T15:49:45.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-11T15:49:45.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-11T15:49:45.685 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-08-11T15:49:45.686 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-08-11T15:49:45.686 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-08-11T15:49:45.687 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:45.687 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-11T15:49:45.748 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-11T15:49:46.061 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-11T15:49:46.172 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:49:46.172 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-11T15:49:46.220 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-11T15:49:46.441 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-11T15:49:46.521 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-08-11T15:49:46.521 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-08-11T15:49:46.522 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066', 'branch': 'wip-rishabh-2023Aug1-b4', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}', 'email': None, 'first_in_suite': False, 'job_id': '7366066', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023Aug1-b4'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-2023Aug1-b4', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-2023Aug1-b4', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi027.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIeYzfE3NPS3k8lesPvSilWbv9aC2F0ABFsxvm0DjICD7uFmKzpbjIT4K0oyiq4G8JVQbSTXUg2NPF/FJWCfmoc=', 'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGd8+I5tYPlLin+mFo6DMZQ8wbSlDqoZJKOfAPw6zm6bEe2ff7EDCAnT3VVnHewnFpsguxwJp+ip/NUNaR9JwnI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_quota']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '7fda95956ac10132c9b74016ba832db907df09fa', 'timestamp': '2023-08-10_20:16:46', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2224'} 2023-08-11T15:49:46.523 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-11T15:49:46.532 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-11T15:49:46.588 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/run/ceph 2023-08-11T15:49:46.595 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2023-08-11T15:49:46.701 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:46.702 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2023-08-11T15:49:46.764 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-11T15:49:46.764 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2023-08-11T15:49:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-11T15:49:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:49:46.824 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 622 Links: 1 2023-08-11T15:49:46.825 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:49:46.825 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:49:46.825 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-08-11 15:45:19.438945781 +0000 2023-08-11T15:49:46.825 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-08-11 15:42:26.201763488 +0000 2023-08-11T15:49:46.825 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-08-11 15:42:26.201763488 +0000 2023-08-11T15:49:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-08-11 15:42:26.201763488 +0000 2023-08-11T15:49:46.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-11T15:49:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-08-11T15:49:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-08-11T15:49:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000380565 s, 1.3 MB/s 2023-08-11T15:49:46.933 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-11T15:49:46.996 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2023-08-11T15:49:47.050 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-11T15:49:47.051 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:49:47.051 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 640 Links: 1 2023-08-11T15:49:47.051 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:49:47.051 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:49:47.051 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-08-11 15:45:19.439945756 +0000 2023-08-11T15:49:47.051 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-08-11 15:42:27.336734410 +0000 2023-08-11T15:49:47.052 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-08-11 15:42:27.336734410 +0000 2023-08-11T15:49:47.052 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-08-11 15:42:27.336734410 +0000 2023-08-11T15:49:47.052 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-11T15:49:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-08-11T15:49:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-08-11T15:49:47.158 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000407115 s, 1.3 MB/s 2023-08-11T15:49:47.162 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-11T15:49:47.227 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2023-08-11T15:49:47.283 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-11T15:49:47.283 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:49:47.283 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 658 Links: 1 2023-08-11T15:49:47.283 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:49:47.283 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:49:47.283 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-08-11 15:45:19.440945731 +0000 2023-08-11T15:49:47.284 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-08-11 15:42:28.417706717 +0000 2023-08-11T15:49:47.284 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-08-11 15:42:28.417706717 +0000 2023-08-11T15:49:47.284 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-08-11 15:42:28.417706717 +0000 2023-08-11T15:49:47.284 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-11T15:49:47.387 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-08-11T15:49:47.387 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-08-11T15:49:47.387 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000371144 s, 1.4 MB/s 2023-08-11T15:49:47.390 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-11T15:49:47.453 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2023-08-11T15:49:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-11T15:49:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:49:47.507 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 678 Links: 1 2023-08-11T15:49:47.507 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:49:47.508 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:49:47.508 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-08-11 15:45:19.441945705 +0000 2023-08-11T15:49:47.508 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-08-11 15:42:29.562677383 +0000 2023-08-11T15:49:47.508 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-08-11 15:42:29.562677383 +0000 2023-08-11T15:49:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-08-11 15:42:29.562677383 +0000 2023-08-11T15:49:47.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-11T15:49:47.628 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-08-11T15:49:47.629 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-08-11T15:49:47.629 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000493266 s, 1.0 MB/s 2023-08-11T15:49:47.633 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-11T15:49:47.696 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-11T15:49:47.697 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:49:47.697 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2023-08-11T15:49:47.744 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-11T15:49:47.744 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2023-08-11T15:49:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-11T15:49:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:49:47.829 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 625 Links: 1 2023-08-11T15:49:47.829 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:49:47.829 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:49:47.830 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-08-11 15:49:35.294915632 +0000 2023-08-11T15:49:47.830 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-08-11 15:41:53.507734330 +0000 2023-08-11T15:49:47.830 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-08-11 15:41:53.507734330 +0000 2023-08-11T15:49:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-08-11 15:41:53.507734330 +0000 2023-08-11T15:49:47.830 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-11T15:49:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-08-11T15:49:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-08-11T15:49:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000355723 s, 1.4 MB/s 2023-08-11T15:49:47.944 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-11T15:49:47.988 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2023-08-11T15:49:48.075 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-11T15:49:48.076 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:49:48.076 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 642 Links: 1 2023-08-11T15:49:48.076 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:49:48.077 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:49:48.077 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-08-11 15:49:35.295915607 +0000 2023-08-11T15:49:48.077 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-08-11 15:41:54.553708963 +0000 2023-08-11T15:49:48.077 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-08-11 15:41:54.553708963 +0000 2023-08-11T15:49:48.078 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-08-11 15:41:54.553708963 +0000 2023-08-11T15:49:48.078 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-11T15:49:48.206 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-08-11T15:49:48.207 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-08-11T15:49:48.207 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000327788 s, 1.6 MB/s 2023-08-11T15:49:48.210 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-11T15:49:48.256 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2023-08-11T15:49:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-11T15:49:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:49:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 659 Links: 1 2023-08-11T15:49:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:49:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:49:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-08-11 15:49:35.295915607 +0000 2023-08-11T15:49:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-08-11 15:41:55.614683231 +0000 2023-08-11T15:49:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-08-11 15:41:55.614683231 +0000 2023-08-11T15:49:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-08-11 15:41:55.614683231 +0000 2023-08-11T15:49:48.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-11T15:49:48.443 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-08-11T15:49:48.444 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-08-11T15:49:48.444 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000352578 s, 1.5 MB/s 2023-08-11T15:49:48.447 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-11T15:49:48.506 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2023-08-11T15:49:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-11T15:49:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:49:48.558 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 677 Links: 1 2023-08-11T15:49:48.558 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:49:48.558 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:49:48.559 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-08-11 15:49:35.296915583 +0000 2023-08-11T15:49:48.559 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-08-11 15:41:56.682657330 +0000 2023-08-11T15:49:48.559 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-08-11 15:41:56.682657330 +0000 2023-08-11T15:49:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-08-11 15:41:56.682657330 +0000 2023-08-11T15:49:48.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-11T15:49:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-08-11T15:49:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-08-11T15:49:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.00037856 s, 1.4 MB/s 2023-08-11T15:49:48.652 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-11T15:49:48.707 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-11T15:49:48.708 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-11T15:49:48.708 INFO:tasks.ceph:Generating config... 2023-08-11T15:49:48.710 INFO:tasks.ceph:[client] client mount timeout = 600 2023-08-11T15:49:48.711 INFO:tasks.ceph:[client] debug client = 20 2023-08-11T15:49:48.711 INFO:tasks.ceph:[client] debug ms = 1 2023-08-11T15:49:48.712 INFO:tasks.ceph:[client] pid file = 2023-08-11T15:49:48.712 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-08-11T15:49:48.712 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-08-11T15:49:48.712 INFO:tasks.ceph:[mds] debug mds = 20 2023-08-11T15:49:48.713 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-08-11T15:49:48.713 INFO:tasks.ceph:[mds] debug ms = 1 2023-08-11T15:49:48.713 INFO:tasks.ceph:[mds] mds debug frag = True 2023-08-11T15:49:48.713 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-08-11T15:49:48.713 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-08-11T15:49:48.714 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-08-11T15:49:48.714 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-08-11T15:49:48.714 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-08-11T15:49:48.714 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-08-11T15:49:48.715 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-08-11T15:49:48.715 INFO:tasks.ceph:[mgr] debug ms = 1 2023-08-11T15:49:48.715 INFO:tasks.ceph:[mon] debug mon = 20 2023-08-11T15:49:48.715 INFO:tasks.ceph:[mon] debug ms = 1 2023-08-11T15:49:48.716 INFO:tasks.ceph:[mon] debug paxos = 20 2023-08-11T15:49:48.716 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-08-11T15:49:48.716 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-08-11T15:49:48.716 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-08-11T15:49:48.717 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-08-11T15:49:48.717 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-08-11T15:49:48.717 INFO:tasks.ceph:[osd] debug ms = 1 2023-08-11T15:49:48.717 INFO:tasks.ceph:[osd] debug osd = 20 2023-08-11T15:49:48.717 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-08-11T15:49:48.718 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-08-11T15:49:48.718 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-08-11T15:49:48.718 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-08-11T15:49:48.718 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-08-11T15:49:48.718 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-08-11T15:49:48.719 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-08-11T15:49:48.719 INFO:tasks.ceph:Setting up mon.a... 2023-08-11T15:49:48.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-08-11T15:49:48.887 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.keyring 2023-08-11T15:49:48.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-08-11T15:49:49.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-11T15:49:49.151 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.27'), ('mon.b', '172.21.15.119'), ('mon.c', '[v2:172.21.15.119:3301,v1:172.21.15.119:6790]')] 2023-08-11T15:49:49.152 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.27,172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-08-11T15:49:49.152 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:49.153 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-08-11T15:49:49.213 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.27 --add b 172.21.15.119 --addv c '[v2:172.21.15.119:3301,v1:172.21.15.119:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:49:49.333 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:49:49.333 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: generated fsid bee60691-4c46-4397-a560-d178aa4db950 2023-08-11T15:49:49.333 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = pacific 2023-08-11T15:49:49.333 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 0 2023-08-11T15:49:49.333 INFO:teuthology.orchestra.run.smithi027.stdout:fsid bee60691-4c46-4397-a560-d178aa4db950 2023-08-11T15:49:49.334 INFO:teuthology.orchestra.run.smithi027.stdout:last_changed 2023-08-11T15:49:49.332352+0000 2023-08-11T15:49:49.334 INFO:teuthology.orchestra.run.smithi027.stdout:created 2023-08-11T15:49:49.332352+0000 2023-08-11T15:49:49.334 INFO:teuthology.orchestra.run.smithi027.stdout:min_mon_release 16 (pacific) 2023-08-11T15:49:49.334 INFO:teuthology.orchestra.run.smithi027.stdout:election_strategy: 1 2023-08-11T15:49:49.334 INFO:teuthology.orchestra.run.smithi027.stdout:0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2023-08-11T15:49:49.334 INFO:teuthology.orchestra.run.smithi027.stdout:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2023-08-11T15:49:49.334 INFO:teuthology.orchestra.run.smithi027.stdout:2: [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] mon.c 2023-08-11T15:49:49.334 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-08-11T15:49:49.336 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-08-11T15:49:49.390 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID bee60691-4c46-4397-a560-d178aa4db950... 2023-08-11T15:49:49.391 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-11T15:49:49.398 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-11T15:49:49.578 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2023-08-11T15:49:49.579 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2023-08-11T15:49:49.579 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-11T15:49:49.579 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2023-08-11T15:49:49.579 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.579 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2023-08-11T15:49:49.579 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.579 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2023-08-11T15:49:49.580 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.580 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2023-08-11T15:49:49.580 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2023-08-11T15:49:49.580 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.580 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2023-08-11T15:49:49.581 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2023-08-11T15:49:49.581 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.581 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-11T15:49:49.581 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2023-08-11T15:49:49.581 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.581 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2023-08-11T15:49:49.581 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-11T15:49:49.582 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.582 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2023-08-11T15:49:49.582 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.582 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2023-08-11T15:49:49.582 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2023-08-11T15:49:49.583 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2023-08-11T15:49:49.583 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2023-08-11T15:49:49.583 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2023-08-11T15:49:49.584 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2023-08-11T15:49:49.584 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2023-08-11T15:49:49.584 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-11T15:49:49.584 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-11T15:49:49.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2023-08-11T15:49:49.585 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.585 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-11T15:49:49.585 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.586 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2023-08-11T15:49:49.586 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.587 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2023-08-11T15:49:49.587 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.587 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2023-08-11T15:49:49.587 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2023-08-11T15:49:49.588 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2023-08-11T15:49:49.588 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.27,172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790]" 2023-08-11T15:49:49.588 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = bee60691-4c46-4397-a560-d178aa4db950 2023-08-11T15:49:49.588 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.588 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2023-08-11T15:49:49.589 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2023-08-11T15:49:49.589 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.589 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2023-08-11T15:49:49.589 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2023-08-11T15:49:49.589 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2023-08-11T15:49:49.589 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.589 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2023-08-11T15:49:49.589 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2023-08-11T15:49:49.590 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.590 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2023-08-11T15:49:49.590 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2023-08-11T15:49:49.590 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2023-08-11T15:49:49.590 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.590 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2023-08-11T15:49:49.590 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2023-08-11T15:49:49.591 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.591 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2023-08-11T15:49:49.591 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.591 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2023-08-11T15:49:49.591 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.591 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2023-08-11T15:49:49.591 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.592 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2023-08-11T15:49:49.592 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2023-08-11T15:49:49.592 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.592 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2023-08-11T15:49:49.592 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2023-08-11T15:49:49.592 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2023-08-11T15:49:49.592 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2023-08-11T15:49:49.593 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 20 2023-08-11T15:49:49.593 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 20 2023-08-11T15:49:49.593 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-08-11T15:49:49.593 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2023-08-11T15:49:49.593 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 10 2023-08-11T15:49:49.593 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2023-08-11T15:49:49.593 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2023-08-11T15:49:49.593 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2023-08-11T15:49:49.594 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2023-08-11T15:49:49.594 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2023-08-11T15:49:49.594 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-08-11T15:49:49.594 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.594 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2023-08-11T15:49:49.594 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-08-11T15:49:49.594 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2023-08-11T15:49:49.595 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-08-11T15:49:49.595 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-08-11T15:49:49.595 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-08-11T15:49:49.595 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-08-11T15:49:49.595 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2023-08-11T15:49:49.595 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.595 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2023-08-11T15:49:49.596 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-08-11T15:49:49.596 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-08-11T15:49:49.596 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2023-08-11T15:49:49.596 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-08-11T15:49:49.596 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2023-08-11T15:49:49.596 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2023-08-11T15:49:49.597 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-08-11T15:49:49.597 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2023-08-11T15:49:49.597 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2023-08-11T15:49:49.597 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-08-11T15:49:49.597 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.597 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-11T15:49:49.597 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2023-08-11T15:49:49.597 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2023-08-11T15:49:49.598 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.598 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2023-08-11T15:49:49.598 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-11T15:49:49.598 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-11T15:49:49.598 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.598 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2023-08-11T15:49:49.598 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2023-08-11T15:49:49.599 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.599 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2023-08-11T15:49:49.599 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2023-08-11T15:49:49.599 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-08-11T15:49:49.599 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2023-08-11T15:49:49.599 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2023-08-11T15:49:49.599 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2023-08-11T15:49:49.600 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2023-08-11T15:49:49.600 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-11T15:49:49.600 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-11T15:49:49.600 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2023-08-11T15:49:49.600 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2023-08-11T15:49:49.600 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-08-11T15:49:49.600 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2023-08-11T15:49:49.601 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-08-11T15:49:49.601 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-08-11T15:49:49.601 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2023-08-11T15:49:49.601 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2023-08-11T15:49:49.601 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2023-08-11T15:49:49.601 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2023-08-11T15:49:49.601 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2023-08-11T15:49:49.601 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2023-08-11T15:49:49.602 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2023-08-11T15:49:49.602 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2023-08-11T15:49:49.602 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2023-08-11T15:49:49.602 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2023-08-11T15:49:49.602 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-08-11T15:49:49.603 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2023-08-11T15:49:49.603 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = True 2023-08-11T15:49:49.603 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2023-08-11T15:49:49.603 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = True 2023-08-11T15:49:49.603 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-08-11T15:49:49.603 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-08-11T15:49:49.603 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-08-11T15:49:49.604 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2023-08-11T15:49:49.604 INFO:teuthology.orchestra.run.smithi027.stdout: chdir = "" 2023-08-11T15:49:49.604 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-11T15:49:49.604 INFO:teuthology.orchestra.run.smithi027.stdout: auth supported = cephx 2023-08-11T15:49:49.605 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.605 INFO:teuthology.orchestra.run.smithi027.stdout: filestore xattr use omap = true 2023-08-11T15:49:49.605 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.605 INFO:teuthology.orchestra.run.smithi027.stdout: mon clock drift allowed = 1.000 2023-08-11T15:49:49.605 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.605 INFO:teuthology.orchestra.run.smithi027.stdout: osd crush chooseleaf type = 0 2023-08-11T15:49:49.605 INFO:teuthology.orchestra.run.smithi027.stdout: auth debug = true 2023-08-11T15:49:49.606 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.606 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on old message = true 2023-08-11T15:49:49.606 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on bug = true 2023-08-11T15:49:49.606 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.606 INFO:teuthology.orchestra.run.smithi027.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-11T15:49:49.606 INFO:teuthology.orchestra.run.smithi027.stdout: mon pg warn max object skew = 0 2023-08-11T15:49:49.607 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.607 INFO:teuthology.orchestra.run.smithi027.stdout: # disable pg_autoscaler by default for new pools 2023-08-11T15:49:49.607 INFO:teuthology.orchestra.run.smithi027.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-11T15:49:49.607 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.607 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default size = 2 2023-08-11T15:49:49.607 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.607 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow primary affinity = true 2023-08-11T15:49:49.608 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow pg remap = true 2023-08-11T15:49:49.608 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on legacy crush tunables = false 2023-08-11T15:49:49.608 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on crush straw calc version zero = false 2023-08-11T15:49:49.608 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on no sortbitwise = false 2023-08-11T15:49:49.608 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on osd down out interval zero = false 2023-08-11T15:49:49.608 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on too few osds = false 2023-08-11T15:49:49.608 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-11T15:49:49.609 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-11T15:49:49.609 INFO:teuthology.orchestra.run.smithi027.stdout: mon_allow_pool_size_one = true 2023-08-11T15:49:49.609 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.609 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-11T15:49:49.609 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.609 INFO:teuthology.orchestra.run.smithi027.stdout: osd default data pool replay window = 5 2023-08-11T15:49:49.609 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.609 INFO:teuthology.orchestra.run.smithi027.stdout: mon allow pool delete = true 2023-08-11T15:49:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: mon cluster log file level = debug 2023-08-11T15:49:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: debug asserts on shutdown = true 2023-08-11T15:49:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: mon health detail to clog = false 2023-08-11T15:49:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: mon host = "172.21.15.27,172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790]" 2023-08-11T15:49:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = bee60691-4c46-4397-a560-d178aa4db950 2023-08-11T15:49:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.610 INFO:teuthology.orchestra.run.smithi027.stdout:[osd] 2023-08-11T15:49:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: osd journal size = 100 2023-08-11T15:49:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub load threshold = 5.0 2023-08-11T15:49:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub max interval = 600 2023-08-11T15:49:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock profile = high_recovery_ops 2023-08-11T15:49:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: osd recover clone overlap = true 2023-08-11T15:49:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: osd recovery max chunk = 1048576 2023-08-11T15:49:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug shutdown = true 2023-08-11T15:49:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug op order = true 2023-08-11T15:49:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify stray on activate = true 2023-08-11T15:49:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: osd open classes on start = true 2023-08-11T15:49:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug pg log writeout = true 2023-08-11T15:49:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: osd deep scrub update digest min age = 30 2023-08-11T15:49:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: osd map max advance = 10 2023-08-11T15:49:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: journal zero on create = true 2023-08-11T15:49:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: filestore ondisk finisher threads = 3 2023-08-11T15:49:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: filestore apply finisher threads = 3 2023-08-11T15:49:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: bdev debug aio = true 2023-08-11T15:49:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug misdirected ops = true 2023-08-11T15:49:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore block size = 96636764160 2023-08-11T15:49:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore fsck on mount = True 2023-08-11T15:49:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluefs = 20 2023-08-11T15:49:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluestore = 20 2023-08-11T15:49:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-08-11T15:49:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: debug osd = 20 2023-08-11T15:49:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: debug rocksdb = 10 2023-08-11T15:49:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd backfillfull_ratio = 0.85 2023-08-11T15:49:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd full ratio = 0.9 2023-08-11T15:49:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd nearfull ratio = 0.8 2023-08-11T15:49:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: osd failsafe full ratio = 0.95 2023-08-11T15:49:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: osd objectstore = bluestore 2023-08-11T15:49:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2023-08-11T15:49:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.617 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr] 2023-08-11T15:49:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-08-11T15:49:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: debug mgr = 20 2023-08-11T15:49:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2023-08-11T15:49:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2023-08-11T15:49:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2023-08-11T15:49:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2023-08-11T15:49:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: mgr/telemetry/nag = false 2023-08-11T15:49:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.619 INFO:teuthology.orchestra.run.smithi027.stdout:[mon] 2023-08-11T15:49:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-08-11T15:49:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2023-08-11T15:49:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: debug paxos = 20 2023-08-11T15:49:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2023-08-11T15:49:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: mon data avail warn = 5 2023-08-11T15:49:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: mon mgr mkfs grace = 240 2023-08-11T15:49:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2023-08-11T15:49:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd reporter subtree level = osd 2023-08-11T15:49:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd prime pg temp = true 2023-08-11T15:49:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2023-08-11T15:49:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-11T15:49:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: auth mon ticket ttl = 660 # 11m 2023-08-11T15:49:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: auth service ticket ttl = 240 # 4m 2023-08-11T15:49:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: # don't complain about insecure global_id in the test suite 2023-08-11T15:49:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-11T15:49:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-11T15:49:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: # 1m isn't quite enough 2023-08-11T15:49:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: mon_down_mkfs_grace = 2m 2023-08-11T15:49:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_filestore_osds = false 2023-08-11T15:49:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: mon op complaint time = 120 2023-08-11T15:49:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:49:49.625 INFO:teuthology.orchestra.run.smithi027.stdout:[client] 2023-08-11T15:49:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: rgw cache enabled = true 2023-08-11T15:49:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable ops log = true 2023-08-11T15:49:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable usage log = true 2023-08-11T15:49:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-11T15:49:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-11T15:49:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: client mount timeout = 600 2023-08-11T15:49:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: debug client = 20 2023-08-11T15:49:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-08-11T15:49:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = "" 2023-08-11T15:49:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2023-08-11T15:49:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2023-08-11T15:49:49.628 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.a] 2023-08-11T15:49:49.628 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.b] 2023-08-11T15:49:49.629 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.c] 2023-08-11T15:49:49.629 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2023-08-11T15:49:49.629 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c] 2023-08-11T15:49:49.629 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b] 2023-08-11T15:49:49.630 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d] 2023-08-11T15:49:49.630 INFO:teuthology.orchestra.run.smithi027.stdout:[mds] 2023-08-11T15:49:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds = 20 2023-08-11T15:49:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds balancer = 20 2023-08-11T15:49:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-08-11T15:49:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug frag = True 2023-08-11T15:49:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug scatterstat = True 2023-08-11T15:49:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: mds op complaint time = 180 2023-08-11T15:49:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: mds verify scatter = True 2023-08-11T15:49:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2023-08-11T15:49:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2023-08-11T15:49:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2023-08-11T15:49:49.651 INFO:tasks.ceph:Creating admin key on mon.a... 2023-08-11T15:49:49.652 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-08-11T15:49:49.812 INFO:tasks.ceph:Copying monmap to all nodes... 2023-08-11T15:49:49.813 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:49.813 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-08-11T15:49:49.870 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:49.871 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-08-11T15:49:49.927 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:49:49.928 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:49.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-11T15:49:49.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-11T15:49:50.096 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:49:50.097 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:49:50.200 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:49:50.201 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:49:50.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-11T15:49:50.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-11T15:49:50.362 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:49:50.363 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:49:50.461 INFO:tasks.ceph:Setting up mon nodes... 2023-08-11T15:49:50.461 INFO:tasks.ceph:Setting up mgr nodes... 2023-08-11T15:49:50.462 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-08-11T15:49:50.694 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-08-11T15:49:50.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-08-11T15:49:50.916 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-08-11T15:49:50.927 INFO:tasks.ceph:Setting up mds nodes... 2023-08-11T15:49:50.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-08-11T15:49:51.154 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-08-11T15:49:51.164 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-08-11T15:49:51.279 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-08-11T15:49:51.474 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-08-11T15:49:51.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-08-11T15:49:51.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-08-11T15:49:51.783 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-08-11T15:49:51.792 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-08-11T15:49:51.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-08-11T15:49:52.061 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-08-11T15:49:52.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-08-11T15:49:52.164 INFO:tasks.ceph_client:Setting up client nodes... 2023-08-11T15:49:52.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-11T15:49:52.317 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-08-11T15:49:52.398 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-08-11T15:49:52.542 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-08-11T15:49:52.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-08-11T15:49:52.743 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-08-11T15:49:52.806 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-08-11T15:49:52.931 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-08-11T15:49:52.990 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-08-11T15:49:52.990 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-11T15:49:52.991 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-08-11T15:49:53.105 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-11T15:49:53.106 INFO:tasks.ceph:role: osd.0 2023-08-11T15:49:53.106 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:49:53.106 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-11T15:49:53.222 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-11T15:49:53.223 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-11T15:49:53.223 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-11T15:49:53.223 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-11T15:49:53.223 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-11T15:49:53.223 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-08-11T15:49:53.223 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-11T15:49:53.224 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-11T15:49:53.224 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-11T15:49:53.224 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-11T15:49:53.730 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-08-11T15:49:53.738 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-08-11T15:49:53.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-08-11T15:49:53.939 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-08-11T15:49:54.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-08-11T15:49:54.166 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-11T15:49:54.166 INFO:tasks.ceph:role: osd.1 2023-08-11T15:49:54.166 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:49:54.167 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-11T15:49:54.278 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-11T15:49:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-11T15:49:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-11T15:49:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-11T15:49:54.280 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-11T15:49:54.280 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-08-11T15:49:54.280 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-11T15:49:54.280 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-11T15:49:54.280 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-11T15:49:54.280 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-11T15:49:54.772 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-08-11T15:49:54.780 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-08-11T15:49:54.781 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-08-11T15:49:54.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-08-11T15:49:55.043 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-08-11T15:49:55.153 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-11T15:49:55.153 INFO:tasks.ceph:role: osd.2 2023-08-11T15:49:55.153 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:49:55.154 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-11T15:49:55.266 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-11T15:49:55.267 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-11T15:49:55.267 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-11T15:49:55.267 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-11T15:49:55.267 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-11T15:49:55.267 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-08-11T15:49:55.267 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-11T15:49:55.268 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-11T15:49:55.268 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-11T15:49:55.268 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-11T15:49:55.771 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-08-11T15:49:55.779 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-08-11T15:49:55.779 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-08-11T15:49:55.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-08-11T15:49:56.107 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-08-11T15:49:56.217 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-11T15:49:56.217 INFO:tasks.ceph:role: osd.3 2023-08-11T15:49:56.218 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:49:56.218 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-11T15:49:56.332 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-11T15:49:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-11T15:49:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-11T15:49:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-11T15:49:56.333 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-11T15:49:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-08-11T15:49:56.333 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-11T15:49:56.333 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-11T15:49:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-11T15:49:56.333 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-11T15:49:56.822 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-08-11T15:49:56.829 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-08-11T15:49:56.829 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-08-11T15:49:56.990 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-08-11T15:49:57.102 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:49:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:49:57.284+0000 7fbe4f010500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-08-11T15:49:57.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:49:57.290+0000 7fbe4f010500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-08-11T15:49:57.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:49:57.290+0000 7fbe4f010500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-11T15:49:57.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:49:57.290+0000 7fbe4f010500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-11T15:49:57.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:49:57.290+0000 7fbe4f010500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-11T15:49:57.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:49:57.300+0000 7fbe4f010500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-08-11T15:50:00.502 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-08-11T15:50:00.617 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:50:00.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:00.767+0000 7f720040d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-08-11T15:50:00.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:00.768+0000 7f720040d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-08-11T15:50:00.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:00.768+0000 7f720040d500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-11T15:50:00.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:00.768+0000 7f720040d500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-11T15:50:00.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:00.768+0000 7f720040d500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-11T15:50:00.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:00.770+0000 7f720040d500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-08-11T15:50:03.953 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-08-11T15:50:04.070 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:50:04.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:04.232+0000 7f9d4520d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-08-11T15:50:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:04.233+0000 7f9d4520d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-08-11T15:50:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:04.233+0000 7f9d4520d500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-11T15:50:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:04.233+0000 7f9d4520d500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-11T15:50:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:04.233+0000 7f9d4520d500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-11T15:50:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:04.235+0000 7f9d4520d500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-08-11T15:50:07.174 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-08-11T15:50:07.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:50:07.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:07.458+0000 7fb47228c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-08-11T15:50:07.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:07.458+0000 7fb47228c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-08-11T15:50:07.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:07.459+0000 7fb47228c500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-11T15:50:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:07.459+0000 7fb47228c500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-11T15:50:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:07.459+0000 7fb47228c500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-11T15:50:07.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:07.461+0000 7fb47228c500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-08-11T15:50:10.641 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-08-11T15:50:10.751 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-08-11T15:50:10.855 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-11T15:50:10.855 INFO:tasks.ceph:role: osd.4 2023-08-11T15:50:10.855 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:50:10.855 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-11T15:50:10.957 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-11T15:50:10.957 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-11T15:50:10.957 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-11T15:50:10.957 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-11T15:50:10.957 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-11T15:50:10.957 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-08-11T15:50:10.958 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-11T15:50:10.958 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-11T15:50:10.958 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-11T15:50:10.958 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-11T15:50:11.469 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-08-11T15:50:11.476 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-08-11T15:50:11.476 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-08-11T15:50:11.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-08-11T15:50:11.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-08-11T15:50:11.860 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-11T15:50:11.860 INFO:tasks.ceph:role: osd.5 2023-08-11T15:50:11.861 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:50:11.861 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-11T15:50:11.964 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-11T15:50:11.965 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-11T15:50:11.965 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-11T15:50:11.965 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-11T15:50:11.966 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-11T15:50:11.966 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-08-11T15:50:11.966 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-11T15:50:11.966 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-11T15:50:11.967 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-11T15:50:11.967 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-11T15:50:12.446 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-08-11T15:50:12.452 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-08-11T15:50:12.453 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-08-11T15:50:12.604 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-08-11T15:50:12.706 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-08-11T15:50:12.798 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-11T15:50:12.799 INFO:tasks.ceph:role: osd.6 2023-08-11T15:50:12.799 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:50:12.799 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-11T15:50:12.893 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-11T15:50:12.894 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-11T15:50:12.894 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-11T15:50:12.894 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-11T15:50:12.894 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-11T15:50:12.894 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-08-11T15:50:12.895 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-11T15:50:12.895 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-11T15:50:12.895 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-11T15:50:12.895 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-11T15:50:13.404 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-08-11T15:50:13.411 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-08-11T15:50:13.411 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-08-11T15:50:13.590 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-08-11T15:50:13.696 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-08-11T15:50:13.787 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-11T15:50:13.787 INFO:tasks.ceph:role: osd.7 2023-08-11T15:50:13.788 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:50:13.788 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-11T15:50:13.881 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-11T15:50:13.882 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-11T15:50:13.882 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-11T15:50:13.882 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-11T15:50:13.882 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-11T15:50:13.882 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-08-11T15:50:13.883 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-11T15:50:13.883 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-11T15:50:13.883 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-11T15:50:13.883 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-11T15:50:14.337 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-08-11T15:50:14.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-08-11T15:50:14.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-08-11T15:50:14.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-08-11T15:50:14.583 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:50:14.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:14.728+0000 7f357c974500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-08-11T15:50:14.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:14.728+0000 7f357c974500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-08-11T15:50:14.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:14.729+0000 7f357c974500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-11T15:50:14.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:14.729+0000 7f357c974500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-11T15:50:14.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:14.729+0000 7f357c974500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-11T15:50:14.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:14.732+0000 7f357c974500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-08-11T15:50:16.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:50:16.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:50:17.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-08-11T15:50:17.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:50:17.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:17.899+0000 7f0090606500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-08-11T15:50:17.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:17.900+0000 7f0090606500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-08-11T15:50:17.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:17.900+0000 7f0090606500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-11T15:50:17.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:17.900+0000 7f0090606500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-11T15:50:17.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:17.900+0000 7f0090606500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-11T15:50:17.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:17.902+0000 7f0090606500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-08-11T15:50:21.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-08-11T15:50:21.189 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:50:21.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:21.327+0000 7fd6bec11500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-08-11T15:50:21.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:21.327+0000 7fd6bec11500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-08-11T15:50:21.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:21.327+0000 7fd6bec11500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-11T15:50:21.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:21.327+0000 7fd6bec11500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-11T15:50:21.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:21.327+0000 7fd6bec11500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-11T15:50:21.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:21.330+0000 7fd6bec11500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-08-11T15:50:24.257 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-08-11T15:50:24.360 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:50:24.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:24.500+0000 7f4fb5220500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-08-11T15:50:24.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:24.501+0000 7f4fb5220500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-08-11T15:50:24.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:24.501+0000 7f4fb5220500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-11T15:50:24.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:24.501+0000 7f4fb5220500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-11T15:50:24.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:24.501+0000 7f4fb5220500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-11T15:50:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:24.503+0000 7f4fb5220500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-08-11T15:50:27.445 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-08-11T15:50:27.543 INFO:tasks.ceph:Reading keys from all nodes... 2023-08-11T15:50:27.544 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:27.544 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-08-11T15:50:27.660 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:27.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-08-11T15:50:27.770 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:27.770 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-08-11T15:50:27.864 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:27.864 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-08-11T15:50:27.955 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:27.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-08-11T15:50:28.108 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:28.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-08-11T15:50:28.217 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:28.217 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-08-11T15:50:28.323 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:28.323 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-08-11T15:50:28.427 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:28.427 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-08-11T15:50:28.521 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:28.522 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-08-11T15:50:28.619 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:28.620 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-08-11T15:50:28.713 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:28.713 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-08-11T15:50:28.853 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:28.853 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-08-11T15:50:28.944 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:28.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-08-11T15:50:29.029 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:29.030 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-08-11T15:50:29.093 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:29.094 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-08-11T15:50:29.153 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:29.154 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-08-11T15:50:29.214 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:29.214 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-08-11T15:50:29.265 INFO:tasks.ceph:Adding keys to all mons... 2023-08-11T15:50:29.266 DEBUG:teuthology.orchestra.run.smithi027:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-11T15:50:29.275 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-11T15:50:29.366 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.y] 2023-08-11T15:50:29.366 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCeWNZkx1hfKRAAJyIEwNyF3oZjsXNZJiVABQ== 2023-08-11T15:50:29.366 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2023-08-11T15:50:29.366 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCfWNZkJr0nCRAAt5jrDGXTdREF1aX+JJ6y1g== 2023-08-11T15:50:29.367 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2023-08-11T15:50:29.367 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCfWNZk5UU/HBAAmBzoJNZcHdNXx1RCke7mkQ== 2023-08-11T15:50:29.367 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2023-08-11T15:50:29.367 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQClWNZkD1sWERAAXKnWuWlUQdfesKGslW1yzA== 2023-08-11T15:50:29.367 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2023-08-11T15:50:29.367 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCoWNZkCnrcLRAABS2uWXujbf29rznDr30fuA== 2023-08-11T15:50:29.367 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2023-08-11T15:50:29.367 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCsWNZketPyDRAAY7pNdGg8whP6s3Wkjl+BaQ== 2023-08-11T15:50:29.368 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2023-08-11T15:50:29.368 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCvWNZkvkVeGxAAdnmiTpGGjOzZ2X/aTtZR6A== 2023-08-11T15:50:29.368 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2023-08-11T15:50:29.368 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCeWNZkEM6VNhAAJJYDJ0TrXp2W0ISyeGJ90w== 2023-08-11T15:50:29.368 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2023-08-11T15:50:29.368 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCfWNZklkmgLhAA5Of9CMgj/iq1k0xVRkHrrg== 2023-08-11T15:50:29.368 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2023-08-11T15:50:29.369 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCgWNZkYGufAxAALxZd1NcHLT2Y46Nfb+5wMA== 2023-08-11T15:50:29.369 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.4] 2023-08-11T15:50:29.369 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQC2WNZkMdJ+KxAAmyGKek7HeVzxQbmtt+V/+Q== 2023-08-11T15:50:29.369 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.5] 2023-08-11T15:50:29.369 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQC5WNZkSlW9NRAAp+mD8cyG363Kula78uB33A== 2023-08-11T15:50:29.369 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.6] 2023-08-11T15:50:29.369 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQC9WNZkwxCbExAAelXzMnXkgUUHE2kgRKoVwA== 2023-08-11T15:50:29.369 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.7] 2023-08-11T15:50:29.370 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDAWNZkZMPxHRAAEM/ndOZQ1wWBI3yhmVhqKA== 2023-08-11T15:50:29.370 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2023-08-11T15:50:29.370 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCgWNZkD5/iEhAAZ4I6N46HjdmeGnn6p1vIvg== 2023-08-11T15:50:29.370 INFO:teuthology.orchestra.run.smithi119.stdout:[client.2] 2023-08-11T15:50:29.370 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCgWNZkaw5GIBAA2hBr1tFhgg8c8XwHQkYngg== 2023-08-11T15:50:29.370 INFO:teuthology.orchestra.run.smithi119.stdout:[client.1] 2023-08-11T15:50:29.370 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCgWNZk7aNKLBAAewcNiTQiIudZtZI3a8Lzng== 2023-08-11T15:50:29.370 INFO:teuthology.orchestra.run.smithi119.stdout:[client.3] 2023-08-11T15:50:29.371 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCgWNZk7Rp+NxAAA2iTgfWpzexHHUd2PpVsig== 2023-08-11T15:50:29.382 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.y] 2023-08-11T15:50:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCeWNZkx1hfKRAAJyIEwNyF3oZjsXNZJiVABQ== 2023-08-11T15:50:29.382 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2023-08-11T15:50:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCfWNZkJr0nCRAAt5jrDGXTdREF1aX+JJ6y1g== 2023-08-11T15:50:29.382 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c] 2023-08-11T15:50:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCfWNZk5UU/HBAAmBzoJNZcHdNXx1RCke7mkQ== 2023-08-11T15:50:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.0] 2023-08-11T15:50:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQClWNZkD1sWERAAXKnWuWlUQdfesKGslW1yzA== 2023-08-11T15:50:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.1] 2023-08-11T15:50:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCoWNZkCnrcLRAABS2uWXujbf29rznDr30fuA== 2023-08-11T15:50:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.2] 2023-08-11T15:50:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCsWNZketPyDRAAY7pNdGg8whP6s3Wkjl+BaQ== 2023-08-11T15:50:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.3] 2023-08-11T15:50:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCvWNZkvkVeGxAAdnmiTpGGjOzZ2X/aTtZR6A== 2023-08-11T15:50:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.x] 2023-08-11T15:50:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCeWNZkEM6VNhAAJJYDJ0TrXp2W0ISyeGJ90w== 2023-08-11T15:50:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b] 2023-08-11T15:50:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCfWNZklkmgLhAA5Of9CMgj/iq1k0xVRkHrrg== 2023-08-11T15:50:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d] 2023-08-11T15:50:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCgWNZkYGufAxAALxZd1NcHLT2Y46Nfb+5wMA== 2023-08-11T15:50:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.4] 2023-08-11T15:50:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQC2WNZkMdJ+KxAAmyGKek7HeVzxQbmtt+V/+Q== 2023-08-11T15:50:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.5] 2023-08-11T15:50:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQC5WNZkSlW9NRAAp+mD8cyG363Kula78uB33A== 2023-08-11T15:50:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.6] 2023-08-11T15:50:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQC9WNZkwxCbExAAelXzMnXkgUUHE2kgRKoVwA== 2023-08-11T15:50:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.7] 2023-08-11T15:50:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDAWNZkZMPxHRAAEM/ndOZQ1wWBI3yhmVhqKA== 2023-08-11T15:50:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2023-08-11T15:50:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCgWNZkD5/iEhAAZ4I6N46HjdmeGnn6p1vIvg== 2023-08-11T15:50:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:[client.2] 2023-08-11T15:50:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCgWNZkaw5GIBAA2hBr1tFhgg8c8XwHQkYngg== 2023-08-11T15:50:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:[client.1] 2023-08-11T15:50:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCgWNZk7aNKLBAAewcNiTQiIudZtZI3a8Lzng== 2023-08-11T15:50:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:[client.3] 2023-08-11T15:50:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCgWNZk7Rp+NxAAA2iTgfWpzexHHUd2PpVsig== 2023-08-11T15:50:29.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-11T15:50:29.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-11T15:50:29.550 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-11T15:50:29.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-11T15:50:29.715 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-11T15:50:29.722 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-11T15:50:29.875 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:29.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:30.022 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:30.027 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:30.181 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:30.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:30.331 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:30.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:30.489 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-11T15:50:30.497 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-11T15:50:30.644 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-11T15:50:30.649 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-11T15:50:30.806 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-11T15:50:30.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-11T15:50:30.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:30.978 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:31.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:31.147 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:31.309 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:31.315 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:31.471 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:31.478 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-11T15:50:31.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-11T15:50:31.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-11T15:50:31.796 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-11T15:50:31.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-11T15:50:31.961 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-11T15:50:31.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-11T15:50:32.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-11T15:50:32.132 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-11T15:50:32.288 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-08-11T15:50:32.289 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-08-11T15:50:32.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-11T15:50:32.550 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-08-11T15:50:32.651 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-08-11T15:50:32.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-11T15:50:32.906 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-08-11T15:50:32.998 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-08-11T15:50:33.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-11T15:50:33.248 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-08-11T15:50:33.334 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:50:33.344 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-11T15:50:33.397 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-08-11T15:50:33.397 INFO:tasks.ceph.mon.a:Restarting daemon 2023-08-11T15:50:33.398 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-08-11T15:50:33.403 INFO:tasks.ceph.mon.a:Started 2023-08-11T15:50:33.404 INFO:tasks.ceph.mon.b:Restarting daemon 2023-08-11T15:50:33.404 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-08-11T15:50:33.413 INFO:tasks.ceph.mon.b:Started 2023-08-11T15:50:33.413 INFO:tasks.ceph.mon.c:Restarting daemon 2023-08-11T15:50:33.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-08-11T15:50:33.422 INFO:tasks.ceph.mon.c:Started 2023-08-11T15:50:33.422 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-08-11T15:50:33.423 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-08-11T15:50:33.424 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-08-11T15:50:33.430 INFO:tasks.ceph.mgr.y:Started 2023-08-11T15:50:33.430 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-08-11T15:50:33.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-08-11T15:50:33.436 INFO:tasks.ceph.mgr.x:Started 2023-08-11T15:50:33.436 INFO:tasks.ceph:Setting crush tunables to default 2023-08-11T15:50:33.436 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd crush tunables default 2023-08-11T15:50:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.625+0000 7f8fc114e640 1 Processor -- start 2023-08-11T15:50:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.625+0000 7f8fc114e640 1 -- start start 2023-08-11T15:50:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.626+0000 7f8fc114e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8fbc15c550 0x7f8fbc0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.626+0000 7f8fc114e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8fbc0d1440 0x7f8fbc0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.626+0000 7f8fc114e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fbc0b19c0 con 0x7f8fbc0cd170 2023-08-11T15:50:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.626+0000 7f8fc114e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fbc0cd5e0 con 0x7f8fbc15c550 2023-08-11T15:50:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.626+0000 7f8fc114e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fbc0d28e0 con 0x7f8fbc0d1440 2023-08-11T15:50:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fbad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8fbc15c550 0x7f8fbc0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fbad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8fbc15c550 0x7f8fbc0d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38856/0 (socket says 172.21.15.27:38856) 2023-08-11T15:50:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fbad9d640 1 -- 172.21.15.27:0/2306237403 learned_addr learned my addr 172.21.15.27:0/2306237403 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:33.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fba59c640 1 --2- 172.21.15.27:0/2306237403 >> v2:172.21.15.119:3300/0 conn(0x7f8fbc0d1440 0x7f8fbc0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:33.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fbad9d640 1 -- 172.21.15.27:0/2306237403 >> v2:172.21.15.27:3300/0 conn(0x7f8fbc15c550 msgr2=0x7f8fbc0d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-11T15:50:33.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fbad9d640 1 -- 172.21.15.27:0/2306237403 >> v2:172.21.15.27:3300/0 conn(0x7f8fbc15c550 msgr2=0x7f8fbc0d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-11T15:50:33.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fbad9d640 1 --2- 172.21.15.27:0/2306237403 >> v2:172.21.15.27:3300/0 conn(0x7f8fbc15c550 0x7f8fbc0d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-11T15:50:33.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fbad9d640 1 --2- 172.21.15.27:0/2306237403 >> v2:172.21.15.27:3300/0 conn(0x7f8fbc15c550 0x7f8fbc0d0f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:50:33.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fba59c640 1 -- 172.21.15.27:0/2306237403 >> v2:172.21.15.119:3300/0 conn(0x7f8fbc0d1440 msgr2=0x7f8fbc0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-08-11T15:50:33.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fba59c640 1 -- 172.21.15.27:0/2306237403 >> v2:172.21.15.119:3300/0 conn(0x7f8fbc0d1440 msgr2=0x7f8fbc0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-11T15:50:33.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fba59c640 1 --2- 172.21.15.27:0/2306237403 >> v2:172.21.15.119:3300/0 conn(0x7f8fbc0d1440 0x7f8fbc0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-11T15:50:33.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.627+0000 7f8fba59c640 1 --2- 172.21.15.27:0/2306237403 >> v2:172.21.15.119:3300/0 conn(0x7f8fbc0d1440 0x7f8fbc0ccc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:50:33.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641692657 0 0) 0x7f8fbc0b19c0 con 0x7f8fbc0cd170 2023-08-11T15:50:33.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fa4003520 con 0x7f8fbc0cd170 2023-08-11T15:50:33.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3326630018 0 0) 0x7f8fa4003520 con 0x7f8fbc0cd170 2023-08-11T15:50:33.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 >> v2:172.21.15.119:3300/0 conn(0x7f8fbc0d1440 msgr2=0x7f8fbc0ccc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:50:33.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 --2- 172.21.15.27:0/2306237403 >> v2:172.21.15.119:3300/0 conn(0x7f8fbc0d1440 0x7f8fbc0ccc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 >> v2:172.21.15.27:3300/0 conn(0x7f8fbc15c550 msgr2=0x7f8fbc0d0f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:50:33.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 --2- 172.21.15.27:0/2306237403 >> v2:172.21.15.27:3300/0 conn(0x7f8fbc15c550 0x7f8fbc0d0f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc0d1a30 con 0x7f8fbc0cd170 2023-08-11T15:50:33.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f8fb40019f0 con 0x7f8fbc0cd170 2023-08-11T15:50:33.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 >> v1:172.21.15.27:6789/0 conn(0x7f8fbc0cd170 legacy=0x7f8fbc0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:33.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa4003970 0x7f8fa4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:33.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fa40047c0 0x7f8fa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:33.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa4005180 0x7f8fa400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:33.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0cd5e0 con 0x7f8fa4005180 2023-08-11T15:50:33.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0d28e0 con 0x7f8fa4003970 2023-08-11T15:50:33.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.643+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa400c080 con 0x7f8fa40047c0 2023-08-11T15:50:33.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.644+0000 7f8fbb59e640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa4005180 0x7f8fa400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:33.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.644+0000 7f8fbad9d640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa4003970 0x7f8fa4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:33.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.644+0000 7f8fba59c640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fa40047c0 0x7f8fa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:33.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.644+0000 7f8fbb59e640 1 -- 172.21.15.27:0/2306237403 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fa40047c0 msgr2=0x7f8fa4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:33.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.644+0000 7f8fbb59e640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fa40047c0 0x7f8fa4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.644+0000 7f8fbb59e640 1 -- 172.21.15.27:0/2306237403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa4003970 msgr2=0x7f8fa4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:33.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.644+0000 7f8fbb59e640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa4003970 0x7f8fa4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.644+0000 7f8fbb59e640 1 -- 172.21.15.27:0/2306237403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa4003520 con 0x7f8fa4005180 2023-08-11T15:50:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.645+0000 7f8fbb59e640 1 -- 172.21.15.27:0/2306237403 reap_dead start 2023-08-11T15:50:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.646+0000 7f8fbb59e640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa4005180 0x7f8fa400b980 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400c4c0 tx=0x7f8fa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfe53479c22fdb7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.646+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb402cd80 con 0x7f8fa4005180 2023-08-11T15:50:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.646+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb4034440 con 0x7f8fa4005180 2023-08-11T15:50:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.646+0000 7f8fb9d9b640 1 -- 172.21.15.27:0/2306237403 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40347f0 con 0x7f8fa4005180 2023-08-11T15:50:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.647+0000 7f8fc114e640 1 -- 172.21.15.27:0/2306237403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa4005180 msgr2=0x7f8fa400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.647+0000 7f8fc114e640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa4005180 0x7f8fa400b980 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400c4c0 tx=0x7f8fa40057f0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.647+0000 7f8fc114e640 1 -- 172.21.15.27:0/2306237403 shutdown_connections 2023-08-11T15:50:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.647+0000 7f8fc114e640 1 --2- 172.21.15.27:0/2306237403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fa4005180 0x7f8fa400b980 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.647+0000 7f8fc114e640 1 -- 172.21.15.27:0/2306237403 >> 172.21.15.27:0/2306237403 conn(0x7f8fbc071320 msgr2=0x7f8fbc071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.647+0000 7f8fc114e640 1 -- 172.21.15.27:0/2306237403 shutdown_connections 2023-08-11T15:50:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.647+0000 7f8fc114e640 1 -- 172.21.15.27:0/2306237403 wait complete. 2023-08-11T15:50:33.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.649+0000 7f8fc114e640 1 Processor -- start 2023-08-11T15:50:33.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.650+0000 7f8fc114e640 1 -- start start 2023-08-11T15:50:33.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fc114e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fa4005180 0x7f8fbc0c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:33.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fc114e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fbc0cc2b0 0x7f8fbc0a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:33.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fc114e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fbc1f1e10 0x7f8fbc1f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:33.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fc114e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc15bf50 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fc114e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0c7fb0 con 0x7f8fbc1f1e10 2023-08-11T15:50:33.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fc114e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0cae50 con 0x7f8fa4005180 2023-08-11T15:50:33.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fbad9d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fa4005180 0x7f8fbc0c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:33.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fbad9d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fa4005180 0x7f8fbc0c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:58416/0 (socket says 172.21.15.27:58416) 2023-08-11T15:50:33.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fbad9d640 1 -- 172.21.15.27:0/3570866807 learned_addr learned my addr 172.21.15.27:0/3570866807 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:33.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fba59c640 1 --2- 172.21.15.27:0/3570866807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fbc0cc2b0 0x7f8fbc0a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:33.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.651+0000 7f8fbb59e640 1 --2- 172.21.15.27:0/3570866807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fbc1f1e10 0x7f8fbc1f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:33.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.652+0000 7f8fba59c640 1 -- 172.21.15.27:0/3570866807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fa4005180 msgr2=0x7f8fbc0c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:33.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.652+0000 7f8fba59c640 1 --2- 172.21.15.27:0/3570866807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fa4005180 0x7f8fbc0c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.652+0000 7f8fba59c640 1 -- 172.21.15.27:0/3570866807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fbc1f1e10 msgr2=0x7f8fbc1f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:33.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.652+0000 7f8fba59c640 1 --2- 172.21.15.27:0/3570866807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fbc1f1e10 0x7f8fbc1f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.652+0000 7f8fba59c640 1 -- 172.21.15.27:0/3570866807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc0cb130 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.652+0000 7f8fba59c640 1 --2- 172.21.15.27:0/3570866807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fbc0cc2b0 0x7f8fbc0a8d80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0cedb0 tx=0x7f8fb000f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:33.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.653+0000 7f8fa37fe640 1 -- 172.21.15.27:0/3570866807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0004440 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.653+0000 7f8fa37fe640 1 -- 172.21.15.27:0/3570866807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0003610 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.653+0000 7f8fa37fe640 1 -- 172.21.15.27:0/3570866807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00039a0 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.653+0000 7f8fc114e640 1 -- 172.21.15.27:0/3570866807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fbc0c7960 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.653+0000 7f8fc114e640 1 -- 172.21.15.27:0/3570866807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc0a92c0 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.653+0000 7f8fa37fe640 1 -- 172.21.15.27:0/3570866807 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8fb001f460 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.653+0000 7f8fa37fe640 1 -- 172.21.15.27:0/3570866807 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8fb0015410 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.654+0000 7f8fc114e640 1 -- 172.21.15.27:0/3570866807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc0c8120 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.657+0000 7f8fa37fe640 1 -- 172.21.15.27:0/3570866807 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f8fb001f8f0 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.731+0000 7f8fc114e640 1 -- 172.21.15.27:0/3570866807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8fbc073990 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.737+0000 7f8fa37fe640 1 -- 172.21.15.27:0/3570866807 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8fb001fa90 con 0x7f8fbc0cc2b0 2023-08-11T15:50:33.740 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2023-08-11T15:50:33.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.739+0000 7f8fc114e640 1 -- 172.21.15.27:0/3570866807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fbc0cc2b0 msgr2=0x7f8fbc0a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.739+0000 7f8fc114e640 1 --2- 172.21.15.27:0/3570866807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fbc0cc2b0 0x7f8fbc0a8d80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0cedb0 tx=0x7f8fb000f8f0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.739+0000 7f8fc114e640 1 -- 172.21.15.27:0/3570866807 shutdown_connections 2023-08-11T15:50:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.739+0000 7f8fc114e640 1 --2- 172.21.15.27:0/3570866807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fbc1f1e10 0x7f8fbc1f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.739+0000 7f8fc114e640 1 --2- 172.21.15.27:0/3570866807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fbc0cc2b0 0x7f8fbc0a8d80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.739+0000 7f8fc114e640 1 --2- 172.21.15.27:0/3570866807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fa4005180 0x7f8fbc0c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.739+0000 7f8fc114e640 1 -- 172.21.15.27:0/3570866807 >> 172.21.15.27:0/3570866807 conn(0x7f8fbc071320 msgr2=0x7f8fbc0b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.739+0000 7f8fc114e640 1 -- 172.21.15.27:0/3570866807 shutdown_connections 2023-08-11T15:50:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:33.739+0000 7f8fc114e640 1 -- 172.21.15.27:0/3570866807 wait complete. 2023-08-11T15:50:33.759 INFO:tasks.ceph:check_enable_crimson: False 2023-08-11T15:50:33.759 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-08-11T15:50:33.760 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:33.760 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-08-11T15:50:33.836 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:33.833+0000 7ff44e432040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T15:50:33.851 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:33.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-08-11T15:50:33.948 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:33.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-08-11T15:50:33.981 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:33.980+0000 7f849d04f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T15:50:34.050 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:50:34.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-08-11T15:50:34.142 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:34.142 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-08-11T15:50:34.234 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:34.235 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-08-11T15:50:34.320 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:34.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-08-11T15:50:34.455 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:50:34.456 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-08-11T15:50:34.549 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new c6070354-9b1d-4274-9672-2ae236bbb57f 0 2023-08-11T15:50:34.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.792+0000 7fc544a4d640 1 Processor -- start 2023-08-11T15:50:34.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.792+0000 7fc544a4d640 1 -- start start 2023-08-11T15:50:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.793+0000 7fc544a4d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54015c620 0x7fc5400b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.793+0000 7fc544a4d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5400cfb00 0x7fc5400d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.793+0000 7fc544a4d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5400cda30 con 0x7fc5400b82f0 2023-08-11T15:50:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.793+0000 7fc544a4d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5400ccc10 con 0x7fc5400cfb00 2023-08-11T15:50:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.793+0000 7fc544a4d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc540161110 con 0x7fc54015c620 2023-08-11T15:50:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.794+0000 7fc53e59c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54015c620 0x7fc5400b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.794+0000 7fc537fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc5400b82f0 0x7fc5400cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46104/0 (socket says 172.21.15.119:46104) 2023-08-11T15:50:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.794+0000 7fc537fff640 1 -- 172.21.15.119:0/1602352076 learned_addr learned my addr 172.21.15.119:0/1602352076 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-08-11T15:50:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.794+0000 7fc53ed9d640 1 --2- 172.21.15.119:0/1602352076 >> v2:172.21.15.119:3300/0 conn(0x7fc5400cfb00 0x7fc5400d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.794+0000 7fc53e59c640 1 -- 172.21.15.119:0/1602352076 >> v2:172.21.15.119:3300/0 conn(0x7fc5400cfb00 msgr2=0x7fc5400d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.794+0000 7fc53e59c640 1 --2- 172.21.15.119:0/1602352076 >> v2:172.21.15.119:3300/0 conn(0x7fc5400cfb00 0x7fc5400d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.794+0000 7fc53dd9b640 1 -- 172.21.15.119:0/1602352076 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839237775 0 0) 0x7fc5400cda30 con 0x7fc5400b82f0 2023-08-11T15:50:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.795+0000 7fc53e59c640 1 -- 172.21.15.119:0/1602352076 >> v1:172.21.15.119:6789/0 conn(0x7fc5400b82f0 legacy=0x7fc5400cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.795+0000 7fc53e59c640 1 -- 172.21.15.119:0/1602352076 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5400cce20 con 0x7fc54015c620 2023-08-11T15:50:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.796+0000 7fc53e59c640 1 --2- 172.21.15.119:0/1602352076 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54015c620 0x7fc5400b7db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc5300097a0 tx=0x7fc53002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d74d344ebe212df server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.796+0000 7fc53dd9b640 1 -- 172.21.15.119:0/1602352076 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53002d040 con 0x7fc54015c620 2023-08-11T15:50:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.796+0000 7fc53dd9b640 1 -- 172.21.15.119:0/1602352076 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc530031810 con 0x7fc54015c620 2023-08-11T15:50:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.796+0000 7fc53dd9b640 1 -- 172.21.15.119:0/1602352076 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530031ba0 con 0x7fc54015c620 2023-08-11T15:50:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.796+0000 7fc544a4d640 1 -- 172.21.15.119:0/1602352076 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54015c620 msgr2=0x7fc5400b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.796+0000 7fc544a4d640 1 --2- 172.21.15.119:0/1602352076 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54015c620 0x7fc5400b7db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc5300097a0 tx=0x7fc53002b8b0 comp rx=0 tx=0).stop 2023-08-11T15:50:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.797+0000 7fc544a4d640 1 -- 172.21.15.119:0/1602352076 shutdown_connections 2023-08-11T15:50:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.797+0000 7fc544a4d640 1 --2- 172.21.15.119:0/1602352076 >> v2:172.21.15.119:3300/0 conn(0x7fc5400cfb00 0x7fc5400d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.797+0000 7fc544a4d640 1 --2- 172.21.15.119:0/1602352076 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54015c620 0x7fc5400b7db0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.797+0000 7fc544a4d640 1 -- 172.21.15.119:0/1602352076 >> 172.21.15.119:0/1602352076 conn(0x7fc540071110 msgr2=0x7fc5400714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:34.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.797+0000 7fc544a4d640 1 -- 172.21.15.119:0/1602352076 shutdown_connections 2023-08-11T15:50:34.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.797+0000 7fc544a4d640 1 -- 172.21.15.119:0/1602352076 wait complete. 2023-08-11T15:50:34.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.798+0000 7fc544a4d640 1 Processor -- start 2023-08-11T15:50:34.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.799+0000 7fc544a4d640 1 -- start start 2023-08-11T15:50:34.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.799+0000 7fc544a4d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5400b82f0 0x7fc540156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:34.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.799+0000 7fc544a4d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5400cfb00 0x7fc5400ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:34.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.799+0000 7fc544a4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54015c620 0x7fc5400d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:34.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.799+0000 7fc544a4d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54015e0c0 con 0x7fc5400b82f0 2023-08-11T15:50:34.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.799+0000 7fc544a4d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54015d4b0 con 0x7fc54015c620 2023-08-11T15:50:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc544a4d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54015d790 con 0x7fc5400cfb00 2023-08-11T15:50:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc53ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54015c620 0x7fc5400d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc537fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5400cfb00 0x7fc5400ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc53ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54015c620 0x7fc5400d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44706/0 (socket says 172.21.15.119:44706) 2023-08-11T15:50:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc53ed9d640 1 -- 172.21.15.119:0/739167803 learned_addr learned my addr 172.21.15.119:0/739167803 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc53e59c640 1 --2- 172.21.15.119:0/739167803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5400b82f0 0x7fc540156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc53ed9d640 1 -- 172.21.15.119:0/739167803 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5400cfb00 msgr2=0x7fc5400ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc53ed9d640 1 --2- 172.21.15.119:0/739167803 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5400cfb00 0x7fc5400ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc53ed9d640 1 -- 172.21.15.119:0/739167803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5400b82f0 msgr2=0x7fc540156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc53ed9d640 1 --2- 172.21.15.119:0/739167803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5400b82f0 0x7fc540156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc53ed9d640 1 -- 172.21.15.119:0/739167803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc530009400 con 0x7fc54015c620 2023-08-11T15:50:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.800+0000 7fc53ed9d640 1 --2- 172.21.15.119:0/739167803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54015c620 0x7fc5400d0b00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc53800efc0 tx=0x7fc53800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.801+0000 7fc536ffd640 1 -- 172.21.15.119:0/739167803 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc538009ca0 con 0x7fc54015c620 2023-08-11T15:50:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.801+0000 7fc536ffd640 1 -- 172.21.15.119:0/739167803 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53800f040 con 0x7fc54015c620 2023-08-11T15:50:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.801+0000 7fc544a4d640 1 -- 172.21.15.119:0/739167803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54015da70 con 0x7fc54015c620 2023-08-11T15:50:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.801+0000 7fc536ffd640 1 -- 172.21.15.119:0/739167803 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5380146c0 con 0x7fc54015c620 2023-08-11T15:50:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.801+0000 7fc544a4d640 1 -- 172.21.15.119:0/739167803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5400ca550 con 0x7fc54015c620 2023-08-11T15:50:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.801+0000 7fc536ffd640 1 -- 172.21.15.119:0/739167803 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc538014b20 con 0x7fc54015c620 2023-08-11T15:50:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.801+0000 7fc536ffd640 1 -- 172.21.15.119:0/739167803 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc53801de20 con 0x7fc54015c620 2023-08-11T15:50:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.802+0000 7fc544a4d640 1 -- 172.21.15.119:0/739167803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54015e230 con 0x7fc54015c620 2023-08-11T15:50:34.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.805+0000 7fc536ffd640 1 -- 172.21.15.119:0/739167803 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fc53803bb20 con 0x7fc54015c620 2023-08-11T15:50:34.836 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:34.834+0000 7ff44e432040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T15:50:34.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.883+0000 7fc544a4d640 1 -- 172.21.15.119:0/739167803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c6070354-9b1d-4274-9672-2ae236bbb57f", "id": 0} v 0) v1 -- 0x7fc540073990 con 0x7fc54015c620 2023-08-11T15:50:34.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.890+0000 7fc536ffd640 1 -- 172.21.15.119:0/739167803 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c6070354-9b1d-4274-9672-2ae236bbb57f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc53802a030 con 0x7fc54015c620 2023-08-11T15:50:34.892 INFO:teuthology.orchestra.run.smithi119.stdout:0 2023-08-11T15:50:34.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.891+0000 7fc544a4d640 1 -- 172.21.15.119:0/739167803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54015c620 msgr2=0x7fc5400d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:34.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.891+0000 7fc544a4d640 1 --2- 172.21.15.119:0/739167803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54015c620 0x7fc5400d0b00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc53800efc0 tx=0x7fc53800c490 comp rx=0 tx=0).stop 2023-08-11T15:50:34.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.891+0000 7fc544a4d640 1 -- 172.21.15.119:0/739167803 shutdown_connections 2023-08-11T15:50:34.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.891+0000 7fc544a4d640 1 --2- 172.21.15.119:0/739167803 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5400cfb00 0x7fc5400ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:34.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.891+0000 7fc544a4d640 1 --2- 172.21.15.119:0/739167803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54015c620 0x7fc5400d0b00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:34.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.891+0000 7fc544a4d640 1 --2- 172.21.15.119:0/739167803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5400b82f0 0x7fc540156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:34.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.891+0000 7fc544a4d640 1 -- 172.21.15.119:0/739167803 >> 172.21.15.119:0/739167803 conn(0x7fc540071110 msgr2=0x7fc5400cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:34.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.891+0000 7fc544a4d640 1 -- 172.21.15.119:0/739167803 shutdown_connections 2023-08-11T15:50:34.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:34.891+0000 7fc544a4d640 1 -- 172.21.15.119:0/739167803 wait complete. 2023-08-11T15:50:34.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 2eed2b1c-7d9f-40fb-a154-cbf8b99dd988 1 2023-08-11T15:50:34.976 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:34.974+0000 7f849d04f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T15:50:35.003 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:35.000+0000 7ff44e432040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T15:50:35.063 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:35.060+0000 7ff44e432040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T15:50:35.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.099+0000 7f1d2f045640 1 Processor -- start 2023-08-11T15:50:35.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.100+0000 7f1d2f045640 1 -- start start 2023-08-11T15:50:35.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.101+0000 7f1d2f045640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1d280cc990 0x7f1d280cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.101+0000 7f1d2f045640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280d0d20 0x7f1d280ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.101+0000 7f1d2f045640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d280cabd0 con 0x7f1d280ccd70 2023-08-11T15:50:35.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.101+0000 7f1d2f045640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d280ceff0 con 0x7f1d280cc990 2023-08-11T15:50:35.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.101+0000 7f1d2f045640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280cae00 con 0x7f1d280d0d20 2023-08-11T15:50:35.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.101+0000 7f1d2e043640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1d280cc990 0x7f1d280cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.101+0000 7f1d2e844640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1d280ccd70 0x7f1d280cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46108/0 (socket says 172.21.15.119:46108) 2023-08-11T15:50:35.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.102+0000 7f1d2e844640 1 -- 172.21.15.119:0/1108445122 learned_addr learned my addr 172.21.15.119:0/1108445122 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-08-11T15:50:35.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.102+0000 7f1d2d842640 1 --2- 172.21.15.119:0/1108445122 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280d0d20 0x7f1d280ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.102+0000 7f1d2e043640 1 -- 172.21.15.119:0/1108445122 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280d0d20 msgr2=0x7f1d280ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:35.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.102+0000 7f1d2e043640 1 --2- 172.21.15.119:0/1108445122 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280d0d20 0x7f1d280ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.102+0000 7f1d2e043640 1 -- 172.21.15.119:0/1108445122 >> v1:172.21.15.119:6789/0 conn(0x7f1d280ccd70 legacy=0x7f1d280cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.102+0000 7f1d2e043640 1 -- 172.21.15.119:0/1108445122 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d280cb0e0 con 0x7f1d280cc990 2023-08-11T15:50:35.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.103+0000 7f1d2e043640 1 --2- 172.21.15.119:0/1108445122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d280cc990 0x7f1d280cbbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009790 tx=0x7f1d2402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c6a7f29f5dc0169 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:35.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.103+0000 7f1d2d041640 1 -- 172.21.15.119:0/1108445122 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2402d040 con 0x7f1d280cc990 2023-08-11T15:50:35.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.103+0000 7f1d2d041640 1 -- 172.21.15.119:0/1108445122 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d24031810 con 0x7f1d280cc990 2023-08-11T15:50:35.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.104+0000 7f1d2d041640 1 -- 172.21.15.119:0/1108445122 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24031ba0 con 0x7f1d280cc990 2023-08-11T15:50:35.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.104+0000 7f1d2f045640 1 -- 172.21.15.119:0/1108445122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d280cc990 msgr2=0x7f1d280cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.104+0000 7f1d2f045640 1 --2- 172.21.15.119:0/1108445122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d280cc990 0x7f1d280cbbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009790 tx=0x7f1d2402b8b0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.104+0000 7f1d2f045640 1 -- 172.21.15.119:0/1108445122 shutdown_connections 2023-08-11T15:50:35.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.104+0000 7f1d2f045640 1 --2- 172.21.15.119:0/1108445122 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280d0d20 0x7f1d280ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.104+0000 7f1d2f045640 1 --2- 172.21.15.119:0/1108445122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d280cc990 0x7f1d280cbbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.104+0000 7f1d2f045640 1 -- 172.21.15.119:0/1108445122 >> 172.21.15.119:0/1108445122 conn(0x7f1d280993d0 msgr2=0x7f1d280d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:35.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.104+0000 7f1d2f045640 1 -- 172.21.15.119:0/1108445122 shutdown_connections 2023-08-11T15:50:35.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.104+0000 7f1d2f045640 1 -- 172.21.15.119:0/1108445122 wait complete. 2023-08-11T15:50:35.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.106+0000 7f1d2f045640 1 Processor -- start 2023-08-11T15:50:35.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.106+0000 7f1d2f045640 1 -- start start 2023-08-11T15:50:35.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.107+0000 7f1d2f045640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d280cc990 0x7f1d280c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.107+0000 7f1d2f045640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280ccd70 0x7f1d280c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.107+0000 7f1d2f045640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d280d0d20 0x7f1d280b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.107+0000 7f1d2f045640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280b2fc0 con 0x7f1d280d0d20 2023-08-11T15:50:35.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.107+0000 7f1d2f045640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280c6290 con 0x7f1d280cc990 2023-08-11T15:50:35.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.107+0000 7f1d2f045640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280c3e20 con 0x7f1d280ccd70 2023-08-11T15:50:35.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.107+0000 7f1d2d842640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280ccd70 0x7f1d280c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.107+0000 7f1d2e043640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d280cc990 0x7f1d280c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.107+0000 7f1d2d842640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280ccd70 0x7f1d280c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:32808/0 (socket says 172.21.15.119:32808) 2023-08-11T15:50:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.107+0000 7f1d2d842640 1 -- 172.21.15.119:0/2122293003 learned_addr learned my addr 172.21.15.119:0/2122293003 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d2e844640 1 --2- 172.21.15.119:0/2122293003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d280d0d20 0x7f1d280b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d2d842640 1 -- 172.21.15.119:0/2122293003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d280cc990 msgr2=0x7f1d280c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d2d842640 1 --2- 172.21.15.119:0/2122293003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d280cc990 0x7f1d280c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d2d842640 1 -- 172.21.15.119:0/2122293003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d280d0d20 msgr2=0x7f1d280b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d2d842640 1 --2- 172.21.15.119:0/2122293003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d280d0d20 0x7f1d280b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d2d842640 1 -- 172.21.15.119:0/2122293003 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d240093a0 con 0x7f1d280ccd70 2023-08-11T15:50:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d2e043640 1 --2- 172.21.15.119:0/2122293003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d280cc990 0x7f1d280c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:35.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d2d842640 1 --2- 172.21.15.119:0/2122293003 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280ccd70 0x7f1d280c5d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800e870 tx=0x7f1d1800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:35.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d1effd640 1 -- 172.21.15.119:0/2122293003 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d180094f0 con 0x7f1d280ccd70 2023-08-11T15:50:35.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d1effd640 1 -- 172.21.15.119:0/2122293003 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1800c8e0 con 0x7f1d280ccd70 2023-08-11T15:50:35.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d2f045640 1 -- 172.21.15.119:0/2122293003 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d280c43c0 con 0x7f1d280ccd70 2023-08-11T15:50:35.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.108+0000 7f1d1effd640 1 -- 172.21.15.119:0/2122293003 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800cc90 con 0x7f1d280ccd70 2023-08-11T15:50:35.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.109+0000 7f1d2f045640 1 -- 172.21.15.119:0/2122293003 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d280b5000 con 0x7f1d280ccd70 2023-08-11T15:50:35.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.109+0000 7f1d1effd640 1 -- 172.21.15.119:0/2122293003 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1d18005c90 con 0x7f1d280ccd70 2023-08-11T15:50:35.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.109+0000 7f1d1effd640 1 -- 172.21.15.119:0/2122293003 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f1d18013820 con 0x7f1d280ccd70 2023-08-11T15:50:35.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.110+0000 7f1d2f045640 1 -- 172.21.15.119:0/2122293003 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d280c6c30 con 0x7f1d280ccd70 2023-08-11T15:50:35.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.112+0000 7f1d1effd640 1 -- 172.21.15.119:0/2122293003 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f1d18021060 con 0x7f1d280ccd70 2023-08-11T15:50:35.118 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:35.116+0000 7ff44e432040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T15:50:35.140 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:35.139+0000 7f849d04f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T15:50:35.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.190+0000 7f1d2f045640 1 -- 172.21.15.119:0/2122293003 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2eed2b1c-7d9f-40fb-a154-cbf8b99dd988", "id": 1} v 0) v1 -- 0x7f1d280cb650 con 0x7f1d280ccd70 2023-08-11T15:50:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.196+0000 7f1d1effd640 1 -- 172.21.15.119:0/2122293003 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2eed2b1c-7d9f-40fb-a154-cbf8b99dd988", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1d18017070 con 0x7f1d280ccd70 2023-08-11T15:50:35.198 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-08-11T15:50:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.198+0000 7f1d2f045640 1 -- 172.21.15.119:0/2122293003 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280ccd70 msgr2=0x7f1d280c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.198+0000 7f1d2f045640 1 --2- 172.21.15.119:0/2122293003 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280ccd70 0x7f1d280c5d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800e870 tx=0x7f1d1800ed30 comp rx=0 tx=0).stop 2023-08-11T15:50:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.198+0000 7f1d2f045640 1 -- 172.21.15.119:0/2122293003 shutdown_connections 2023-08-11T15:50:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.198+0000 7f1d2f045640 1 --2- 172.21.15.119:0/2122293003 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d280ccd70 0x7f1d280c5d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.198+0000 7f1d2f045640 1 --2- 172.21.15.119:0/2122293003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d280d0d20 0x7f1d280b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.198+0000 7f1d2f045640 1 --2- 172.21.15.119:0/2122293003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d280cc990 0x7f1d280c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.198+0000 7f1d2f045640 1 -- 172.21.15.119:0/2122293003 >> 172.21.15.119:0/2122293003 conn(0x7f1d280993d0 msgr2=0x7f1d2806b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:35.201 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:35.198+0000 7f849d04f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T15:50:35.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.198+0000 7f1d2f045640 1 -- 172.21.15.119:0/2122293003 shutdown_connections 2023-08-11T15:50:35.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.198+0000 7f1d2f045640 1 -- 172.21.15.119:0/2122293003 wait complete. 2023-08-11T15:50:35.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 1b1cef5e-9225-4bf6-a05a-1ef4ce488827 2 2023-08-11T15:50:35.256 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:35.255+0000 7f849d04f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T15:50:35.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.401+0000 7f25ebafa640 1 Processor -- start 2023-08-11T15:50:35.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.401+0000 7f25ebafa640 1 -- start start 2023-08-11T15:50:35.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25ebafa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 0x7f25e40b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25ebafa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e40cda30 con 0x7f25e40cfb00 2023-08-11T15:50:35.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25ebafa640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e4161110 con 0x7f25e40b82f0 2023-08-11T15:50:35.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25ebafa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40ccc10 con 0x7f25e415c620 2023-08-11T15:50:35.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25e9095640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f25e40b82f0 0x7f25e40cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46114/0 (socket says 172.21.15.119:46114) 2023-08-11T15:50:35.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25e9095640 1 -- 172.21.15.119:0/2662811911 learned_addr learned my addr 172.21.15.119:0/2662811911 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-08-11T15:50:35.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25e9896640 1 --2- 172.21.15.119:0/2662811911 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 0x7f25e40b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25e8894640 1 -- 172.21.15.119:0/2662811911 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248907051 0 0) 0x7f25e4161110 con 0x7f25e40b82f0 2023-08-11T15:50:35.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25e8894640 1 -- 172.21.15.119:0/2662811911 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25d0003520 con 0x7f25e40b82f0 2023-08-11T15:50:35.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25e9896640 1 -- 172.21.15.119:0/2662811911 >> v1:172.21.15.119:6789/0 conn(0x7f25e40b82f0 legacy=0x7f25e40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.403+0000 7f25e9896640 1 -- 172.21.15.119:0/2662811911 >> v1:172.21.15.27:6789/0 conn(0x7f25e40cfb00 legacy=0x7f25e40d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:35.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.404+0000 7f25e9896640 1 -- 172.21.15.119:0/2662811911 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e40cce20 con 0x7f25e415c620 2023-08-11T15:50:35.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.405+0000 7f25e9896640 1 --2- 172.21.15.119:0/2662811911 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 0x7f25e40b7db0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f25e00097d0 tx=0x7f25e002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b51e7894f285e10b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:35.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.405+0000 7f25e8894640 1 -- 172.21.15.119:0/2662811911 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0031800 con 0x7f25e415c620 2023-08-11T15:50:35.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.405+0000 7f25e8894640 1 -- 172.21.15.119:0/2662811911 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e0031960 con 0x7f25e415c620 2023-08-11T15:50:35.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.405+0000 7f25e8894640 1 -- 172.21.15.119:0/2662811911 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0031cd0 con 0x7f25e415c620 2023-08-11T15:50:35.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.405+0000 7f25ebafa640 1 -- 172.21.15.119:0/2662811911 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 msgr2=0x7f25e40b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.405+0000 7f25ebafa640 1 --2- 172.21.15.119:0/2662811911 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 0x7f25e40b7db0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f25e00097d0 tx=0x7f25e002b8a0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.406+0000 7f25ebafa640 1 -- 172.21.15.119:0/2662811911 shutdown_connections 2023-08-11T15:50:35.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.406+0000 7f25ebafa640 1 --2- 172.21.15.119:0/2662811911 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 0x7f25e40b7db0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.406+0000 7f25ebafa640 1 -- 172.21.15.119:0/2662811911 >> 172.21.15.119:0/2662811911 conn(0x7f25e4071110 msgr2=0x7f25e40714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:35.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.406+0000 7f25ebafa640 1 -- 172.21.15.119:0/2662811911 shutdown_connections 2023-08-11T15:50:35.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.406+0000 7f25ebafa640 1 -- 172.21.15.119:0/2662811911 wait complete. 2023-08-11T15:50:35.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.407+0000 7f25ebafa640 1 Processor -- start 2023-08-11T15:50:35.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.407+0000 7f25ebafa640 1 -- start start 2023-08-11T15:50:35.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.408+0000 7f25ebafa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e40b82f0 0x7f25e40cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.408+0000 7f25ebafa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e40cfb00 0x7f25e40c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.408+0000 7f25ebafa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 0x7f25e40ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.408+0000 7f25ebafa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e415ca00 con 0x7f25e40b82f0 2023-08-11T15:50:35.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.408+0000 7f25ebafa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e415cce0 con 0x7f25e40cfb00 2023-08-11T15:50:35.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.408+0000 7f25ebafa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e415cfc0 con 0x7f25e415c620 2023-08-11T15:50:35.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9095640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e40cfb00 0x7f25e40c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9095640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e40cfb00 0x7f25e40c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44732/0 (socket says 172.21.15.119:44732) 2023-08-11T15:50:35.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9095640 1 -- 172.21.15.119:0/3579189292 learned_addr learned my addr 172.21.15.119:0/3579189292 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:35.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25ea097640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 0x7f25e40ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9896640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e40b82f0 0x7f25e40cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9095640 1 -- 172.21.15.119:0/3579189292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 msgr2=0x7f25e40ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9095640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 0x7f25e40ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9095640 1 -- 172.21.15.119:0/3579189292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e40b82f0 msgr2=0x7f25e40cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9095640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e40b82f0 0x7f25e40cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9095640 1 -- 172.21.15.119:0/3579189292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0009400 con 0x7f25e40cfb00 2023-08-11T15:50:35.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25ea097640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 0x7f25e40ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:50:35.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9896640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e40b82f0 0x7f25e40cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:35.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25e9095640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e40cfb00 0x7f25e40c9ef0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f25d400c4d0 tx=0x7f25d400c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:35.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25da7fc640 1 -- 172.21.15.119:0/3579189292 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4005ce0 con 0x7f25e40cfb00 2023-08-11T15:50:35.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25da7fc640 1 -- 172.21.15.119:0/3579189292 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d4005e40 con 0x7f25e40cfb00 2023-08-11T15:50:35.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.409+0000 7f25da7fc640 1 -- 172.21.15.119:0/3579189292 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4007270 con 0x7f25e40cfb00 2023-08-11T15:50:35.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.410+0000 7f25ebafa640 1 -- 172.21.15.119:0/3579189292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e415d2a0 con 0x7f25e40cfb00 2023-08-11T15:50:35.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.410+0000 7f25ebafa640 1 -- 172.21.15.119:0/3579189292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e415d860 con 0x7f25e40cfb00 2023-08-11T15:50:35.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.410+0000 7f25da7fc640 1 -- 172.21.15.119:0/3579189292 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f25d401c440 con 0x7f25e40cfb00 2023-08-11T15:50:35.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.410+0000 7f25da7fc640 1 -- 172.21.15.119:0/3579189292 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f25d401cb10 con 0x7f25e40cfb00 2023-08-11T15:50:35.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.411+0000 7f25ebafa640 1 -- 172.21.15.119:0/3579189292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e415da50 con 0x7f25e40cfb00 2023-08-11T15:50:35.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.413+0000 7f25da7fc640 1 -- 172.21.15.119:0/3579189292 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f25d4021060 con 0x7f25e40cfb00 2023-08-11T15:50:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.491+0000 7f25ebafa640 1 -- 172.21.15.119:0/3579189292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1b1cef5e-9225-4bf6-a05a-1ef4ce488827", "id": 2} v 0) v1 -- 0x7f25e4073990 con 0x7f25e40cfb00 2023-08-11T15:50:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.499+0000 7f25da7fc640 1 -- 172.21.15.119:0/3579189292 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1b1cef5e-9225-4bf6-a05a-1ef4ce488827", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f25d4017070 con 0x7f25e40cfb00 2023-08-11T15:50:35.501 INFO:teuthology.orchestra.run.smithi119.stdout:2 2023-08-11T15:50:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.500+0000 7f25ebafa640 1 -- 172.21.15.119:0/3579189292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e40cfb00 msgr2=0x7f25e40c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.500+0000 7f25ebafa640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e40cfb00 0x7f25e40c9ef0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f25d400c4d0 tx=0x7f25d400c990 comp rx=0 tx=0).stop 2023-08-11T15:50:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.501+0000 7f25ebafa640 1 -- 172.21.15.119:0/3579189292 shutdown_connections 2023-08-11T15:50:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.501+0000 7f25ebafa640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25e415c620 0x7f25e40ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.501+0000 7f25ebafa640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e40cfb00 0x7f25e40c9ef0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.501+0000 7f25ebafa640 1 --2- 172.21.15.119:0/3579189292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e40b82f0 0x7f25e40cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.501+0000 7f25ebafa640 1 -- 172.21.15.119:0/3579189292 >> 172.21.15.119:0/3579189292 conn(0x7f25e4071110 msgr2=0x7f25e40b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.501+0000 7f25ebafa640 1 -- 172.21.15.119:0/3579189292 shutdown_connections 2023-08-11T15:50:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.501+0000 7f25ebafa640 1 -- 172.21.15.119:0/3579189292 wait complete. 2023-08-11T15:50:35.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 79b66624-0fba-44ac-8abf-98da48240e5a 3 2023-08-11T15:50:35.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.754+0000 7ffb1a5cf640 1 Processor -- start 2023-08-11T15:50:35.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.754+0000 7ffb1a5cf640 1 -- start start 2023-08-11T15:50:35.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.755+0000 7ffb1a5cf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffb140ccc30 0x7ffb140cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.755+0000 7ffb1a5cf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb140b5460 0x7ffb140b5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.755+0000 7ffb1a5cf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb140b19c0 con 0x7ffb1415c550 2023-08-11T15:50:35.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.755+0000 7ffb1a5cf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb140d28e0 con 0x7ffb140ccc30 2023-08-11T15:50:35.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.755+0000 7ffb1a5cf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb140d1a30 con 0x7ffb140b5460 2023-08-11T15:50:35.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.756+0000 7ffb18b6c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb140b5460 0x7ffb140b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.756+0000 7ffb18b6c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb140b5460 0x7ffb140b5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44748/0 (socket says 172.21.15.119:44748) 2023-08-11T15:50:35.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.756+0000 7ffb18b6c640 1 -- 172.21.15.119:0/424117870 learned_addr learned my addr 172.21.15.119:0/424117870 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:35.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.756+0000 7ffb137fe640 1 --2- 172.21.15.119:0/424117870 >> v2:172.21.15.27:3300/0 conn(0x7ffb140ccc30 0x7ffb140cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.756+0000 7ffb18b6c640 1 -- 172.21.15.119:0/424117870 >> v2:172.21.15.27:3300/0 conn(0x7ffb140ccc30 msgr2=0x7ffb140cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:35.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.756+0000 7ffb18b6c640 1 --2- 172.21.15.119:0/424117870 >> v2:172.21.15.27:3300/0 conn(0x7ffb140ccc30 0x7ffb140cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.756+0000 7ffb18b6c640 1 -- 172.21.15.119:0/424117870 >> v1:172.21.15.27:6789/0 conn(0x7ffb1415c550 legacy=0x7ffb140d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:35.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.756+0000 7ffb18b6c640 1 -- 172.21.15.119:0/424117870 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140cd5d0 con 0x7ffb140b5460 2023-08-11T15:50:35.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.757+0000 7ffb18b6c640 1 --2- 172.21.15.119:0/424117870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140b5460 0x7ffb140b5910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffb140d3470 tx=0x7ffb0c02bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f728b107c45e13f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:35.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.757+0000 7ffb12ffd640 1 -- 172.21.15.119:0/424117870 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c02cb20 con 0x7ffb140b5460 2023-08-11T15:50:35.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.757+0000 7ffb12ffd640 1 -- 172.21.15.119:0/424117870 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c02cc80 con 0x7ffb140b5460 2023-08-11T15:50:35.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.757+0000 7ffb12ffd640 1 -- 172.21.15.119:0/424117870 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0343f0 con 0x7ffb140b5460 2023-08-11T15:50:35.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.758+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/424117870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140b5460 msgr2=0x7ffb140b5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.758+0000 7ffb1a5cf640 1 --2- 172.21.15.119:0/424117870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140b5460 0x7ffb140b5910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffb140d3470 tx=0x7ffb0c02bf80 comp rx=0 tx=0).stop 2023-08-11T15:50:35.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.758+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/424117870 shutdown_connections 2023-08-11T15:50:35.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.758+0000 7ffb1a5cf640 1 --2- 172.21.15.119:0/424117870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140b5460 0x7ffb140b5910 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.758+0000 7ffb1a5cf640 1 --2- 172.21.15.119:0/424117870 >> v2:172.21.15.27:3300/0 conn(0x7ffb140ccc30 0x7ffb140cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.758+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/424117870 >> 172.21.15.119:0/424117870 conn(0x7ffb14071320 msgr2=0x7ffb14071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:35.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.758+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/424117870 shutdown_connections 2023-08-11T15:50:35.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.758+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/424117870 wait complete. 2023-08-11T15:50:35.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.760+0000 7ffb1a5cf640 1 Processor -- start 2023-08-11T15:50:35.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.760+0000 7ffb1a5cf640 1 -- start start 2023-08-11T15:50:35.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb1a5cf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb140b5460 0x7ffb140cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb1a5cf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140ccc30 0x7ffb141f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb1a5cf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb1415c550 0x7ffb141f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:35.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb1a5cf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c8920 con 0x7ffb140b5460 2023-08-11T15:50:35.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb1a5cf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14152010 con 0x7ffb140ccc30 2023-08-11T15:50:35.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb1a5cf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c7da0 con 0x7ffb1415c550 2023-08-11T15:50:35.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb137fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140ccc30 0x7ffb141f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb18b6c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb1415c550 0x7ffb141f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb137fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140ccc30 0x7ffb141f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44760/0 (socket says 172.21.15.119:44760) 2023-08-11T15:50:35.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb137fe640 1 -- 172.21.15.119:0/1744347955 learned_addr learned my addr 172.21.15.119:0/1744347955 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:35.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb13fff640 1 --2- 172.21.15.119:0/1744347955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb140b5460 0x7ffb140cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:35.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb137fe640 1 -- 172.21.15.119:0/1744347955 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb1415c550 msgr2=0x7ffb141f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb137fe640 1 --2- 172.21.15.119:0/1744347955 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb1415c550 0x7ffb141f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb137fe640 1 -- 172.21.15.119:0/1744347955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb140b5460 msgr2=0x7ffb140cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb137fe640 1 --2- 172.21.15.119:0/1744347955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb140b5460 0x7ffb140cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb137fe640 1 -- 172.21.15.119:0/1744347955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c009b30 con 0x7ffb140ccc30 2023-08-11T15:50:35.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.761+0000 7ffb18b6c640 1 --2- 172.21.15.119:0/1744347955 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb1415c550 0x7ffb141f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:35.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.762+0000 7ffb137fe640 1 --2- 172.21.15.119:0/1744347955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140ccc30 0x7ffb141f4fe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800d800 tx=0x7ffb0800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:35.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.762+0000 7ffb10ff9640 1 -- 172.21.15.119:0/1744347955 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08018070 con 0x7ffb140ccc30 2023-08-11T15:50:35.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.762+0000 7ffb10ff9640 1 -- 172.21.15.119:0/1744347955 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb08004020 con 0x7ffb140ccc30 2023-08-11T15:50:35.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.762+0000 7ffb10ff9640 1 -- 172.21.15.119:0/1744347955 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080041b0 con 0x7ffb140ccc30 2023-08-11T15:50:35.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.762+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/1744347955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb140c8080 con 0x7ffb140ccc30 2023-08-11T15:50:35.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.762+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/1744347955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb140c8640 con 0x7ffb140ccc30 2023-08-11T15:50:35.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.762+0000 7ffb10ff9640 1 -- 172.21.15.119:0/1744347955 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffb0800b6c0 con 0x7ffb140ccc30 2023-08-11T15:50:35.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.762+0000 7ffb10ff9640 1 -- 172.21.15.119:0/1744347955 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7ffb0800b970 con 0x7ffb140ccc30 2023-08-11T15:50:35.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.763+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/1744347955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb140d2d50 con 0x7ffb140ccc30 2023-08-11T15:50:35.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.765+0000 7ffb10ff9640 1 -- 172.21.15.119:0/1744347955 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7ffb140d2d50 con 0x7ffb140ccc30 2023-08-11T15:50:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.849+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/1744347955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "79b66624-0fba-44ac-8abf-98da48240e5a", "id": 3} v 0) v1 -- 0x7ffb14073990 con 0x7ffb140ccc30 2023-08-11T15:50:35.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.856+0000 7ffb10ff9640 1 -- 172.21.15.119:0/1744347955 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "79b66624-0fba-44ac-8abf-98da48240e5a", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffb0800bb30 con 0x7ffb140ccc30 2023-08-11T15:50:35.858 INFO:teuthology.orchestra.run.smithi119.stdout:3 2023-08-11T15:50:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.857+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/1744347955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140ccc30 msgr2=0x7ffb141f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.857+0000 7ffb1a5cf640 1 --2- 172.21.15.119:0/1744347955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140ccc30 0x7ffb141f4fe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800d800 tx=0x7ffb0800dcc0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.857+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/1744347955 shutdown_connections 2023-08-11T15:50:35.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.857+0000 7ffb1a5cf640 1 --2- 172.21.15.119:0/1744347955 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb1415c550 0x7ffb141f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.857+0000 7ffb1a5cf640 1 --2- 172.21.15.119:0/1744347955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb140ccc30 0x7ffb141f4fe0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.857+0000 7ffb1a5cf640 1 --2- 172.21.15.119:0/1744347955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb140b5460 0x7ffb140cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:35.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.857+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/1744347955 >> 172.21.15.119:0/1744347955 conn(0x7ffb14071320 msgr2=0x7ffb140b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:35.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.857+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/1744347955 shutdown_connections 2023-08-11T15:50:35.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:35.857+0000 7ffb1a5cf640 1 -- 172.21.15.119:0/1744347955 wait complete. 2023-08-11T15:50:35.875 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new b7d2aeeb-47c1-4156-9720-82225a6b196b 4 2023-08-11T15:50:36.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.054+0000 7f78edffc640 1 Processor -- start 2023-08-11T15:50:36.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.054+0000 7f78edffc640 1 -- start start 2023-08-11T15:50:36.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.055+0000 7f78edffc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f78e815f8d0 0x7f78e80ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.055+0000 7f78edffc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e80b5390 0x7f78e80cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.055+0000 7f78edffc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e80b3df0 con 0x7f78e80d1340 2023-08-11T15:50:36.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.055+0000 7f78edffc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e80d2050 con 0x7f78e815f8d0 2023-08-11T15:50:36.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.055+0000 7f78edffc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e815fcb0 con 0x7f78e80b5390 2023-08-11T15:50:36.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.056+0000 7f78e7fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e80b5390 0x7f78e80cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.056+0000 7f78ed7fb640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f78e80d1340 0x7f78e80d17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46118/0 (socket says 172.21.15.119:46118) 2023-08-11T15:50:36.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.056+0000 7f78e7fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e80b5390 0x7f78e80cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:32850/0 (socket says 172.21.15.119:32850) 2023-08-11T15:50:36.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.056+0000 7f78ed7fb640 1 -- 172.21.15.119:0/933097873 learned_addr learned my addr 172.21.15.119:0/933097873 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-08-11T15:50:36.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.056+0000 7f78ecffa640 1 --2- 172.21.15.119:0/933097873 >> v2:172.21.15.27:3300/0 conn(0x7f78e815f8d0 0x7f78e80ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.056+0000 7f78e7fff640 1 -- 172.21.15.119:0/933097873 >> v2:172.21.15.27:3300/0 conn(0x7f78e815f8d0 msgr2=0x7f78e80ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:36.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.056+0000 7f78e7fff640 1 --2- 172.21.15.119:0/933097873 >> v2:172.21.15.27:3300/0 conn(0x7f78e815f8d0 0x7f78e80ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.056+0000 7f78e7fff640 1 -- 172.21.15.119:0/933097873 >> v1:172.21.15.119:6789/0 conn(0x7f78e80d1340 legacy=0x7f78e80d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.056+0000 7f78e7fff640 1 -- 172.21.15.119:0/933097873 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e80b4090 con 0x7f78e80b5390 2023-08-11T15:50:36.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.056+0000 7f78ecffa640 1 --2- 172.21.15.119:0/933097873 >> v2:172.21.15.27:3300/0 conn(0x7f78e815f8d0 0x7f78e80ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:36.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.057+0000 7f78e7fff640 1 --2- 172.21.15.119:0/933097873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e80b5390 0x7f78e80cd450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f78d40097a0 tx=0x7f78d402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e29afa48cee954a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:36.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78e77fe640 1 -- 172.21.15.119:0/933097873 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d4031830 con 0x7f78e80b5390 2023-08-11T15:50:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78e77fe640 1 -- 172.21.15.119:0/933097873 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d4031990 con 0x7f78e80b5390 2023-08-11T15:50:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78e77fe640 1 -- 172.21.15.119:0/933097873 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d4031d20 con 0x7f78e80b5390 2023-08-11T15:50:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78edffc640 1 -- 172.21.15.119:0/933097873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e80b5390 msgr2=0x7f78e80cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78edffc640 1 --2- 172.21.15.119:0/933097873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e80b5390 0x7f78e80cd450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f78d40097a0 tx=0x7f78d402b8d0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78edffc640 1 -- 172.21.15.119:0/933097873 shutdown_connections 2023-08-11T15:50:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78edffc640 1 --2- 172.21.15.119:0/933097873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e80b5390 0x7f78e80cd450 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78edffc640 1 --2- 172.21.15.119:0/933097873 >> v2:172.21.15.27:3300/0 conn(0x7f78e815f8d0 0x7f78e80ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78edffc640 1 -- 172.21.15.119:0/933097873 >> 172.21.15.119:0/933097873 conn(0x7f78e806a250 msgr2=0x7f78e806a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:36.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78edffc640 1 -- 172.21.15.119:0/933097873 shutdown_connections 2023-08-11T15:50:36.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.058+0000 7f78edffc640 1 -- 172.21.15.119:0/933097873 wait complete. 2023-08-11T15:50:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.060+0000 7f78edffc640 1 Processor -- start 2023-08-11T15:50:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.060+0000 7f78edffc640 1 -- start start 2023-08-11T15:50:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.061+0000 7f78edffc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78e80b5390 0x7f78e814ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.061+0000 7f78edffc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78e80d1340 0x7f78e8158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.061+0000 7f78edffc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e815f8d0 0x7f78e8158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.061+0000 7f78edffc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8151f30 con 0x7f78e80d1340 2023-08-11T15:50:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.061+0000 7f78edffc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80cb870 con 0x7f78e80b5390 2023-08-11T15:50:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.061+0000 7f78edffc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80cbb50 con 0x7f78e815f8d0 2023-08-11T15:50:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.061+0000 7f78ecffa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78e80b5390 0x7f78e814ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.061+0000 7f78ecffa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78e80b5390 0x7f78e814ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44776/0 (socket says 172.21.15.119:44776) 2023-08-11T15:50:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.061+0000 7f78ecffa640 1 -- 172.21.15.119:0/3031413644 learned_addr learned my addr 172.21.15.119:0/3031413644 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.061+0000 7f78ed7fb640 1 --2- 172.21.15.119:0/3031413644 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e815f8d0 0x7f78e8158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78e7fff640 1 --2- 172.21.15.119:0/3031413644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78e80d1340 0x7f78e8158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78ecffa640 1 -- 172.21.15.119:0/3031413644 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e815f8d0 msgr2=0x7f78e8158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78ecffa640 1 --2- 172.21.15.119:0/3031413644 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e815f8d0 0x7f78e8158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78ecffa640 1 -- 172.21.15.119:0/3031413644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78e80d1340 msgr2=0x7f78e8158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78ecffa640 1 --2- 172.21.15.119:0/3031413644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78e80d1340 0x7f78e8158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78ecffa640 1 -- 172.21.15.119:0/3031413644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d4009400 con 0x7f78e80b5390 2023-08-11T15:50:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78ed7fb640 1 --2- 172.21.15.119:0/3031413644 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e815f8d0 0x7f78e8158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78ecffa640 1 --2- 172.21.15.119:0/3031413644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78e80b5390 0x7f78e814ed50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00c7f0 tx=0x7f78dc00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78e57fa640 1 -- 172.21.15.119:0/3031413644 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc007820 con 0x7f78e80b5390 2023-08-11T15:50:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78e57fa640 1 -- 172.21.15.119:0/3031413644 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc007980 con 0x7f78e80b5390 2023-08-11T15:50:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.062+0000 7f78e57fa640 1 -- 172.21.15.119:0/3031413644 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc007cf0 con 0x7f78e80b5390 2023-08-11T15:50:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.063+0000 7f78edffc640 1 -- 172.21.15.119:0/3031413644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e8158200 con 0x7f78e80b5390 2023-08-11T15:50:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.063+0000 7f78edffc640 1 -- 172.21.15.119:0/3031413644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e8157e20 con 0x7f78e80b5390 2023-08-11T15:50:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.063+0000 7f78e57fa640 1 -- 172.21.15.119:0/3031413644 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f78dc01a660 con 0x7f78e80b5390 2023-08-11T15:50:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.063+0000 7f78e57fa640 1 -- 172.21.15.119:0/3031413644 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f78dc0115d0 con 0x7f78e80b5390 2023-08-11T15:50:36.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.064+0000 7f78edffc640 1 -- 172.21.15.119:0/3031413644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e8158010 con 0x7f78e80b5390 2023-08-11T15:50:36.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.068+0000 7f78e57fa640 1 -- 172.21.15.119:0/3031413644 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f78dc039b20 con 0x7f78e80b5390 2023-08-11T15:50:36.119 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:36.117+0000 7ff44e432040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T15:50:36.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.146+0000 7f78edffc640 1 -- 172.21.15.119:0/3031413644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b7d2aeeb-47c1-4156-9720-82225a6b196b", "id": 4} v 0) v1 -- 0x7f78e804e420 con 0x7f78e80b5390 2023-08-11T15:50:36.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.152+0000 7f78e57fa640 1 -- 172.21.15.119:0/3031413644 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b7d2aeeb-47c1-4156-9720-82225a6b196b", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f78dc028070 con 0x7f78e80b5390 2023-08-11T15:50:36.154 INFO:teuthology.orchestra.run.smithi119.stdout:4 2023-08-11T15:50:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.154+0000 7f78edffc640 1 -- 172.21.15.119:0/3031413644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78e80b5390 msgr2=0x7f78e814ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.154+0000 7f78edffc640 1 --2- 172.21.15.119:0/3031413644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78e80b5390 0x7f78e814ed50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00c7f0 tx=0x7f78dc00ccb0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.154+0000 7f78edffc640 1 -- 172.21.15.119:0/3031413644 shutdown_connections 2023-08-11T15:50:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.154+0000 7f78edffc640 1 --2- 172.21.15.119:0/3031413644 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78e815f8d0 0x7f78e8158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.154+0000 7f78edffc640 1 --2- 172.21.15.119:0/3031413644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78e80d1340 0x7f78e8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.154+0000 7f78edffc640 1 --2- 172.21.15.119:0/3031413644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78e80b5390 0x7f78e814ed50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.154+0000 7f78edffc640 1 -- 172.21.15.119:0/3031413644 >> 172.21.15.119:0/3031413644 conn(0x7f78e806a250 msgr2=0x7f78e80d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.154+0000 7f78edffc640 1 -- 172.21.15.119:0/3031413644 shutdown_connections 2023-08-11T15:50:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.154+0000 7f78edffc640 1 -- 172.21.15.119:0/3031413644 wait complete. 2023-08-11T15:50:36.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 4623837e-0e1a-42c4-8394-ec0fa2ba4a98 5 2023-08-11T15:50:36.187 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:36.184+0000 7ff44e432040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T15:50:36.221 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:36.220+0000 7f849d04f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T15:50:36.244 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:36.241+0000 7ff44e432040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T15:50:36.290 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:36.289+0000 7f849d04f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T15:50:36.349 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:36.348+0000 7f849d04f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T15:50:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.354+0000 7f23a759e640 1 Processor -- start 2023-08-11T15:50:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.354+0000 7f23a759e640 1 -- start start 2023-08-11T15:50:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.355+0000 7f23a759e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a80cd4f0 0x7f23a80cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23a80cd8d0 0x7f23a8070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a80cd110 con 0x7f23a806bd50 2023-08-11T15:50:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a80cae80 con 0x7f23a80cd8d0 2023-08-11T15:50:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a759e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80ca270 con 0x7f23a80cd4f0 2023-08-11T15:50:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a659c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a80cd4f0 0x7f23a80cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a659c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a80cd4f0 0x7f23a80cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:32872/0 (socket says 172.21.15.119:32872) 2023-08-11T15:50:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a659c640 1 -- 172.21.15.119:0/2401262844 learned_addr learned my addr 172.21.15.119:0/2401262844 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a5d9b640 1 --2- 172.21.15.119:0/2401262844 >> v2:172.21.15.27:3300/0 conn(0x7f23a80cd8d0 0x7f23a8070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a659c640 1 -- 172.21.15.119:0/2401262844 >> v2:172.21.15.27:3300/0 conn(0x7f23a80cd8d0 msgr2=0x7f23a8070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a659c640 1 --2- 172.21.15.119:0/2401262844 >> v2:172.21.15.27:3300/0 conn(0x7f23a80cd8d0 0x7f23a8070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a659c640 1 -- 172.21.15.119:0/2401262844 >> v1:172.21.15.27:6789/0 conn(0x7f23a806bd50 legacy=0x7f23a806c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.356+0000 7f23a659c640 1 -- 172.21.15.119:0/2401262844 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a80ca550 con 0x7f23a80cd4f0 2023-08-11T15:50:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.357+0000 7f23a659c640 1 --2- 172.21.15.119:0/2401262844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a80cd4f0 0x7f23a80cb310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2398009870 tx=0x7f239802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef7344dd33c6d55 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.358+0000 7f23a559a640 1 -- 172.21.15.119:0/2401262844 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398031640 con 0x7f23a80cd4f0 2023-08-11T15:50:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.358+0000 7f23a559a640 1 -- 172.21.15.119:0/2401262844 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23980317a0 con 0x7f23a80cd4f0 2023-08-11T15:50:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.358+0000 7f23a559a640 1 -- 172.21.15.119:0/2401262844 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398031b30 con 0x7f23a80cd4f0 2023-08-11T15:50:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.359+0000 7f23a759e640 1 -- 172.21.15.119:0/2401262844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a80cd4f0 msgr2=0x7f23a80cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.359+0000 7f23a759e640 1 --2- 172.21.15.119:0/2401262844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a80cd4f0 0x7f23a80cb310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2398009870 tx=0x7f239802b8f0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.359+0000 7f23a759e640 1 -- 172.21.15.119:0/2401262844 shutdown_connections 2023-08-11T15:50:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.359+0000 7f23a759e640 1 --2- 172.21.15.119:0/2401262844 >> v2:172.21.15.27:3300/0 conn(0x7f23a80cd8d0 0x7f23a8070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.359+0000 7f23a759e640 1 --2- 172.21.15.119:0/2401262844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a80cd4f0 0x7f23a80cb310 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.359+0000 7f23a759e640 1 -- 172.21.15.119:0/2401262844 >> 172.21.15.119:0/2401262844 conn(0x7f23a80ceea0 msgr2=0x7f23a80cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.359+0000 7f23a759e640 1 -- 172.21.15.119:0/2401262844 shutdown_connections 2023-08-11T15:50:36.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.359+0000 7f23a759e640 1 -- 172.21.15.119:0/2401262844 wait complete. 2023-08-11T15:50:36.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.361+0000 7f23a759e640 1 Processor -- start 2023-08-11T15:50:36.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.362+0000 7f23a759e640 1 -- start start 2023-08-11T15:50:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.363+0000 7f23a759e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a806bd50 0x7f23a80c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.363+0000 7f23a759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23a80cd4f0 0x7f23a80c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.363+0000 7f23a759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a80cd8d0 0x7f23a814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.363+0000 7f23a759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a81e29d0 con 0x7f23a80cd4f0 2023-08-11T15:50:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.363+0000 7f23a759e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a8159f10 con 0x7f23a80cd8d0 2023-08-11T15:50:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.363+0000 7f23a759e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a815a1f0 con 0x7f23a806bd50 2023-08-11T15:50:36.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a659c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a806bd50 0x7f23a80c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a659c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a806bd50 0x7f23a80c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:32878/0 (socket says 172.21.15.119:32878) 2023-08-11T15:50:36.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a659c640 1 -- 172.21.15.119:0/1393214045 learned_addr learned my addr 172.21.15.119:0/1393214045 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:36.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a6d9d640 1 --2- 172.21.15.119:0/1393214045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a80cd8d0 0x7f23a814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a5d9b640 1 --2- 172.21.15.119:0/1393214045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23a80cd4f0 0x7f23a80c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a659c640 1 -- 172.21.15.119:0/1393214045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a80cd8d0 msgr2=0x7f23a814f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a659c640 1 --2- 172.21.15.119:0/1393214045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a80cd8d0 0x7f23a814f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a659c640 1 -- 172.21.15.119:0/1393214045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23a80cd4f0 msgr2=0x7f23a80c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a659c640 1 --2- 172.21.15.119:0/1393214045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23a80cd4f0 0x7f23a80c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a659c640 1 -- 172.21.15.119:0/1393214045 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980094d0 con 0x7f23a806bd50 2023-08-11T15:50:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a5d9b640 1 --2- 172.21.15.119:0/1393214045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23a80cd4f0 0x7f23a80c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.364+0000 7f23a659c640 1 --2- 172.21.15.119:0/1393214045 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a806bd50 0x7f23a80c5f70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2398009870 tx=0x7f23980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.365+0000 7f2396ffd640 1 -- 172.21.15.119:0/1393214045 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239803ac60 con 0x7f23a806bd50 2023-08-11T15:50:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.365+0000 7f2396ffd640 1 -- 172.21.15.119:0/1393214045 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239803ade0 con 0x7f23a806bd50 2023-08-11T15:50:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.365+0000 7f2396ffd640 1 -- 172.21.15.119:0/1393214045 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398044630 con 0x7f23a806bd50 2023-08-11T15:50:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.365+0000 7f23a759e640 1 -- 172.21.15.119:0/1393214045 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a81e2660 con 0x7f23a806bd50 2023-08-11T15:50:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.365+0000 7f23a759e640 1 -- 172.21.15.119:0/1393214045 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a80c8be0 con 0x7f23a806bd50 2023-08-11T15:50:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.365+0000 7f2396ffd640 1 -- 172.21.15.119:0/1393214045 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f239802f050 con 0x7f23a806bd50 2023-08-11T15:50:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.365+0000 7f2396ffd640 1 -- 172.21.15.119:0/1393214045 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f2398044a90 con 0x7f23a806bd50 2023-08-11T15:50:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.366+0000 7f23a759e640 1 -- 172.21.15.119:0/1393214045 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a81e27f0 con 0x7f23a806bd50 2023-08-11T15:50:36.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.368+0000 7f2396ffd640 1 -- 172.21.15.119:0/1393214045 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f2398035020 con 0x7f23a806bd50 2023-08-11T15:50:36.378 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:36.376+0000 7ff44e432040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T15:50:36.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.468+0000 7f23a759e640 1 -- 172.21.15.119:0/1393214045 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4623837e-0e1a-42c4-8394-ec0fa2ba4a98", "id": 5} v 0) v1 -- 0x7f23a804e420 con 0x7f23a806bd50 2023-08-11T15:50:36.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.477+0000 7f2396ffd640 1 -- 172.21.15.119:0/1393214045 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4623837e-0e1a-42c4-8394-ec0fa2ba4a98", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f239802d040 con 0x7f23a806bd50 2023-08-11T15:50:36.478 INFO:teuthology.orchestra.run.smithi119.stdout:5 2023-08-11T15:50:36.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.478+0000 7f23a759e640 1 -- 172.21.15.119:0/1393214045 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a806bd50 msgr2=0x7f23a80c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.478+0000 7f23a759e640 1 --2- 172.21.15.119:0/1393214045 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a806bd50 0x7f23a80c5f70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2398009870 tx=0x7f23980046c0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.478+0000 7f23a759e640 1 -- 172.21.15.119:0/1393214045 shutdown_connections 2023-08-11T15:50:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.478+0000 7f23a759e640 1 --2- 172.21.15.119:0/1393214045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23a80cd8d0 0x7f23a814f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.478+0000 7f23a759e640 1 --2- 172.21.15.119:0/1393214045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23a80cd4f0 0x7f23a80c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.478+0000 7f23a759e640 1 --2- 172.21.15.119:0/1393214045 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23a806bd50 0x7f23a80c5f70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.478+0000 7f23a759e640 1 -- 172.21.15.119:0/1393214045 >> 172.21.15.119:0/1393214045 conn(0x7f23a80ceea0 msgr2=0x7f23a80cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.478+0000 7f23a759e640 1 -- 172.21.15.119:0/1393214045 shutdown_connections 2023-08-11T15:50:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.478+0000 7f23a759e640 1 -- 172.21.15.119:0/1393214045 wait complete. 2023-08-11T15:50:36.497 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new f0604e90-c352-436c-92de-01c1d363207b 6 2023-08-11T15:50:36.514 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:36.513+0000 7f849d04f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T15:50:36.548 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:36.545+0000 7ff44e432040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:50:36.664 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:36.662+0000 7ff44e432040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:50:36.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.681+0000 7f53bfbe8640 1 Processor -- start 2023-08-11T15:50:36.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.682+0000 7f53bfbe8640 1 -- start start 2023-08-11T15:50:36.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.683+0000 7f53bfbe8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f53b80b55c0 0x7f53b80cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.683+0000 7f53bfbe8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53b80cf930 0x7f53b80d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.683+0000 7f53bfbe8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b80cfd80 con 0x7f53b815c620 2023-08-11T15:50:36.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.683+0000 7f53bfbe8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b80ccc10 con 0x7f53b80b55c0 2023-08-11T15:50:36.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.683+0000 7f53bfbe8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b80cda30 con 0x7f53b80cf930 2023-08-11T15:50:36.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.683+0000 7f53bd984640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f53b815c620 0x7f53b80b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46134/0 (socket says 172.21.15.119:46134) 2023-08-11T15:50:36.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.683+0000 7f53bd984640 1 -- 172.21.15.119:0/1873472643 learned_addr learned my addr 172.21.15.119:0/1873472643 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-08-11T15:50:36.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.683+0000 7f53be185640 1 --2- 172.21.15.119:0/1873472643 >> v2:172.21.15.119:3300/0 conn(0x7f53b80cf930 0x7f53b80d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.684+0000 7f53bd183640 1 --2- 172.21.15.119:0/1873472643 >> v2:172.21.15.27:3300/0 conn(0x7f53b80b55c0 0x7f53b80cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.684+0000 7f53bc982640 1 -- 172.21.15.119:0/1873472643 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873621607 0 0) 0x7f53b4001110 con 0x7f53b815c620 2023-08-11T15:50:36.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.684+0000 7f53bc982640 1 -- 172.21.15.119:0/1873472643 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a4003520 con 0x7f53b815c620 2023-08-11T15:50:36.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.684+0000 7f53be185640 1 -- 172.21.15.119:0/1873472643 >> v2:172.21.15.27:3300/0 conn(0x7f53b80b55c0 msgr2=0x7f53b80cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:36.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.684+0000 7f53be185640 1 --2- 172.21.15.119:0/1873472643 >> v2:172.21.15.27:3300/0 conn(0x7f53b80b55c0 0x7f53b80cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.684+0000 7f53be185640 1 -- 172.21.15.119:0/1873472643 >> v1:172.21.15.119:6789/0 conn(0x7f53b815c620 legacy=0x7f53b80b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.684+0000 7f53be185640 1 -- 172.21.15.119:0/1873472643 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b8161110 con 0x7f53b80cf930 2023-08-11T15:50:36.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.684+0000 7f53bd183640 1 --2- 172.21.15.119:0/1873472643 >> v2:172.21.15.27:3300/0 conn(0x7f53b80b55c0 0x7f53b80cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:50:36.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.685+0000 7f53be185640 1 --2- 172.21.15.119:0/1873472643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 0x7f53b80d0b00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f53b80c9640 tx=0x7f53b002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82fa47bb6f0fbef2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:36.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.685+0000 7f53bc982640 1 -- 172.21.15.119:0/1873472643 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b002cc80 con 0x7f53b80cf930 2023-08-11T15:50:36.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.685+0000 7f53bc982640 1 -- 172.21.15.119:0/1873472643 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b002cde0 con 0x7f53b80cf930 2023-08-11T15:50:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.685+0000 7f53bc982640 1 -- 172.21.15.119:0/1873472643 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0034640 con 0x7f53b80cf930 2023-08-11T15:50:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.686+0000 7f53bfbe8640 1 -- 172.21.15.119:0/1873472643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 msgr2=0x7f53b80d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.686+0000 7f53bfbe8640 1 --2- 172.21.15.119:0/1873472643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 0x7f53b80d0b00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f53b80c9640 tx=0x7f53b002bed0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.686+0000 7f53bfbe8640 1 -- 172.21.15.119:0/1873472643 shutdown_connections 2023-08-11T15:50:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.686+0000 7f53bfbe8640 1 --2- 172.21.15.119:0/1873472643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 0x7f53b80d0b00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.686+0000 7f53bfbe8640 1 --2- 172.21.15.119:0/1873472643 >> v2:172.21.15.27:3300/0 conn(0x7f53b80b55c0 0x7f53b80cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.686+0000 7f53bfbe8640 1 -- 172.21.15.119:0/1873472643 >> 172.21.15.119:0/1873472643 conn(0x7f53b8071110 msgr2=0x7f53b80714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.686+0000 7f53bfbe8640 1 -- 172.21.15.119:0/1873472643 shutdown_connections 2023-08-11T15:50:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.686+0000 7f53bfbe8640 1 -- 172.21.15.119:0/1873472643 wait complete. 2023-08-11T15:50:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.688+0000 7f53bfbe8640 1 Processor -- start 2023-08-11T15:50:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.688+0000 7f53bfbe8640 1 -- start start 2023-08-11T15:50:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53bfbe8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53b80b55c0 0x7f53b814e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53bfbe8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 0x7f53b815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53bfbe8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53b815c620 0x7f53b815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53bfbe8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b81f1dc0 con 0x7f53b80b55c0 2023-08-11T15:50:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53bfbe8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80cbfc0 con 0x7f53b80cf930 2023-08-11T15:50:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53bfbe8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80a9b90 con 0x7f53b815c620 2023-08-11T15:50:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53bd183640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 0x7f53b815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53be185640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53b815c620 0x7f53b815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53bd183640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 0x7f53b815a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44816/0 (socket says 172.21.15.119:44816) 2023-08-11T15:50:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53bd183640 1 -- 172.21.15.119:0/3277886449 learned_addr learned my addr 172.21.15.119:0/3277886449 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.689+0000 7f53bd984640 1 --2- 172.21.15.119:0/3277886449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53b80b55c0 0x7f53b814e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53bd183640 1 -- 172.21.15.119:0/3277886449 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53b815c620 msgr2=0x7f53b815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53bd183640 1 --2- 172.21.15.119:0/3277886449 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53b815c620 0x7f53b815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53bd183640 1 -- 172.21.15.119:0/3277886449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53b80b55c0 msgr2=0x7f53b814e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53bd183640 1 --2- 172.21.15.119:0/3277886449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53b80b55c0 0x7f53b814e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53bd183640 1 -- 172.21.15.119:0/3277886449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b0009a70 con 0x7f53b80cf930 2023-08-11T15:50:36.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53be185640 1 --2- 172.21.15.119:0/3277886449 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53b815c620 0x7f53b815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:36.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53bd183640 1 --2- 172.21.15.119:0/3277886449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 0x7f53b815a3f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f53a800d7d0 tx=0x7f53a800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:36.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53ae7fc640 1 -- 172.21.15.119:0/3277886449 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8004030 con 0x7f53b80cf930 2023-08-11T15:50:36.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53ae7fc640 1 -- 172.21.15.119:0/3277886449 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a8004190 con 0x7f53b80cf930 2023-08-11T15:50:36.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53ae7fc640 1 -- 172.21.15.119:0/3277886449 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8004520 con 0x7f53b80cf930 2023-08-11T15:50:36.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53bfbe8640 1 -- 172.21.15.119:0/3277886449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b80a9e70 con 0x7f53b80cf930 2023-08-11T15:50:36.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.690+0000 7f53bfbe8640 1 -- 172.21.15.119:0/3277886449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b81f2000 con 0x7f53b80cf930 2023-08-11T15:50:36.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.691+0000 7f53ae7fc640 1 -- 172.21.15.119:0/3277886449 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f53a80099a0 con 0x7f53b80cf930 2023-08-11T15:50:36.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.691+0000 7f53ae7fc640 1 -- 172.21.15.119:0/3277886449 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f53a8003d90 con 0x7f53b80cf930 2023-08-11T15:50:36.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.692+0000 7f53bfbe8640 1 -- 172.21.15.119:0/3277886449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b80aa390 con 0x7f53b80cf930 2023-08-11T15:50:36.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.694+0000 7f53ae7fc640 1 -- 172.21.15.119:0/3277886449 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f53b80aa390 con 0x7f53b80cf930 2023-08-11T15:50:36.699 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:36.697+0000 7f849d04f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:50:36.717 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:36.715+0000 7ff44e432040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T15:50:36.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.772+0000 7f53bfbe8640 1 -- 172.21.15.119:0/3277886449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f0604e90-c352-436c-92de-01c1d363207b", "id": 6} v 0) v1 -- 0x7f53b81f2190 con 0x7f53b80cf930 2023-08-11T15:50:36.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.779+0000 7f53ae7fc640 1 -- 172.21.15.119:0/3277886449 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f0604e90-c352-436c-92de-01c1d363207b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f53a8009d20 con 0x7f53b80cf930 2023-08-11T15:50:36.780 INFO:teuthology.orchestra.run.smithi119.stdout:6 2023-08-11T15:50:36.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.780+0000 7f53bfbe8640 1 -- 172.21.15.119:0/3277886449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 msgr2=0x7f53b815a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:36.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.780+0000 7f53bfbe8640 1 --2- 172.21.15.119:0/3277886449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 0x7f53b815a3f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f53a800d7d0 tx=0x7f53a800dc90 comp rx=0 tx=0).stop 2023-08-11T15:50:36.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.780+0000 7f53bfbe8640 1 -- 172.21.15.119:0/3277886449 shutdown_connections 2023-08-11T15:50:36.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.780+0000 7f53bfbe8640 1 --2- 172.21.15.119:0/3277886449 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53b815c620 0x7f53b815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.780+0000 7f53bfbe8640 1 --2- 172.21.15.119:0/3277886449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53b80cf930 0x7f53b815a3f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.780+0000 7f53bfbe8640 1 --2- 172.21.15.119:0/3277886449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53b80b55c0 0x7f53b814e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:36.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.780+0000 7f53bfbe8640 1 -- 172.21.15.119:0/3277886449 >> 172.21.15.119:0/3277886449 conn(0x7f53b8071110 msgr2=0x7f53b815edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:36.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.780+0000 7f53bfbe8640 1 -- 172.21.15.119:0/3277886449 shutdown_connections 2023-08-11T15:50:36.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.780+0000 7f53bfbe8640 1 -- 172.21.15.119:0/3277886449 wait complete. 2023-08-11T15:50:36.796 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1 7 2023-08-11T15:50:36.817 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:36.816+0000 7f849d04f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:50:36.869 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:36.868+0000 7f849d04f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T15:50:36.894 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:36.891+0000 7ff44e432040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T15:50:36.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.996+0000 7fb7e9396640 1 Processor -- start 2023-08-11T15:50:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.996+0000 7fb7e9396640 1 -- start start 2023-08-11T15:50:37.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.997+0000 7fb7e9396640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7e415c810 0x7fb7e406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:37.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.997+0000 7fb7e9396640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e406aa40 0x7fb7e40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:37.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.997+0000 7fb7e9396640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e406c870 con 0x7fb7e40cd1d0 2023-08-11T15:50:37.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.997+0000 7fb7e9396640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e406d4f0 con 0x7fb7e415c810 2023-08-11T15:50:37.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.997+0000 7fb7e9396640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e40cd660 con 0x7fb7e406aa40 2023-08-11T15:50:37.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.998+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e406aa40 0x7fb7e40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.998+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e406aa40 0x7fb7e40ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:32896/0 (socket says 172.21.15.119:32896) 2023-08-11T15:50:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.998+0000 7fb7dbfff640 1 -- 172.21.15.119:0/1742629829 learned_addr learned my addr 172.21.15.119:0/1742629829 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.998+0000 7fb7e37fe640 1 --1- 172.21.15.119:0/1742629829 >> v1:172.21.15.119:6789/0 conn(0x7fb7e40cd1d0 0x7fb7e4069800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46148/0 (socket says 172.21.15.119:46148) 2023-08-11T15:50:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.998+0000 7fb7e2ffd640 1 --2- 172.21.15.119:0/1742629829 >> v2:172.21.15.27:3300/0 conn(0x7fb7e415c810 0x7fb7e406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:37.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.998+0000 7fb7dbfff640 1 -- 172.21.15.119:0/1742629829 >> v2:172.21.15.27:3300/0 conn(0x7fb7e415c810 msgr2=0x7fb7e406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:37.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.998+0000 7fb7dbfff640 1 --2- 172.21.15.119:0/1742629829 >> v2:172.21.15.27:3300/0 conn(0x7fb7e415c810 0x7fb7e406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.998+0000 7fb7dbfff640 1 -- 172.21.15.119:0/1742629829 >> v1:172.21.15.119:6789/0 conn(0x7fb7e40cd1d0 legacy=0x7fb7e4069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.998+0000 7fb7dbfff640 1 -- 172.21.15.119:0/1742629829 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e406cb50 con 0x7fb7e406aa40 2023-08-11T15:50:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.998+0000 7fb7e27fc640 1 -- 172.21.15.119:0/1742629829 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036184766 0 0) 0x7fb7e406c870 con 0x7fb7e40cd1d0 2023-08-11T15:50:37.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:36.999+0000 7fb7dbfff640 1 --2- 172.21.15.119:0/1742629829 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e406aa40 0x7fb7e40ccc90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009870 tx=0x7fb7cc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8291a3efc4ff4334 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:37.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e27fc640 1 -- 172.21.15.119:0/1742629829 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc030840 con 0x7fb7e406aa40 2023-08-11T15:50:37.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e27fc640 1 -- 172.21.15.119:0/1742629829 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7cc0309a0 con 0x7fb7e406aa40 2023-08-11T15:50:37.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e27fc640 1 -- 172.21.15.119:0/1742629829 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc030d30 con 0x7fb7e406aa40 2023-08-11T15:50:37.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e9396640 1 -- 172.21.15.119:0/1742629829 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e406aa40 msgr2=0x7fb7e40ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:37.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e9396640 1 --2- 172.21.15.119:0/1742629829 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e406aa40 0x7fb7e40ccc90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009870 tx=0x7fb7cc02b980 comp rx=0 tx=0).stop 2023-08-11T15:50:37.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e9396640 1 -- 172.21.15.119:0/1742629829 shutdown_connections 2023-08-11T15:50:37.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e9396640 1 --2- 172.21.15.119:0/1742629829 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e406aa40 0x7fb7e40ccc90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e9396640 1 --2- 172.21.15.119:0/1742629829 >> v2:172.21.15.27:3300/0 conn(0x7fb7e415c810 0x7fb7e406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e9396640 1 -- 172.21.15.119:0/1742629829 >> 172.21.15.119:0/1742629829 conn(0x7fb7e40ce980 msgr2=0x7fb7e40ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:37.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e9396640 1 -- 172.21.15.119:0/1742629829 shutdown_connections 2023-08-11T15:50:37.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.000+0000 7fb7e9396640 1 -- 172.21.15.119:0/1742629829 wait complete. 2023-08-11T15:50:37.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.002+0000 7fb7e9396640 1 Processor -- start 2023-08-11T15:50:37.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.002+0000 7fb7e9396640 1 -- start start 2023-08-11T15:50:37.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.003+0000 7fb7e9396640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e406aa40 0x7fb7e4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:37.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.003+0000 7fb7e9396640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e40cd1d0 0x7fb7e414b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:37.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.003+0000 7fb7e9396640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e415c810 0x7fb7e41f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:37.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.003+0000 7fb7e9396640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e4150f70 con 0x7fb7e406aa40 2023-08-11T15:50:37.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.003+0000 7fb7e9396640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e41503a0 con 0x7fb7e40cd1d0 2023-08-11T15:50:37.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.003+0000 7fb7e9396640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e4150640 con 0x7fb7e415c810 2023-08-11T15:50:37.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.003+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e40cd1d0 0x7fb7e414b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:37.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.003+0000 7fb7e37fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e415c810 0x7fb7e41f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:37.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.003+0000 7fb7e37fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e415c810 0x7fb7e41f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:32910/0 (socket says 172.21.15.119:32910) 2023-08-11T15:50:37.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.003+0000 7fb7e37fe640 1 -- 172.21.15.119:0/3785662303 learned_addr learned my addr 172.21.15.119:0/3785662303 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-08-11T15:50:37.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7e2ffd640 1 --2- 172.21.15.119:0/3785662303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e406aa40 0x7fb7e4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:37.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7e37fe640 1 -- 172.21.15.119:0/3785662303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e40cd1d0 msgr2=0x7fb7e414b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:37.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7e37fe640 1 --2- 172.21.15.119:0/3785662303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e40cd1d0 0x7fb7e414b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7e37fe640 1 -- 172.21.15.119:0/3785662303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e406aa40 msgr2=0x7fb7e4070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:37.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7e37fe640 1 --2- 172.21.15.119:0/3785662303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e406aa40 0x7fb7e4070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7e37fe640 1 -- 172.21.15.119:0/3785662303 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0094d0 con 0x7fb7e415c810 2023-08-11T15:50:37.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7e37fe640 1 --2- 172.21.15.119:0/3785662303 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e415c810 0x7fb7e41f30e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00c7e0 tx=0x7fb7dc00cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:37.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7db7fe640 1 -- 172.21.15.119:0/3785662303 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00a640 con 0x7fb7e415c810 2023-08-11T15:50:37.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7db7fe640 1 -- 172.21.15.119:0/3785662303 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7dc00a7c0 con 0x7fb7e415c810 2023-08-11T15:50:37.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7db7fe640 1 -- 172.21.15.119:0/3785662303 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00ab70 con 0x7fb7e415c810 2023-08-11T15:50:37.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.004+0000 7fb7e9396640 1 -- 172.21.15.119:0/3785662303 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e4150920 con 0x7fb7e415c810 2023-08-11T15:50:37.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.005+0000 7fb7e9396640 1 -- 172.21.15.119:0/3785662303 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e4070aa0 con 0x7fb7e415c810 2023-08-11T15:50:37.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.005+0000 7fb7db7fe640 1 -- 172.21.15.119:0/3785662303 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb7dc0070d0 con 0x7fb7e415c810 2023-08-11T15:50:37.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.005+0000 7fb7db7fe640 1 -- 172.21.15.119:0/3785662303 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fb7dc0124f0 con 0x7fb7e415c810 2023-08-11T15:50:37.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.006+0000 7fb7e9396640 1 -- 172.21.15.119:0/3785662303 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e4073990 con 0x7fb7e415c810 2023-08-11T15:50:37.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.008+0000 7fb7db7fe640 1 -- 172.21.15.119:0/3785662303 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb7dc018070 con 0x7fb7e415c810 2023-08-11T15:50:37.043 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:37.042+0000 7f849d04f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T15:50:37.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.088+0000 7fb7e9396640 1 -- 172.21.15.119:0/3785662303 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1", "id": 7} v 0) v1 -- 0x7fb7e40b36e0 con 0x7fb7e415c810 2023-08-11T15:50:37.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.095+0000 7fb7db7fe640 1 -- 172.21.15.119:0/3785662303 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb7dc007450 con 0x7fb7e415c810 2023-08-11T15:50:37.096 INFO:teuthology.orchestra.run.smithi119.stdout:7 2023-08-11T15:50:37.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.096+0000 7fb7e9396640 1 -- 172.21.15.119:0/3785662303 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e415c810 msgr2=0x7fb7e41f30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:37.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.096+0000 7fb7e9396640 1 --2- 172.21.15.119:0/3785662303 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e415c810 0x7fb7e41f30e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00c7e0 tx=0x7fb7dc00cca0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.096+0000 7fb7e9396640 1 -- 172.21.15.119:0/3785662303 shutdown_connections 2023-08-11T15:50:37.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.096+0000 7fb7e9396640 1 --2- 172.21.15.119:0/3785662303 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7e415c810 0x7fb7e41f30e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.096+0000 7fb7e9396640 1 --2- 172.21.15.119:0/3785662303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e40cd1d0 0x7fb7e414b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.096+0000 7fb7e9396640 1 --2- 172.21.15.119:0/3785662303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e406aa40 0x7fb7e4070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.096+0000 7fb7e9396640 1 -- 172.21.15.119:0/3785662303 >> 172.21.15.119:0/3785662303 conn(0x7fb7e40ce980 msgr2=0x7fb7e40b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:37.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.096+0000 7fb7e9396640 1 -- 172.21.15.119:0/3785662303 shutdown_connections 2023-08-11T15:50:37.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:50:37.096+0000 7fb7e9396640 1 -- 172.21.15.119:0/3785662303 wait complete. 2023-08-11T15:50:37.112 INFO:tasks.ceph.osd.0:Restarting daemon 2023-08-11T15:50:37.112 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-08-11T15:50:37.121 INFO:tasks.ceph.osd.0:Started 2023-08-11T15:50:37.122 INFO:tasks.ceph.osd.1:Restarting daemon 2023-08-11T15:50:37.122 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-08-11T15:50:37.131 INFO:tasks.ceph.osd.1:Started 2023-08-11T15:50:37.132 INFO:tasks.ceph.osd.2:Restarting daemon 2023-08-11T15:50:37.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-08-11T15:50:37.138 INFO:tasks.ceph.osd.2:Started 2023-08-11T15:50:37.139 INFO:tasks.ceph.osd.3:Restarting daemon 2023-08-11T15:50:37.139 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-08-11T15:50:37.147 INFO:tasks.ceph.osd.3:Started 2023-08-11T15:50:37.148 INFO:tasks.ceph.osd.4:Restarting daemon 2023-08-11T15:50:37.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-08-11T15:50:37.158 INFO:tasks.ceph.osd.4:Started 2023-08-11T15:50:37.158 INFO:tasks.ceph.osd.5:Restarting daemon 2023-08-11T15:50:37.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-08-11T15:50:37.166 INFO:tasks.ceph.osd.5:Started 2023-08-11T15:50:37.166 INFO:tasks.ceph.osd.6:Restarting daemon 2023-08-11T15:50:37.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-08-11T15:50:37.173 INFO:tasks.ceph.osd.6:Started 2023-08-11T15:50:37.173 INFO:tasks.ceph.osd.7:Restarting daemon 2023-08-11T15:50:37.174 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-08-11T15:50:37.181 INFO:tasks.ceph.osd.7:Started 2023-08-11T15:50:37.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:50:37.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.420+0000 7fb201199640 1 Processor -- start 2023-08-11T15:50:37.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.421+0000 7fb201199640 1 -- start start 2023-08-11T15:50:37.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.422+0000 7fb201199640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1fc06ae40 0x7fb1fc0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.422+0000 7fb201199640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1fc06dba0 0x7fb1fc06e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.422+0000 7fb201199640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc06b220 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.422+0000 7fb201199640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1fc06e570 con 0x7fb1fc06ae40 2023-08-11T15:50:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.422+0000 7fb201199640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc06f590 con 0x7fb1fc06dba0 2023-08-11T15:50:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.422+0000 7fb1fad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1fc06ae40 0x7fb1fc0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:37.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.422+0000 7fb1fad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1fc06ae40 0x7fb1fc0cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38910/0 (socket says 172.21.15.27:38910) 2023-08-11T15:50:37.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.422+0000 7fb1fad9d640 1 -- 172.21.15.27:0/2970475608 learned_addr learned my addr 172.21.15.27:0/2970475608 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:37.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.422+0000 7fb1fb59e640 1 --2- 172.21.15.27:0/2970475608 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1fc06dba0 0x7fb1fc06e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:37.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.423+0000 7fb1fad9d640 1 -- 172.21.15.27:0/2970475608 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1fc06dba0 msgr2=0x7fb1fc06e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:37.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.423+0000 7fb1fad9d640 1 --2- 172.21.15.27:0/2970475608 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1fc06dba0 0x7fb1fc06e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.423+0000 7fb1fad9d640 1 -- 172.21.15.27:0/2970475608 >> v1:172.21.15.27:6789/0 conn(0x7fb1fc06f0b0 legacy=0x7fb1fc15cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:37.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.423+0000 7fb1fad9d640 1 -- 172.21.15.27:0/2970475608 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc06f870 con 0x7fb1fc06ae40 2023-08-11T15:50:37.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.424+0000 7fb1fad9d640 1 --2- 172.21.15.27:0/2970475608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06ae40 0x7fb1fc0cf900 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009870 tx=0x7fb1ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213a4829edc6bde0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:37.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.424+0000 7fb1f9d9b640 1 -- 172.21.15.27:0/2970475608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec031640 con 0x7fb1fc06ae40 2023-08-11T15:50:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.424+0000 7fb1f9d9b640 1 -- 172.21.15.27:0/2970475608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec0317a0 con 0x7fb1fc06ae40 2023-08-11T15:50:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.424+0000 7fb1f9d9b640 1 -- 172.21.15.27:0/2970475608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec031b30 con 0x7fb1fc06ae40 2023-08-11T15:50:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.425+0000 7fb201199640 1 -- 172.21.15.27:0/2970475608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06ae40 msgr2=0x7fb1fc0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.425+0000 7fb201199640 1 --2- 172.21.15.27:0/2970475608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06ae40 0x7fb1fc0cf900 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009870 tx=0x7fb1ec02b8f0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.425+0000 7fb201199640 1 -- 172.21.15.27:0/2970475608 shutdown_connections 2023-08-11T15:50:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.425+0000 7fb201199640 1 --2- 172.21.15.27:0/2970475608 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1fc06dba0 0x7fb1fc06e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.425+0000 7fb201199640 1 --2- 172.21.15.27:0/2970475608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06ae40 0x7fb1fc0cf900 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.425+0000 7fb201199640 1 -- 172.21.15.27:0/2970475608 >> 172.21.15.27:0/2970475608 conn(0x7fb1fc0d0220 msgr2=0x7fb1fc0d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.425+0000 7fb201199640 1 -- 172.21.15.27:0/2970475608 shutdown_connections 2023-08-11T15:50:37.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.425+0000 7fb201199640 1 -- 172.21.15.27:0/2970475608 wait complete. 2023-08-11T15:50:37.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.427+0000 7fb201199640 1 Processor -- start 2023-08-11T15:50:37.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.427+0000 7fb201199640 1 -- start start 2023-08-11T15:50:37.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb201199640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc06ae40 0x7fb1fc0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:37.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb201199640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1fc06dba0 0x7fb1fc15f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:37.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb201199640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06f0b0 0x7fb1fc15f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:37.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb201199640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0c6ea0 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb201199640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc14ef60 con 0x7fb1fc06ae40 2023-08-11T15:50:37.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb201199640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc14f240 con 0x7fb1fc06dba0 2023-08-11T15:50:37.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb1fb59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06f0b0 0x7fb1fc15f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:37.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb1fb59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06f0b0 0x7fb1fc15f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38912/0 (socket says 172.21.15.27:38912) 2023-08-11T15:50:37.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb1fb59e640 1 -- 172.21.15.27:0/3747408475 learned_addr learned my addr 172.21.15.27:0/3747408475 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:37.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb1fa59c640 1 --2- 172.21.15.27:0/3747408475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1fc06dba0 0x7fb1fc15f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:37.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.428+0000 7fb1fad9d640 1 --2- 172.21.15.27:0/3747408475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc06ae40 0x7fb1fc0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:37.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.429+0000 7fb1fb59e640 1 -- 172.21.15.27:0/3747408475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1fc06dba0 msgr2=0x7fb1fc15f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:37.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.429+0000 7fb1fb59e640 1 --2- 172.21.15.27:0/3747408475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1fc06dba0 0x7fb1fc15f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.429+0000 7fb1fb59e640 1 -- 172.21.15.27:0/3747408475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc06ae40 msgr2=0x7fb1fc0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:37.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.429+0000 7fb1fb59e640 1 --2- 172.21.15.27:0/3747408475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc06ae40 0x7fb1fc0c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.429+0000 7fb1fb59e640 1 -- 172.21.15.27:0/3747408475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec0094d0 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.429+0000 7fb1fb59e640 1 --2- 172.21.15.27:0/3747408475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06f0b0 0x7fb1fc15f7d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400efc0 tx=0x7fb1f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.429+0000 7fb1e37fe640 1 -- 172.21.15.27:0/3747408475 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4014480 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.429+0000 7fb1e37fe640 1 -- 172.21.15.27:0/3747408475 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f400f040 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.429+0000 7fb1e37fe640 1 -- 172.21.15.27:0/3747408475 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4014820 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.429+0000 7fb201199640 1 -- 172.21.15.27:0/3747408475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1fc14f520 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.430+0000 7fb201199640 1 -- 172.21.15.27:0/3747408475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc14fae0 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.430+0000 7fb1e37fe640 1 -- 172.21.15.27:0/3747408475 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb1f4014c80 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.430+0000 7fb1e37fe640 1 -- 172.21.15.27:0/3747408475 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb1f4013ab0 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.431+0000 7fb201199640 1 -- 172.21.15.27:0/3747408475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc073990 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.433+0000 7fb1e37fe640 1 -- 172.21.15.27:0/3747408475 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb1f4018020 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.454 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:37.452+0000 7ff44e432040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T15:50:37.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.500+0000 7fb201199640 1 -- 172.21.15.27:0/3747408475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1fc0d1050 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.501+0000 7fb1e37fe640 1 -- 172.21.15.27:0/3747408475 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb1f4013d90 con 0x7fb1fc06f0b0 2023-08-11T15:50:37.505 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:50:37.505 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:50:37.090267+0000","last_up_change":"0.000000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:50:37.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.503+0000 7fb201199640 1 -- 172.21.15.27:0/3747408475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06f0b0 msgr2=0x7fb1fc15f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:37.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.503+0000 7fb201199640 1 --2- 172.21.15.27:0/3747408475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06f0b0 0x7fb1fc15f7d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400efc0 tx=0x7fb1f400c490 comp rx=0 tx=0).stop 2023-08-11T15:50:37.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.503+0000 7fb201199640 1 -- 172.21.15.27:0/3747408475 shutdown_connections 2023-08-11T15:50:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.503+0000 7fb201199640 1 --2- 172.21.15.27:0/3747408475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1fc06dba0 0x7fb1fc15f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.503+0000 7fb201199640 1 --2- 172.21.15.27:0/3747408475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1fc06f0b0 0x7fb1fc15f7d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.503+0000 7fb201199640 1 --2- 172.21.15.27:0/3747408475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc06ae40 0x7fb1fc0c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.503+0000 7fb201199640 1 -- 172.21.15.27:0/3747408475 >> 172.21.15.27:0/3747408475 conn(0x7fb1fc0d0220 msgr2=0x7fb1fc0cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.503+0000 7fb201199640 1 -- 172.21.15.27:0/3747408475 shutdown_connections 2023-08-11T15:50:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:37.503+0000 7fb201199640 1 -- 172.21.15.27:0/3747408475 wait complete. 2023-08-11T15:50:37.522 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-08-11T15:50:37.522 INFO:tasks.ceph:Waiting for OSDs to come up 2023-08-11T15:50:37.538 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:37.536+0000 7f849d04f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T15:50:37.689 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:37.686+0000 7ff44e432040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T15:50:37.754 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:37.751+0000 7ff44e432040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T15:50:37.756 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:37.755+0000 7f849d04f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T15:50:37.810 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:37.808+0000 7ff44e432040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T15:50:37.816 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:37.815+0000 7f849d04f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T15:50:37.846 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-08-11T15:50:37.844+0000 7f281249a500 -1 Falling back to public interface 2023-08-11T15:50:37.856 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-08-11T15:50:37.855+0000 7fd3bb40c500 -1 Falling back to public interface 2023-08-11T15:50:37.866 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-08-11T15:50:37.864+0000 7fbabbca9500 -1 Falling back to public interface 2023-08-11T15:50:37.869 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-08-11T15:50:37.868+0000 7f68e841e500 -1 Falling back to public interface 2023-08-11T15:50:37.870 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:37.868+0000 7f849d04f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T15:50:37.875 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-08-11T15:50:37.874+0000 7f73fed24500 -1 Falling back to public interface 2023-08-11T15:50:37.876 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:37.873+0000 7ff44e432040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T15:50:37.878 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-08-11T15:50:37.875+0000 7fcdb140c500 -1 Falling back to public interface 2023-08-11T15:50:37.883 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-08-11T15:50:37.881+0000 7f1732c0d500 -1 Falling back to public interface 2023-08-11T15:50:37.910 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-08-11T15:50:37.909+0000 7fc46dc69500 -1 Falling back to public interface 2023-08-11T15:50:37.933 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:37.932+0000 7f849d04f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T15:50:38.197 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:38.195+0000 7ff44e432040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T15:50:38.234 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:38.233+0000 7f849d04f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T15:50:38.262 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:38.259+0000 7ff44e432040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T15:50:38.294 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:38.292+0000 7f849d04f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T15:50:38.327 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-11T15:50:38.337 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:38.330+0000 7ff44e432040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T15:50:38.349 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:38.348+0000 7f849d04f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T15:50:38.499 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.497+0000 7f93fa8d2640 1 Processor -- start 2023-08-11T15:50:38.500 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.497+0000 7f93fa8d2640 1 -- start start 2023-08-11T15:50:38.501 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.498+0000 7f93fa8d2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93f40cf970 0x7f93f40afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:38.501 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.498+0000 7f93fa8d2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f40cdb10 con 0x7f93f40af200 2023-08-11T15:50:38.501 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.498+0000 7f93fa8d2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f40cddf0 con 0x7f93f40d0530 2023-08-11T15:50:38.501 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.498+0000 7f93fa8d2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f40ce0d0 con 0x7f93f40cf970 2023-08-11T15:50:38.502 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.499+0000 7f93f37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f93f40af200 0x7f93f40af6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36916/0 (socket says 172.21.15.27:36916) 2023-08-11T15:50:38.502 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.499+0000 7f93f37fe640 1 -- 172.21.15.27:0/3268949198 learned_addr learned my addr 172.21.15.27:0/3268949198 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:50:38.502 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.499+0000 7f93f8e6f640 1 --2- 172.21.15.27:0/3268949198 >> v2:172.21.15.119:3300/0 conn(0x7f93f40cf970 0x7f93f40afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:38.502 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.499+0000 7f93f2ffd640 1 -- 172.21.15.27:0/3268949198 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863365303 0 0) 0x7f93f40cdb10 con 0x7f93f40af200 2023-08-11T15:50:38.502 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.499+0000 7f93f2ffd640 1 -- 172.21.15.27:0/3268949198 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93dc003520 con 0x7f93f40af200 2023-08-11T15:50:38.503 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.499+0000 7f93f8e6f640 1 -- 172.21.15.27:0/3268949198 >> v1:172.21.15.119:6789/0 conn(0x7f93f40d0530 legacy=0x7f93f40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:38.503 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.499+0000 7f93f8e6f640 1 -- 172.21.15.27:0/3268949198 >> v1:172.21.15.27:6789/0 conn(0x7f93f40af200 legacy=0x7f93f40af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:38.503 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.499+0000 7f93f8e6f640 1 -- 172.21.15.27:0/3268949198 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f40ce3b0 con 0x7f93f40cf970 2023-08-11T15:50:38.504 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.501+0000 7f93f8e6f640 1 --2- 172.21.15.27:0/3268949198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f40cf970 0x7f93f40afdc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009d90 tx=0x7f93ec02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616868ec95e1896 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:38.504 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.501+0000 7f93f2ffd640 1 -- 172.21.15.27:0/3268949198 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec02cc60 con 0x7f93f40cf970 2023-08-11T15:50:38.504 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.501+0000 7f93f2ffd640 1 -- 172.21.15.27:0/3268949198 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ec02cdc0 con 0x7f93f40cf970 2023-08-11T15:50:38.504 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.501+0000 7f93f2ffd640 1 -- 172.21.15.27:0/3268949198 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec034660 con 0x7f93f40cf970 2023-08-11T15:50:38.504 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.501+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3268949198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f40cf970 msgr2=0x7f93f40afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:38.505 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.501+0000 7f93fa8d2640 1 --2- 172.21.15.27:0/3268949198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f40cf970 0x7f93f40afdc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009d90 tx=0x7f93ec02beb0 comp rx=0 tx=0).stop 2023-08-11T15:50:38.505 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.502+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3268949198 shutdown_connections 2023-08-11T15:50:38.505 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.502+0000 7f93fa8d2640 1 --2- 172.21.15.27:0/3268949198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f40cf970 0x7f93f40afdc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:38.505 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.502+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3268949198 >> 172.21.15.27:0/3268949198 conn(0x7f93f40d23c0 msgr2=0x7f93f40d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:38.505 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.502+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3268949198 shutdown_connections 2023-08-11T15:50:38.505 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.502+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3268949198 wait complete. 2023-08-11T15:50:38.506 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.503+0000 7f93fa8d2640 1 Processor -- start 2023-08-11T15:50:38.506 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.503+0000 7f93fa8d2640 1 -- start start 2023-08-11T15:50:38.507 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.504+0000 7f93fa8d2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f40af200 0x7f93f41f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:38.507 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.504+0000 7f93fa8d2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f40cf970 0x7f93f40b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:38.507 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.504+0000 7f93fa8d2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93f40d0530 0x7f93f415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:38.507 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.504+0000 7f93fa8d2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f415ff90 con 0x7f93f40cf970 2023-08-11T15:50:38.507 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.504+0000 7f93fa8d2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40b71a0 con 0x7f93f40af200 2023-08-11T15:50:38.507 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.504+0000 7f93fa8d2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40b7e80 con 0x7f93f40d0530 2023-08-11T15:50:38.508 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f40cf970 0x7f93f40b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:38.509 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f40cf970 0x7f93f40b7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38924/0 (socket says 172.21.15.27:38924) 2023-08-11T15:50:38.509 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f37fe640 1 -- 172.21.15.27:0/3564620218 learned_addr learned my addr 172.21.15.27:0/3564620218 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:38.509 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f3fff640 1 --2- 172.21.15.27:0/3564620218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f40af200 0x7f93f41f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:38.509 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f8e6f640 1 --2- 172.21.15.27:0/3564620218 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93f40d0530 0x7f93f415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:38.509 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f37fe640 1 -- 172.21.15.27:0/3564620218 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93f40d0530 msgr2=0x7f93f415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:38.509 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f37fe640 1 --2- 172.21.15.27:0/3564620218 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93f40d0530 0x7f93f415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:38.510 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f37fe640 1 -- 172.21.15.27:0/3564620218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f40af200 msgr2=0x7f93f41f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:38.510 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f37fe640 1 --2- 172.21.15.27:0/3564620218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f40af200 0x7f93f41f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:38.510 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f37fe640 1 -- 172.21.15.27:0/3564620218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec009a70 con 0x7f93f40cf970 2023-08-11T15:50:38.510 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.505+0000 7f93f37fe640 1 --2- 172.21.15.27:0/3564620218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f40cf970 0x7f93f40b7940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f93e800c500 tx=0x7f93e800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:38.510 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.506+0000 7f93f0ff9640 1 -- 172.21.15.27:0/3564620218 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8005ce0 con 0x7f93f40cf970 2023-08-11T15:50:38.510 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.506+0000 7f93f0ff9640 1 -- 172.21.15.27:0/3564620218 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e8005e40 con 0x7f93f40cf970 2023-08-11T15:50:38.510 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.506+0000 7f93f0ff9640 1 -- 172.21.15.27:0/3564620218 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8007330 con 0x7f93f40cf970 2023-08-11T15:50:38.510 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.506+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3564620218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f40b6590 con 0x7f93f40cf970 2023-08-11T15:50:38.511 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.506+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3564620218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f40b6b50 con 0x7f93f40cf970 2023-08-11T15:50:38.511 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.506+0000 7f93f0ff9640 1 -- 172.21.15.27:0/3564620218 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f93e801c440 con 0x7f93f40cf970 2023-08-11T15:50:38.511 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.506+0000 7f93f0ff9640 1 -- 172.21.15.27:0/3564620218 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f93e8012b70 con 0x7f93f40cf970 2023-08-11T15:50:38.511 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.507+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3564620218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f404e420 con 0x7f93f40cf970 2023-08-11T15:50:38.512 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:38.509+0000 7ff44e432040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T15:50:38.513 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.511+0000 7f93f0ff9640 1 -- 172.21.15.27:0/3564620218 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f93e8017070 con 0x7f93f40cf970 2023-08-11T15:50:38.515 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:38.514+0000 7f849d04f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T15:50:38.564 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:38.563+0000 7f849d04f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T15:50:38.565 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:38.563+0000 7ff44e432040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T15:50:38.580 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.578+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3564620218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93f40b72e0 con 0x7f93f40cf970 2023-08-11T15:50:38.582 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.579+0000 7f93f0ff9640 1 -- 172.21.15.27:0/3564620218 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f93e8012500 con 0x7f93f40cf970 2023-08-11T15:50:38.582 INFO:teuthology.misc.health.smithi027.stdout: 2023-08-11T15:50:38.582 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":10,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:50:37.090267+0000","last_up_change":"0.000000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:50:38.583 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.581+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3564620218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f40cf970 msgr2=0x7f93f40b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:38.583 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.581+0000 7f93fa8d2640 1 --2- 172.21.15.27:0/3564620218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f40cf970 0x7f93f40b7940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f93e800c500 tx=0x7f93e800c9c0 comp rx=0 tx=0).stop 2023-08-11T15:50:38.584 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.581+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3564620218 shutdown_connections 2023-08-11T15:50:38.584 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.581+0000 7f93fa8d2640 1 --2- 172.21.15.27:0/3564620218 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93f40d0530 0x7f93f415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:38.584 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.581+0000 7f93fa8d2640 1 --2- 172.21.15.27:0/3564620218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f40cf970 0x7f93f40b7940 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:38.584 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.581+0000 7f93fa8d2640 1 --2- 172.21.15.27:0/3564620218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f40af200 0x7f93f41f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:38.585 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.581+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3564620218 >> 172.21.15.27:0/3564620218 conn(0x7f93f40d23c0 msgr2=0x7f93f40af690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:38.585 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.581+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3564620218 shutdown_connections 2023-08-11T15:50:38.585 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:38.581+0000 7f93fa8d2640 1 -- 172.21.15.27:0/3564620218 wait complete. 2023-08-11T15:50:38.597 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-08-11T15:50:38.624 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:38.622+0000 7f849d04f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T15:50:38.632 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:38.629+0000 7ff44e432040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T15:50:38.677 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:38.675+0000 7f849d04f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T15:50:38.689 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-08-11T15:50:38.687+0000 7ff44e432040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T15:50:39.729 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:39.727+0000 7f843e4cd640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-08-11T15:50:39.731 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-08-11T15:50:39.730+0000 7f843e4cd640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-08-11T15:50:41.215 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-08-11T15:50:41.214+0000 7f68e841e500 -1 osd.4 0 log_to_monitors true 2023-08-11T15:50:41.259 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-08-11T15:50:41.256+0000 7f1732c0d500 -1 osd.2 0 log_to_monitors true 2023-08-11T15:50:41.409 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-08-11T15:50:41.407+0000 7f281249a500 -1 osd.0 0 log_to_monitors true 2023-08-11T15:50:41.426 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-08-11T15:50:41.424+0000 7fd3bb40c500 -1 osd.6 0 log_to_monitors true 2023-08-11T15:50:41.464 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-08-11T15:50:41.463+0000 7f73fed24500 -1 osd.5 0 log_to_monitors true 2023-08-11T15:50:41.489 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-08-11T15:50:41.488+0000 7fc46dc69500 -1 osd.7 0 log_to_monitors true 2023-08-11T15:50:41.559 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-08-11T15:50:41.557+0000 7fcdb140c500 -1 osd.3 0 log_to_monitors true 2023-08-11T15:50:41.736 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-08-11T15:50:41.734+0000 7fbabbca9500 -1 osd.1 0 log_to_monitors true 2023-08-11T15:50:45.098 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-08-11T15:50:45.095+0000 7f172e68b640 -1 osd.2 0 waiting for initial osdmap 2023-08-11T15:50:45.117 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-08-11T15:50:45.115+0000 7f17264ac640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:50:45.133 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-08-11T15:50:45.128+0000 7f280e426640 -1 osd.0 0 waiting for initial osdmap 2023-08-11T15:50:45.148 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-08-11T15:50:45.147+0000 7f2806247640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:50:45.162 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-08-11T15:50:45.161+0000 7fcdad5cc640 -1 osd.3 0 waiting for initial osdmap 2023-08-11T15:50:45.179 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-08-11T15:50:45.177+0000 7fcda4bd7640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:50:45.269 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-08-11T15:50:45.266+0000 7fc469bf5640 -1 osd.7 0 waiting for initial osdmap 2023-08-11T15:50:45.278 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-08-11T15:50:45.276+0000 7f68e3e0a640 -1 osd.4 0 waiting for initial osdmap 2023-08-11T15:50:45.283 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-08-11T15:50:45.282+0000 7fc461a16640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:50:45.296 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-08-11T15:50:45.295+0000 7f68dbc2b640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:50:45.306 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-08-11T15:50:45.305+0000 7f73facb0640 -1 osd.5 0 waiting for initial osdmap 2023-08-11T15:50:45.319 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-08-11T15:50:45.318+0000 7f73f2ad1640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:50:45.326 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-08-11T15:50:45.325+0000 7fd3b6c95640 -1 osd.6 0 waiting for initial osdmap 2023-08-11T15:50:45.345 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-08-11T15:50:45.344+0000 7fd3aeab6640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:50:45.402 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-11T15:50:45.411 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-08-11T15:50:45.409+0000 7fbab7c35640 -1 osd.1 0 waiting for initial osdmap 2023-08-11T15:50:45.425 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-08-11T15:50:45.424+0000 7fbaafa56640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:50:45.561 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.560+0000 7fae01167640 1 Processor -- start 2023-08-11T15:50:45.561 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.560+0000 7fae01167640 1 -- start start 2023-08-11T15:50:45.563 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.561+0000 7fae01167640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc15e820 0x7fadfc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.563 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.561+0000 7fae01167640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fadfc06d1b0 0x7fadfc0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.563 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.561+0000 7fae01167640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc15eca0 con 0x7fadfc154870 2023-08-11T15:50:45.563 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.561+0000 7fae01167640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadfc15e400 con 0x7fadfc06d1b0 2023-08-11T15:50:45.563 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.561+0000 7fae01167640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc154c50 con 0x7fadfc15e820 2023-08-11T15:50:45.564 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.562+0000 7fadf3fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc15e820 0x7fadfc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.564 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.562+0000 7fae00966640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fadfc06d1b0 0x7fadfc0c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.564 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.562+0000 7fae00966640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fadfc06d1b0 0x7fadfc0c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:49162/0 (socket says 172.21.15.27:49162) 2023-08-11T15:50:45.565 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.562+0000 7fadf3fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc15e820 0x7fadfc06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:35624/0 (socket says 172.21.15.27:35624) 2023-08-11T15:50:45.565 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.562+0000 7fae00966640 1 -- 172.21.15.27:0/2756163460 learned_addr learned my addr 172.21.15.27:0/2756163460 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:45.565 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.563+0000 7fadf3fff640 1 -- 172.21.15.27:0/2756163460 >> v2:172.21.15.119:3300/0 conn(0x7fadfc06d1b0 msgr2=0x7fadfc0c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:45.565 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.563+0000 7fadf3fff640 1 --2- 172.21.15.27:0/2756163460 >> v2:172.21.15.119:3300/0 conn(0x7fadfc06d1b0 0x7fadfc0c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.565 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.563+0000 7fadf3fff640 1 -- 172.21.15.27:0/2756163460 >> v1:172.21.15.119:6789/0 conn(0x7fadfc154870 legacy=0x7fadfc15b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.565 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.563+0000 7fadf3fff640 1 -- 172.21.15.27:0/2756163460 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc06d600 con 0x7fadfc15e820 2023-08-11T15:50:45.565 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.563+0000 7fadfb7fe640 1 -- 172.21.15.27:0/2756163460 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106914827 0 0) 0x7fadfc15eca0 con 0x7fadfc154870 2023-08-11T15:50:45.566 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.563+0000 7fae00966640 1 --2- 172.21.15.27:0/2756163460 >> v2:172.21.15.119:3300/0 conn(0x7fadfc06d1b0 0x7fadfc0c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:45.567 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.564+0000 7fadf3fff640 1 --2- 172.21.15.27:0/2756163460 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc15e820 0x7fadfc06cc70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fade40097a0 tx=0x7fade402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc9f0bbe8939fc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:45.567 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fadfb7fe640 1 -- 172.21.15.27:0/2756163460 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4031780 con 0x7fadfc15e820 2023-08-11T15:50:45.567 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fadfb7fe640 1 -- 172.21.15.27:0/2756163460 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade40318e0 con 0x7fadfc15e820 2023-08-11T15:50:45.567 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fadfb7fe640 1 -- 172.21.15.27:0/2756163460 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4031c70 con 0x7fadfc15e820 2023-08-11T15:50:45.567 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fae01167640 1 -- 172.21.15.27:0/2756163460 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc15e820 msgr2=0x7fadfc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.567 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fae01167640 1 --2- 172.21.15.27:0/2756163460 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc15e820 0x7fadfc06cc70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fade40097a0 tx=0x7fade402b820 comp rx=0 tx=0).stop 2023-08-11T15:50:45.568 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fae01167640 1 -- 172.21.15.27:0/2756163460 shutdown_connections 2023-08-11T15:50:45.568 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fae01167640 1 --2- 172.21.15.27:0/2756163460 >> v2:172.21.15.119:3300/0 conn(0x7fadfc06d1b0 0x7fadfc0c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.568 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fae01167640 1 --2- 172.21.15.27:0/2756163460 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc15e820 0x7fadfc06cc70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.568 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fae01167640 1 -- 172.21.15.27:0/2756163460 >> 172.21.15.27:0/2756163460 conn(0x7fadfc0cdd20 msgr2=0x7fadfc0ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:45.568 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fae01167640 1 -- 172.21.15.27:0/2756163460 shutdown_connections 2023-08-11T15:50:45.568 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.565+0000 7fae01167640 1 -- 172.21.15.27:0/2756163460 wait complete. 2023-08-11T15:50:45.569 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.567+0000 7fae01167640 1 Processor -- start 2023-08-11T15:50:45.569 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.567+0000 7fae01167640 1 -- start start 2023-08-11T15:50:45.570 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.568+0000 7fae01167640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc06d1b0 0x7fadfc1502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.570 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.568+0000 7fae01167640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc154870 0x7fadfc0b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.570 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.568+0000 7fae01167640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc15e820 0x7fadfc0c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.571 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.568+0000 7fae01167640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc150820 con 0x7fadfc154870 2023-08-11T15:50:45.571 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.568+0000 7fae01167640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0cb400 con 0x7fadfc15e820 2023-08-11T15:50:45.571 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.568+0000 7fae01167640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0cc710 con 0x7fadfc06d1b0 2023-08-11T15:50:45.571 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.568+0000 7fadf3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc154870 0x7fadfc0b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.571 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.568+0000 7fadf3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc154870 0x7fadfc0b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37532/0 (socket says 172.21.15.27:37532) 2023-08-11T15:50:45.571 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.568+0000 7fadf3fff640 1 -- 172.21.15.27:0/4040270891 learned_addr learned my addr 172.21.15.27:0/4040270891 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:45.571 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.569+0000 7fadfbfff640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc06d1b0 0x7fadfc1502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.571 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.569+0000 7fae00966640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc15e820 0x7fadfc0c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.572 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.569+0000 7fadf3fff640 1 -- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc06d1b0 msgr2=0x7fadfc1502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.572 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.569+0000 7fadf3fff640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc06d1b0 0x7fadfc1502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.572 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.569+0000 7fadf3fff640 1 -- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc15e820 msgr2=0x7fadfc0c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.572 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.569+0000 7fadf3fff640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc15e820 0x7fadfc0c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.572 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.569+0000 7fadf3fff640 1 -- 172.21.15.27:0/4040270891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4009b30 con 0x7fadfc154870 2023-08-11T15:50:45.573 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.569+0000 7fadf3fff640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc154870 0x7fadfc0b1380 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fade40097a0 tx=0x7fade40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:45.573 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.569+0000 7fadf97fa640 1 -- 172.21.15.27:0/4040270891 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4004230 con 0x7fadfc154870 2023-08-11T15:50:45.573 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.569+0000 7fadf97fa640 1 -- 172.21.15.27:0/4040270891 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade40043b0 con 0x7fadfc154870 2023-08-11T15:50:45.574 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.570+0000 7fadf97fa640 1 -- 172.21.15.27:0/4040270891 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4004520 con 0x7fadfc154870 2023-08-11T15:50:45.574 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.570+0000 7fae01167640 1 -- 172.21.15.27:0/4040270891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade4009400 con 0x7fadfc154870 2023-08-11T15:50:45.574 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.570+0000 7fae01167640 1 -- 172.21.15.27:0/4040270891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc0cb670 con 0x7fadfc154870 2023-08-11T15:50:45.574 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.571+0000 7fae01167640 1 -- 172.21.15.27:0/4040270891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc0669c0 con 0x7fadfc154870 2023-08-11T15:50:45.574 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.571+0000 7fadf97fa640 1 -- 172.21.15.27:0/4040270891 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7fade4032860 con 0x7fadfc154870 2023-08-11T15:50:45.575 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.572+0000 7fadf97fa640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faddc069ff0 0x7faddc06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.575 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.573+0000 7fadf97fa640 1 -- 172.21.15.27:0/4040270891 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fade40a95c0 con 0x7fadfc154870 2023-08-11T15:50:45.575 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.573+0000 7fadfbfff640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faddc069ff0 0x7faddc06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.575 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.574+0000 7fadfbfff640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faddc069ff0 0x7faddc06c4b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0af250 tx=0x7fadec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:45.578 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.577+0000 7fadf97fa640 1 -- 172.21.15.27:0/4040270891 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fade4075d30 con 0x7fadfc154870 2023-08-11T15:50:45.769 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.767+0000 7fae01167640 1 -- 172.21.15.27:0/4040270891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadfc066e80 con 0x7fadfc154870 2023-08-11T15:50:45.771 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.769+0000 7fadf97fa640 1 -- 172.21.15.27:0/4040270891 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fade4079350 con 0x7fadfc154870 2023-08-11T15:50:45.771 INFO:teuthology.misc.health.smithi027.stdout: 2023-08-11T15:50:45.771 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":13,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:50:43.718469+0000","last_up_change":"0.000000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:50:45.773 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.772+0000 7fae01167640 1 -- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faddc069ff0 msgr2=0x7faddc06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.773 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.772+0000 7fae01167640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faddc069ff0 0x7faddc06c4b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0af250 tx=0x7fadec009040 comp rx=0 tx=0).stop 2023-08-11T15:50:45.773 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.772+0000 7fae01167640 1 -- 172.21.15.27:0/4040270891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc154870 msgr2=0x7fadfc0b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.774 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.772+0000 7fae01167640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc154870 0x7fadfc0b1380 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fade40097a0 tx=0x7fade40046c0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.774 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.772+0000 7fae01167640 1 -- 172.21.15.27:0/4040270891 shutdown_connections 2023-08-11T15:50:45.774 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.772+0000 7fae01167640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faddc069ff0 0x7faddc06c4b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.775 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.772+0000 7fae01167640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc15e820 0x7fadfc0c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.775 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.772+0000 7fae01167640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc154870 0x7fadfc0b1380 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.775 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.772+0000 7fae01167640 1 --2- 172.21.15.27:0/4040270891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadfc06d1b0 0x7fadfc1502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.775 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.772+0000 7fae01167640 1 -- 172.21.15.27:0/4040270891 >> 172.21.15.27:0/4040270891 conn(0x7fadfc0cdd20 msgr2=0x7fadfc159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:45.775 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.773+0000 7fae01167640 1 -- 172.21.15.27:0/4040270891 shutdown_connections 2023-08-11T15:50:45.775 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:45.773+0000 7fae01167640 1 -- 172.21.15.27:0/4040270891 wait complete. 2023-08-11T15:50:45.787 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-08-11T15:50:46.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:50:46.718 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:50:52.593 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-11T15:50:52.794 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.792+0000 7fd545614640 1 Processor -- start 2023-08-11T15:50:52.794 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.792+0000 7fd545614640 1 -- start start 2023-08-11T15:50:52.795 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.794+0000 7fd545614640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd540151200 0x7fd540071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:52.795 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.794+0000 7fd545614640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd54006c0c0 0x7fd54006f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:52.795 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.794+0000 7fd545614640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54015ed40 con 0x7fd54006fa30 2023-08-11T15:50:52.796 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.794+0000 7fd545614640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd540071960 con 0x7fd54006c0c0 2023-08-11T15:50:52.796 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.794+0000 7fd545614640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54006fea0 con 0x7fd540151200 2023-08-11T15:50:52.797 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.794+0000 7fd53f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd54006fa30 0x7fd5400b2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33132/0 (socket says 172.21.15.27:33132) 2023-08-11T15:50:52.797 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.794+0000 7fd53f7fe640 1 -- 172.21.15.27:0/87468035 learned_addr learned my addr 172.21.15.27:0/87468035 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:50:52.797 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd5367fc640 1 --2- 172.21.15.27:0/87468035 >> v2:172.21.15.119:3300/0 conn(0x7fd54006c0c0 0x7fd54006f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:52.797 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53effd640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd540151200 0x7fd540071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:52.797 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683076133 0 0) 0x7fd54015ed40 con 0x7fd54006fa30 2023-08-11T15:50:52.797 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd52c003520 con 0x7fd54006fa30 2023-08-11T15:50:52.798 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4011545089 0 0) 0x7fd52c003520 con 0x7fd54006fa30 2023-08-11T15:50:52.798 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd540151200 msgr2=0x7fd540071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:52.798 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd540151200 0x7fd540071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:52.798 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 >> v2:172.21.15.119:3300/0 conn(0x7fd54006c0c0 msgr2=0x7fd54006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:52.798 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 --2- 172.21.15.27:0/87468035 >> v2:172.21.15.119:3300/0 conn(0x7fd54006c0c0 0x7fd54006f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:52.798 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54015b800 con 0x7fd54006fa30 2023-08-11T15:50:52.799 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fd5380019f0 con 0x7fd54006fa30 2023-08-11T15:50:52.799 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 >> v1:172.21.15.27:6789/0 conn(0x7fd54006fa30 legacy=0x7fd5400b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:52.799 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd540151200 0x7fd52c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:52.799 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.795+0000 7fd53e7fc640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 0x7fd52c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:52.799 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd53e7fc640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd52c004d00 0x7fd52c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:52.799 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd540071960 con 0x7fd52c003f20 2023-08-11T15:50:52.799 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54006fea0 con 0x7fd52c004d00 2023-08-11T15:50:52.800 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c00b980 con 0x7fd540151200 2023-08-11T15:50:52.801 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd5367fc640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 0x7fd52c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:52.801 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd53effd640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd540151200 0x7fd52c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:52.801 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd53f7fe640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd52c004d00 0x7fd52c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:52.801 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd5367fc640 1 -- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd540151200 msgr2=0x7fd52c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:52.802 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd5367fc640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd540151200 0x7fd52c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:52.802 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd5367fc640 1 -- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd52c004d00 msgr2=0x7fd52c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:52.802 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd5367fc640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd52c004d00 0x7fd52c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:52.802 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.796+0000 7fd5367fc640 1 -- 172.21.15.27:0/87468035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c003520 con 0x7fd52c003f20 2023-08-11T15:50:52.802 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.798+0000 7fd5367fc640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 0x7fd52c0047c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd528009ac0 tx=0x7fd52c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b0c7a4f144822a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:52.802 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.798+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52802c710 con 0x7fd52c003f20 2023-08-11T15:50:52.802 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.798+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52802c870 con 0x7fd52c003f20 2023-08-11T15:50:52.803 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.798+0000 7fd53e7fc640 1 -- 172.21.15.27:0/87468035 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52802cc00 con 0x7fd52c003f20 2023-08-11T15:50:52.803 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.798+0000 7fd545614640 1 -- 172.21.15.27:0/87468035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 msgr2=0x7fd52c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:52.803 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.798+0000 7fd545614640 1 --2- 172.21.15.27:0/87468035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 0x7fd52c0047c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd528009ac0 tx=0x7fd52c0057f0 comp rx=0 tx=0).stop 2023-08-11T15:50:52.803 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.798+0000 7fd53f7fe640 1 -- 172.21.15.27:0/87468035 reap_dead start 2023-08-11T15:50:52.803 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.799+0000 7fd545614640 1 -- 172.21.15.27:0/87468035 shutdown_connections 2023-08-11T15:50:52.803 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.799+0000 7fd545614640 1 -- 172.21.15.27:0/87468035 >> 172.21.15.27:0/87468035 conn(0x7fd5400afa90 msgr2=0x7fd5400afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:52.803 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.799+0000 7fd545614640 1 -- 172.21.15.27:0/87468035 shutdown_connections 2023-08-11T15:50:52.804 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.799+0000 7fd545614640 1 -- 172.21.15.27:0/87468035 wait complete. 2023-08-11T15:50:52.805 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.801+0000 7fd545614640 1 Processor -- start 2023-08-11T15:50:52.805 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.801+0000 7fd545614640 1 -- start start 2023-08-11T15:50:52.806 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.802+0000 7fd545614640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 0x7fd5401f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:52.806 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.802+0000 7fd545614640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5400cd6c0 0x7fd5400b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:52.806 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.802+0000 7fd545614640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5400b26c0 0x7fd540159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:52.806 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.802+0000 7fd545614640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400cbd60 con 0x7fd52c003f20 2023-08-11T15:50:52.806 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.802+0000 7fd545614640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400cdb40 con 0x7fd5400b26c0 2023-08-11T15:50:52.806 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.802+0000 7fd545614640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5401f5680 con 0x7fd5400cd6c0 2023-08-11T15:50:52.806 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.802+0000 7fd53effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 0x7fd5401f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:52.806 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.802+0000 7fd53effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 0x7fd5401f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49418/0 (socket says 172.21.15.27:49418) 2023-08-11T15:50:52.807 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.802+0000 7fd53effd640 1 -- 172.21.15.27:0/4101864715 learned_addr learned my addr 172.21.15.27:0/4101864715 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:52.807 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd5367fc640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5400cd6c0 0x7fd5400b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:52.807 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd53f7fe640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5400b26c0 0x7fd540159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:52.807 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd53effd640 1 -- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5400cd6c0 msgr2=0x7fd5400b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:52.807 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd53effd640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5400cd6c0 0x7fd5400b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:52.807 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd53effd640 1 -- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5400b26c0 msgr2=0x7fd540159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:52.807 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd53effd640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5400b26c0 0x7fd540159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:52.808 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd53effd640 1 -- 172.21.15.27:0/4101864715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5401f5960 con 0x7fd52c003f20 2023-08-11T15:50:52.808 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd5367fc640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5400cd6c0 0x7fd5400b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:52.808 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd53f7fe640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5400b26c0 0x7fd540159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:52.808 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd53effd640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 0x7fd5401f4f80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd530011600 tx=0x7fd530011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:52.808 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd537fff640 1 -- 172.21.15.27:0/4101864715 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53000f920 con 0x7fd52c003f20 2023-08-11T15:50:52.808 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd537fff640 1 -- 172.21.15.27:0/4101864715 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53000fa80 con 0x7fd52c003f20 2023-08-11T15:50:52.808 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd537fff640 1 -- 172.21.15.27:0/4101864715 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53000fdf0 con 0x7fd52c003f20 2023-08-11T15:50:52.809 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.803+0000 7fd545614640 1 -- 172.21.15.27:0/4101864715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5401f5c40 con 0x7fd52c003f20 2023-08-11T15:50:52.809 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.804+0000 7fd545614640 1 -- 172.21.15.27:0/4101864715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5400b2b40 con 0x7fd52c003f20 2023-08-11T15:50:52.809 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.805+0000 7fd545614640 1 -- 172.21.15.27:0/4101864715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5400cbb30 con 0x7fd52c003f20 2023-08-11T15:50:52.810 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.806+0000 7fd537fff640 1 -- 172.21.15.27:0/4101864715 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd530015460 con 0x7fd52c003f20 2023-08-11T15:50:52.810 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.807+0000 7fd537fff640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd52006a080 0x7fd52006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:52.810 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.807+0000 7fd537fff640 1 -- 172.21.15.27:0/4101864715 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd530088580 con 0x7fd52c003f20 2023-08-11T15:50:52.810 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.807+0000 7fd5367fc640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd52006a080 0x7fd52006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:52.810 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.809+0000 7fd5367fc640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd52006a080 0x7fd52006c540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd528036000 tx=0x7fd528030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:52.814 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:52.812+0000 7fd537fff640 1 -- 172.21.15.27:0/4101864715 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fd5400cbb30 con 0x7fd52c003f20 2023-08-11T15:50:53.007 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.006+0000 7fd545614640 1 -- 172.21.15.27:0/4101864715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5400cbb30 con 0x7fd52c003f20 2023-08-11T15:50:53.010 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.008+0000 7fd537fff640 1 -- 172.21.15.27:0/4101864715 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12850 (secure 0 0 0) 0x7fd5400cbb30 con 0x7fd52c003f20 2023-08-11T15:50:53.010 INFO:teuthology.misc.health.smithi027.stdout: 2023-08-11T15:50:53.011 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":17,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:50:49.161786+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:50:53.013 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.011+0000 7fd545614640 1 -- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd52006a080 msgr2=0x7fd52006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.014 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.011+0000 7fd545614640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd52006a080 0x7fd52006c540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd528036000 tx=0x7fd528030000 comp rx=0 tx=0).stop 2023-08-11T15:50:53.014 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.011+0000 7fd545614640 1 -- 172.21.15.27:0/4101864715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 msgr2=0x7fd5401f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.014 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.011+0000 7fd545614640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 0x7fd5401f4f80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd530011600 tx=0x7fd530011ac0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.014 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.011+0000 7fd545614640 1 -- 172.21.15.27:0/4101864715 shutdown_connections 2023-08-11T15:50:53.014 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.011+0000 7fd545614640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd52006a080 0x7fd52006c540 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.015 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.011+0000 7fd545614640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5400cd6c0 0x7fd5400b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.015 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.011+0000 7fd545614640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5400b26c0 0x7fd540159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.015 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.011+0000 7fd545614640 1 --2- 172.21.15.27:0/4101864715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52c003f20 0x7fd5401f4f80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.015 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.011+0000 7fd545614640 1 -- 172.21.15.27:0/4101864715 >> 172.21.15.27:0/4101864715 conn(0x7fd5400afa90 msgr2=0x7fd54015c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:53.015 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.012+0000 7fd545614640 1 -- 172.21.15.27:0/4101864715 shutdown_connections 2023-08-11T15:50:53.016 INFO:teuthology.misc.health.smithi027.stderr:2023-08-11T15:50:53.012+0000 7fd545614640 1 -- 172.21.15.27:0/4101864715 wait complete. 2023-08-11T15:50:53.027 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-08-11T15:50:53.028 INFO:tasks.ceph:Creating RBD pool 2023-08-11T15:50:53.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd pool create rbd 8 2023-08-11T15:50:53.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.253+0000 7f7e44b42640 1 Processor -- start 2023-08-11T15:50:53.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.253+0000 7f7e44b42640 1 -- start start 2023-08-11T15:50:53.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.255+0000 7f7e44b42640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e4006b670 0x7f7e400d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.255+0000 7f7e44b42640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e400d2850 0x7f7e400af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.255+0000 7f7e44b42640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4006bc10 con 0x7f7e4015c760 2023-08-11T15:50:53.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.255+0000 7f7e44b42640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4006c790 con 0x7f7e4006b670 2023-08-11T15:50:53.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.255+0000 7f7e44b42640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400d13f0 con 0x7f7e400d2850 2023-08-11T15:50:53.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.255+0000 7f7e3e59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7e4015c760 0x7f7e400d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33148/0 (socket says 172.21.15.27:33148) 2023-08-11T15:50:53.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.255+0000 7f7e3e59c640 1 -- 172.21.15.27:0/3133252097 learned_addr learned my addr 172.21.15.27:0/3133252097 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:50:53.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.255+0000 7f7e3dd9b640 1 --2- 172.21.15.27:0/3133252097 >> v2:172.21.15.119:3300/0 conn(0x7f7e4006b670 0x7f7e400d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322187984 0 0) 0x7f7e4006bc10 con 0x7f7e4015c760 2023-08-11T15:50:53.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e20003520 con 0x7f7e4015c760 2023-08-11T15:50:53.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3ed9d640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e400d2850 0x7f7e400af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3690927666 0 0) 0x7f7e20003520 con 0x7f7e4015c760 2023-08-11T15:50:53.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e400d2850 msgr2=0x7f7e400af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:53.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e400d2850 0x7f7e400af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 >> v2:172.21.15.119:3300/0 conn(0x7f7e4006b670 msgr2=0x7f7e400d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:53.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 --2- 172.21.15.27:0/3133252097 >> v2:172.21.15.119:3300/0 conn(0x7f7e4006b670 0x7f7e400d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4006bef0 con 0x7f7e4015c760 2023-08-11T15:50:53.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f7e30001670 con 0x7f7e4015c760 2023-08-11T15:50:53.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3dd9b640 1 --2- 172.21.15.27:0/3133252097 >> v2:172.21.15.119:3300/0 conn(0x7f7e4006b670 0x7f7e400d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:53.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 >> v1:172.21.15.27:6789/0 conn(0x7f7e4015c760 legacy=0x7f7e400d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e20003970 0x7f7e20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e200047c0 0x7f7e20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e400d2850 0x7f7e20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400d13f0 con 0x7f7e200047c0 2023-08-11T15:50:53.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2000b980 con 0x7f7e20003970 2023-08-11T15:50:53.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.256+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2000bac0 con 0x7f7e400d2850 2023-08-11T15:50:53.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.257+0000 7f7e3dd9b640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e200047c0 0x7f7e20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.257+0000 7f7e3e59c640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e20003970 0x7f7e20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.257+0000 7f7e3dd9b640 1 -- 172.21.15.27:0/3133252097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e400d2850 msgr2=0x7f7e20005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:53.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.257+0000 7f7e3dd9b640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e400d2850 0x7f7e20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.257+0000 7f7e3dd9b640 1 -- 172.21.15.27:0/3133252097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e20003970 msgr2=0x7f7e20003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:53.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.257+0000 7f7e3dd9b640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e20003970 0x7f7e20003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.257+0000 7f7e3dd9b640 1 -- 172.21.15.27:0/3133252097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20003520 con 0x7f7e200047c0 2023-08-11T15:50:53.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.258+0000 7f7e3dd9b640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e200047c0 0x7f7e20004c40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800e350 tx=0x7f7e200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923c78af373f12c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:53.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.259+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2802ce00 con 0x7f7e200047c0 2023-08-11T15:50:53.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.259+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2800f040 con 0x7f7e200047c0 2023-08-11T15:50:53.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.259+0000 7f7e3d59a640 1 -- 172.21.15.27:0/3133252097 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28034640 con 0x7f7e200047c0 2023-08-11T15:50:53.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.259+0000 7f7e44b42640 1 -- 172.21.15.27:0/3133252097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e200047c0 msgr2=0x7f7e20004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.259+0000 7f7e44b42640 1 --2- 172.21.15.27:0/3133252097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e200047c0 0x7f7e20004c40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800e350 tx=0x7f7e200057f0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.259+0000 7f7e3ed9d640 1 -- 172.21.15.27:0/3133252097 reap_dead start 2023-08-11T15:50:53.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.259+0000 7f7e44b42640 1 -- 172.21.15.27:0/3133252097 shutdown_connections 2023-08-11T15:50:53.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.259+0000 7f7e44b42640 1 -- 172.21.15.27:0/3133252097 >> 172.21.15.27:0/3133252097 conn(0x7f7e40089ff0 msgr2=0x7f7e4009b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:53.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.259+0000 7f7e44b42640 1 -- 172.21.15.27:0/3133252097 shutdown_connections 2023-08-11T15:50:53.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.259+0000 7f7e44b42640 1 -- 172.21.15.27:0/3133252097 wait complete. 2023-08-11T15:50:53.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.261+0000 7f7e44b42640 1 Processor -- start 2023-08-11T15:50:53.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.261+0000 7f7e44b42640 1 -- start start 2023-08-11T15:50:53.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.262+0000 7f7e44b42640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e200047c0 0x7f7e4014ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.262+0000 7f7e44b42640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e401e6420 0x7f7e401e68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.262+0000 7f7e44b42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e400ca7b0 0x7f7e400cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.262+0000 7f7e44b42640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e401e6de0 con 0x7f7e400ca7b0 2023-08-11T15:50:53.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.262+0000 7f7e44b42640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400cc080 con 0x7f7e401e6420 2023-08-11T15:50:53.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.262+0000 7f7e44b42640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400cb470 con 0x7f7e200047c0 2023-08-11T15:50:53.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e400ca7b0 0x7f7e400cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e400ca7b0 0x7f7e400cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49434/0 (socket says 172.21.15.27:49434) 2023-08-11T15:50:53.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3ed9d640 1 -- 172.21.15.27:0/4257839995 learned_addr learned my addr 172.21.15.27:0/4257839995 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:53.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3e59c640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e200047c0 0x7f7e4014ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3ed9d640 1 -- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e200047c0 msgr2=0x7f7e4014ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3ed9d640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e200047c0 0x7f7e4014ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3dd9b640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e401e6420 0x7f7e401e68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3ed9d640 1 -- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e401e6420 msgr2=0x7f7e401e68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3ed9d640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e401e6420 0x7f7e401e68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3ed9d640 1 -- 172.21.15.27:0/4257839995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e400cb750 con 0x7f7e400ca7b0 2023-08-11T15:50:53.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3e59c640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e200047c0 0x7f7e4014ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:53.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.263+0000 7f7e3ed9d640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e400ca7b0 0x7f7e400cac30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7e400b5260 tx=0x7f7e3800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:53.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.264+0000 7f7e36ffd640 1 -- 172.21.15.27:0/4257839995 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800c680 con 0x7f7e400ca7b0 2023-08-11T15:50:53.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.264+0000 7f7e36ffd640 1 -- 172.21.15.27:0/4257839995 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3800c7e0 con 0x7f7e400ca7b0 2023-08-11T15:50:53.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.264+0000 7f7e36ffd640 1 -- 172.21.15.27:0/4257839995 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800cb90 con 0x7f7e400ca7b0 2023-08-11T15:50:53.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.264+0000 7f7e44b42640 1 -- 172.21.15.27:0/4257839995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e400cba30 con 0x7f7e400ca7b0 2023-08-11T15:50:53.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.264+0000 7f7e44b42640 1 -- 172.21.15.27:0/4257839995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e400cb170 con 0x7f7e400ca7b0 2023-08-11T15:50:53.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.265+0000 7f7e44b42640 1 -- 172.21.15.27:0/4257839995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4004e420 con 0x7f7e400ca7b0 2023-08-11T15:50:53.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.266+0000 7f7e36ffd640 1 -- 172.21.15.27:0/4257839995 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7e3800ccf0 con 0x7f7e400ca7b0 2023-08-11T15:50:53.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.267+0000 7f7e36ffd640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e2406a050 0x7f7e2406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.267+0000 7f7e36ffd640 1 -- 172.21.15.27:0/4257839995 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7e38087a00 con 0x7f7e400ca7b0 2023-08-11T15:50:53.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.268+0000 7f7e3e59c640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e2406a050 0x7f7e2406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.269+0000 7f7e3e59c640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e2406a050 0x7f7e2406c510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7e300058b0 tx=0x7f7e30000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:53.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.271+0000 7f7e36ffd640 1 -- 172.21.15.27:0/4257839995 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f7e38053500 con 0x7f7e400ca7b0 2023-08-11T15:50:53.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.466+0000 7f7e44b42640 1 -- 172.21.15.27:0/4257839995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f7e400d34b0 con 0x7f7e400ca7b0 2023-08-11T15:50:53.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.645+0000 7f7e36ffd640 1 -- 172.21.15.27:0/4257839995 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7f7e38056b20 con 0x7f7e400ca7b0 2023-08-11T15:50:53.647 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'rbd' created 2023-08-11T15:50:53.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.650+0000 7f7e44b42640 1 -- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e2406a050 msgr2=0x7f7e2406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.650+0000 7f7e44b42640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e2406a050 0x7f7e2406c510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7e300058b0 tx=0x7f7e30000ed0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.651+0000 7f7e44b42640 1 -- 172.21.15.27:0/4257839995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e400ca7b0 msgr2=0x7f7e400cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.651+0000 7f7e44b42640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e400ca7b0 0x7f7e400cac30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7e400b5260 tx=0x7f7e3800ec00 comp rx=0 tx=0).stop 2023-08-11T15:50:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.651+0000 7f7e44b42640 1 -- 172.21.15.27:0/4257839995 shutdown_connections 2023-08-11T15:50:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.651+0000 7f7e44b42640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e2406a050 0x7f7e2406c510 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.651+0000 7f7e44b42640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e400ca7b0 0x7f7e400cac30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.651+0000 7f7e44b42640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e401e6420 0x7f7e401e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.651+0000 7f7e44b42640 1 --2- 172.21.15.27:0/4257839995 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e200047c0 0x7f7e4014ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.651+0000 7f7e44b42640 1 -- 172.21.15.27:0/4257839995 >> 172.21.15.27:0/4257839995 conn(0x7f7e40089ff0 msgr2=0x7f7e400cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:53.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.652+0000 7f7e44b42640 1 -- 172.21.15.27:0/4257839995 shutdown_connections 2023-08-11T15:50:53.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.652+0000 7f7e44b42640 1 -- 172.21.15.27:0/4257839995 wait complete. 2023-08-11T15:50:53.676 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-08-11T15:50:53.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.888+0000 7f392b485640 1 Processor -- start 2023-08-11T15:50:53.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.888+0000 7f392b485640 1 -- start start 2023-08-11T15:50:53.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.890+0000 7f392b485640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f392415a1a0 0x7f392415e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.890+0000 7f392b485640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f392406f5f0 0x7f392415b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.890+0000 7f392b485640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392415ec50 con 0x7f39240c8650 2023-08-11T15:50:53.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.890+0000 7f392b485640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392406fd80 con 0x7f392406f5f0 2023-08-11T15:50:53.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.890+0000 7f392b485640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3924070c50 con 0x7f392415a1a0 2023-08-11T15:50:53.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.890+0000 7f3929c82640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f392406f5f0 0x7f392415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.890+0000 7f3929c82640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f392406f5f0 0x7f392415b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49448/0 (socket says 172.21.15.27:49448) 2023-08-11T15:50:53.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.890+0000 7f3929c82640 1 -- 172.21.15.27:0/2710283703 learned_addr learned my addr 172.21.15.27:0/2710283703 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:53.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.891+0000 7f392a483640 1 --2- 172.21.15.27:0/2710283703 >> v2:172.21.15.119:3300/0 conn(0x7f392415a1a0 0x7f392415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.891+0000 7f3929c82640 1 -- 172.21.15.27:0/2710283703 >> v2:172.21.15.119:3300/0 conn(0x7f392415a1a0 msgr2=0x7f392415e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:53.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.891+0000 7f3929c82640 1 --2- 172.21.15.27:0/2710283703 >> v2:172.21.15.119:3300/0 conn(0x7f392415a1a0 0x7f392415e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.891+0000 7f3929c82640 1 -- 172.21.15.27:0/2710283703 >> v1:172.21.15.119:6789/0 conn(0x7f39240c8650 legacy=0x7f39240c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:53.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.891+0000 7f3929c82640 1 -- 172.21.15.27:0/2710283703 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39240718b0 con 0x7f392406f5f0 2023-08-11T15:50:53.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.892+0000 7f3929c82640 1 --2- 172.21.15.27:0/2710283703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392406f5f0 0x7f392415b070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f392406da90 tx=0x7f391402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ccac2081400f75 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:53.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.893+0000 7f3929481640 1 -- 172.21.15.27:0/2710283703 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39140317e0 con 0x7f392406f5f0 2023-08-11T15:50:53.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.893+0000 7f3929481640 1 -- 172.21.15.27:0/2710283703 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3914031940 con 0x7f392406f5f0 2023-08-11T15:50:53.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.893+0000 7f3929481640 1 -- 172.21.15.27:0/2710283703 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914031cd0 con 0x7f392406f5f0 2023-08-11T15:50:53.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.894+0000 7f392b485640 1 -- 172.21.15.27:0/2710283703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392406f5f0 msgr2=0x7f392415b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.894+0000 7f392b485640 1 --2- 172.21.15.27:0/2710283703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392406f5f0 0x7f392415b070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f392406da90 tx=0x7f391402b880 comp rx=0 tx=0).stop 2023-08-11T15:50:53.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.894+0000 7f392b485640 1 -- 172.21.15.27:0/2710283703 shutdown_connections 2023-08-11T15:50:53.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.894+0000 7f392b485640 1 --2- 172.21.15.27:0/2710283703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392406f5f0 0x7f392415b070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.894+0000 7f392b485640 1 --2- 172.21.15.27:0/2710283703 >> v2:172.21.15.119:3300/0 conn(0x7f392415a1a0 0x7f392415e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.894+0000 7f392b485640 1 -- 172.21.15.27:0/2710283703 >> 172.21.15.27:0/2710283703 conn(0x7f39240afb70 msgr2=0x7f39240aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:53.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.894+0000 7f392b485640 1 -- 172.21.15.27:0/2710283703 shutdown_connections 2023-08-11T15:50:53.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.894+0000 7f392b485640 1 -- 172.21.15.27:0/2710283703 wait complete. 2023-08-11T15:50:53.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.896+0000 7f392b485640 1 Processor -- start 2023-08-11T15:50:53.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.896+0000 7f392b485640 1 -- start start 2023-08-11T15:50:53.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f392b485640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392406f5f0 0x7f39240b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f392b485640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39240c8650 0x7f39240c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f392b485640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392415a1a0 0x7f39240b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f392b485640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39240c4910 con 0x7f392415a1a0 2023-08-11T15:50:53.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f392b485640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39240b1e50 con 0x7f392406f5f0 2023-08-11T15:50:53.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f392b485640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39240b1240 con 0x7f39240c8650 2023-08-11T15:50:53.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f392ac84640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392415a1a0 0x7f39240b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f392ac84640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392415a1a0 0x7f39240b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49460/0 (socket says 172.21.15.27:49460) 2023-08-11T15:50:53.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f392ac84640 1 -- 172.21.15.27:0/716409568 learned_addr learned my addr 172.21.15.27:0/716409568 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:53.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f392a483640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392406f5f0 0x7f39240b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.897+0000 7f3929c82640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39240c8650 0x7f39240c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f392ac84640 1 -- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39240c8650 msgr2=0x7f39240c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f392ac84640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39240c8650 0x7f39240c43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f392ac84640 1 -- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392406f5f0 msgr2=0x7f39240b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:53.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f392ac84640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392406f5f0 0x7f39240b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:53.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f392ac84640 1 -- 172.21.15.27:0/716409568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140094d0 con 0x7f392415a1a0 2023-08-11T15:50:53.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f392ac84640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392415a1a0 0x7f39240b3210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c780 tx=0x7f391c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:53.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f391affd640 1 -- 172.21.15.27:0/716409568 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c004b60 con 0x7f392415a1a0 2023-08-11T15:50:53.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f391affd640 1 -- 172.21.15.27:0/716409568 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c00ce90 con 0x7f392415a1a0 2023-08-11T15:50:53.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f391affd640 1 -- 172.21.15.27:0/716409568 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c00a850 con 0x7f392415a1a0 2023-08-11T15:50:53.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f392b485640 1 -- 172.21.15.27:0/716409568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39240b1520 con 0x7f392415a1a0 2023-08-11T15:50:53.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.898+0000 7f392b485640 1 -- 172.21.15.27:0/716409568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39240b1ae0 con 0x7f392415a1a0 2023-08-11T15:50:53.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.899+0000 7f392b485640 1 -- 172.21.15.27:0/716409568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39240b1fe0 con 0x7f392415a1a0 2023-08-11T15:50:53.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.901+0000 7f391affd640 1 -- 172.21.15.27:0/716409568 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f391c006b50 con 0x7f392415a1a0 2023-08-11T15:50:53.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.902+0000 7f391affd640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f390806a080 0x7f390806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:53.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.902+0000 7f391affd640 1 -- 172.21.15.27:0/716409568 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f391c088b70 con 0x7f392415a1a0 2023-08-11T15:50:53.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.903+0000 7f392a483640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f390806a080 0x7f390806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:53.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.904+0000 7f392a483640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f390806a080 0x7f390806c540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3920009a20 tx=0x7f3920008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:53.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:53.906+0000 7f391affd640 1 -- 172.21.15.27:0/716409568 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f391c0545a0 con 0x7f392415a1a0 2023-08-11T15:50:54.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.099+0000 7f392b485640 1 -- 172.21.15.27:0/716409568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f392404e420 con 0x7f392415a1a0 2023-08-11T15:50:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.649+0000 7f391affd640 1 -- 172.21.15.27:0/716409568 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7f391c057bc0 con 0x7f392415a1a0 2023-08-11T15:50:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:enabled application 'rbd' on pool 'rbd' 2023-08-11T15:50:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.653+0000 7f392b485640 1 -- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f390806a080 msgr2=0x7f390806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.654+0000 7f392b485640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f390806a080 0x7f390806c540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3920009a20 tx=0x7f3920008040 comp rx=0 tx=0).stop 2023-08-11T15:50:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.654+0000 7f392b485640 1 -- 172.21.15.27:0/716409568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392415a1a0 msgr2=0x7f39240b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.654+0000 7f392b485640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392415a1a0 0x7f39240b3210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c780 tx=0x7f391c00cc40 comp rx=0 tx=0).stop 2023-08-11T15:50:54.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.654+0000 7f392b485640 1 -- 172.21.15.27:0/716409568 shutdown_connections 2023-08-11T15:50:54.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.654+0000 7f392b485640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f390806a080 0x7f390806c540 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.654+0000 7f392b485640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39240c8650 0x7f39240c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.654+0000 7f392b485640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392415a1a0 0x7f39240b3210 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.654+0000 7f392b485640 1 --2- 172.21.15.27:0/716409568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392406f5f0 0x7f39240b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.654+0000 7f392b485640 1 -- 172.21.15.27:0/716409568 >> 172.21.15.27:0/716409568 conn(0x7f39240afb70 msgr2=0x7f39240ca3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.655+0000 7f392b485640 1 -- 172.21.15.27:0/716409568 shutdown_connections 2023-08-11T15:50:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.655+0000 7f392b485640 1 -- 172.21.15.27:0/716409568 wait complete. 2023-08-11T15:50:54.682 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-08-11T15:50:54.683 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-11T15:50:54.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-11T15:50:54.688 INFO:tasks.ceph.mds.a:Started 2023-08-11T15:50:54.688 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-11T15:50:54.689 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-11T15:50:54.693 INFO:tasks.ceph.mds.c:Started 2023-08-11T15:50:54.694 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-11T15:50:54.695 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-11T15:50:54.703 INFO:tasks.ceph.mds.b:Started 2023-08-11T15:50:54.704 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-11T15:50:54.704 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-11T15:50:54.712 INFO:tasks.ceph.mds.d:Started 2023-08-11T15:50:54.713 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-08-11T15:50:54.713 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:50:54.859 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-08-11T15:50:54.860 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-08-11T15:50:54.871 INFO:tasks.ceph.mds.b.smithi119.stdout:starting mds.b at 2023-08-11T15:50:54.879 INFO:tasks.ceph.mds.d.smithi119.stdout:starting mds.d at 2023-08-11T15:50:54.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.918+0000 7f736ee25640 1 Processor -- start 2023-08-11T15:50:54.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.918+0000 7f736ee25640 1 -- start start 2023-08-11T15:50:54.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.919+0000 7f736ee25640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 0x7f73681609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:54.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.919+0000 7f736ee25640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73680b5170 con 0x7f736815c1f0 2023-08-11T15:50:54.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.919+0000 7f736ee25640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73680b81a0 con 0x7f73680b7d40 2023-08-11T15:50:54.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.919+0000 7f736ee25640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736815d080 con 0x7f7368160510 2023-08-11T15:50:54.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.919+0000 7f736de23640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f736815c1f0 0x7f736815c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33164/0 (socket says 172.21.15.27:33164) 2023-08-11T15:50:54.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.919+0000 7f736de23640 1 -- 172.21.15.27:0/1234758312 learned_addr learned my addr 172.21.15.27:0/1234758312 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:50:54.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.920+0000 7f736e624640 1 --2- 172.21.15.27:0/1234758312 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 0x7f73681609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:54.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.920+0000 7f736ce21640 1 -- 172.21.15.27:0/1234758312 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889764048 0 0) 0x7f73680b5170 con 0x7f736815c1f0 2023-08-11T15:50:54.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.920+0000 7f736ce21640 1 -- 172.21.15.27:0/1234758312 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7354003520 con 0x7f736815c1f0 2023-08-11T15:50:54.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.920+0000 7f736ce21640 1 -- 172.21.15.27:0/1234758312 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813522817 0 0) 0x7f73680b81a0 con 0x7f73680b7d40 2023-08-11T15:50:54.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.920+0000 7f736ce21640 1 -- 172.21.15.27:0/1234758312 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73680b5170 con 0x7f73680b7d40 2023-08-11T15:50:54.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.920+0000 7f736e624640 1 -- 172.21.15.27:0/1234758312 >> v1:172.21.15.119:6789/0 conn(0x7f73680b7d40 legacy=0x7f73680d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:54.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.920+0000 7f736e624640 1 -- 172.21.15.27:0/1234758312 >> v1:172.21.15.27:6789/0 conn(0x7f736815c1f0 legacy=0x7f736815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:54.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.920+0000 7f736e624640 1 -- 172.21.15.27:0/1234758312 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736815d360 con 0x7f7368160510 2023-08-11T15:50:54.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.922+0000 7f736e624640 1 --2- 172.21.15.27:0/1234758312 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 0x7f73681609a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7360009dc0 tx=0x7f736002bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c603994d148326b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:54.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.922+0000 7f736ce21640 1 -- 172.21.15.27:0/1234758312 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736002cd60 con 0x7f7368160510 2023-08-11T15:50:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.922+0000 7f736ce21640 1 -- 172.21.15.27:0/1234758312 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7360034440 con 0x7f7368160510 2023-08-11T15:50:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.922+0000 7f736ce21640 1 -- 172.21.15.27:0/1234758312 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73600347b0 con 0x7f7368160510 2023-08-11T15:50:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.923+0000 7f736ee25640 1 -- 172.21.15.27:0/1234758312 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 msgr2=0x7f73681609a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.923+0000 7f736ee25640 1 --2- 172.21.15.27:0/1234758312 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 0x7f73681609a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7360009dc0 tx=0x7f736002bfb0 comp rx=0 tx=0).stop 2023-08-11T15:50:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.923+0000 7f736ee25640 1 -- 172.21.15.27:0/1234758312 shutdown_connections 2023-08-11T15:50:54.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.923+0000 7f736ee25640 1 --2- 172.21.15.27:0/1234758312 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 0x7f73681609a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:54.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.923+0000 7f736ee25640 1 -- 172.21.15.27:0/1234758312 >> 172.21.15.27:0/1234758312 conn(0x7f736806a250 msgr2=0x7f736806a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:54.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.923+0000 7f736ee25640 1 -- 172.21.15.27:0/1234758312 shutdown_connections 2023-08-11T15:50:54.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.923+0000 7f736ee25640 1 -- 172.21.15.27:0/1234758312 wait complete. 2023-08-11T15:50:54.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.925+0000 7f736ee25640 1 Processor -- start 2023-08-11T15:50:54.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.925+0000 7f736ee25640 1 -- start start 2023-08-11T15:50:54.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736ee25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73680b7d40 0x7f73681f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:54.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736ee25640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f736815c1f0 0x7f736815f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:54.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736ee25640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 0x7f73680b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:54.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736ee25640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736815fc50 con 0x7f736815c1f0 2023-08-11T15:50:54.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736ee25640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73681f2060 con 0x7f73680b7d40 2023-08-11T15:50:54.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736ee25640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73681f2340 con 0x7f7368160510 2023-08-11T15:50:54.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736d622640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f736815c1f0 0x7f736815f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:54.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736d622640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f736815c1f0 0x7f736815f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49478/0 (socket says 172.21.15.27:49478) 2023-08-11T15:50:54.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736d622640 1 -- 172.21.15.27:0/3469693937 learned_addr learned my addr 172.21.15.27:0/3469693937 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:54.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736de23640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73680b7d40 0x7f73681f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:54.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736e624640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 0x7f73680b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:54.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736d622640 1 -- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 msgr2=0x7f73680b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:54.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736d622640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 0x7f73680b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:54.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.926+0000 7f736d622640 1 -- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73680b7d40 msgr2=0x7f73681f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:54.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.927+0000 7f736d622640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73680b7d40 0x7f73681f1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:54.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.927+0000 7f736d622640 1 -- 172.21.15.27:0/3469693937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360009a70 con 0x7f736815c1f0 2023-08-11T15:50:54.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.927+0000 7f736d622640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f736815c1f0 0x7f736815f710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f735800c500 tx=0x7f735800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:54.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.927+0000 7f735e7fc640 1 -- 172.21.15.27:0/3469693937 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73580052d0 con 0x7f736815c1f0 2023-08-11T15:50:54.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.927+0000 7f735e7fc640 1 -- 172.21.15.27:0/3469693937 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358005450 con 0x7f736815c1f0 2023-08-11T15:50:54.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.927+0000 7f735e7fc640 1 -- 172.21.15.27:0/3469693937 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358012870 con 0x7f736815c1f0 2023-08-11T15:50:54.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.927+0000 7f736ee25640 1 -- 172.21.15.27:0/3469693937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73681f2620 con 0x7f736815c1f0 2023-08-11T15:50:54.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.927+0000 7f736ee25640 1 -- 172.21.15.27:0/3469693937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73681f2900 con 0x7f736815c1f0 2023-08-11T15:50:54.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.929+0000 7f736ee25640 1 -- 172.21.15.27:0/3469693937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736804e420 con 0x7f736815c1f0 2023-08-11T15:50:54.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.930+0000 7f735e7fc640 1 -- 172.21.15.27:0/3469693937 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f73580129d0 con 0x7f736815c1f0 2023-08-11T15:50:54.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.931+0000 7f735e7fc640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f734c06a050 0x7f734c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:54.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.931+0000 7f735e7fc640 1 -- 172.21.15.27:0/3469693937 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7358087f10 con 0x7f736815c1f0 2023-08-11T15:50:54.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.931+0000 7f736de23640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f734c06a050 0x7f734c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:54.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.932+0000 7f736de23640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f734c06a050 0x7f734c06c510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7364005880 tx=0x7f7364009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:54.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:54.935+0000 7f735e7fc640 1 -- 172.21.15.27:0/3469693937 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f73580538c0 con 0x7f736815c1f0 2023-08-11T15:50:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.133+0000 7f736ee25640 1 -- 172.21.15.27:0/3469693937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73680cfd80 con 0x7f736815c1f0 2023-08-11T15:50:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.136+0000 7f735e7fc640 1 -- 172.21.15.27:0/3469693937 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14649 (secure 0 0 0) 0x7f7358056ee0 con 0x7f736815c1f0 2023-08-11T15:50:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:50:55.138 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":19,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:50:54.641208+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:50:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.138+0000 7f736ee25640 1 -- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f734c06a050 msgr2=0x7f734c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.138+0000 7f736ee25640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f734c06a050 0x7f734c06c510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7364005880 tx=0x7f7364009040 comp rx=0 tx=0).stop 2023-08-11T15:50:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.138+0000 7f736ee25640 1 -- 172.21.15.27:0/3469693937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f736815c1f0 msgr2=0x7f736815f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.138+0000 7f736ee25640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f736815c1f0 0x7f736815f710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f735800c500 tx=0x7f735800c9c0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.139+0000 7f736ee25640 1 -- 172.21.15.27:0/3469693937 shutdown_connections 2023-08-11T15:50:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.139+0000 7f736ee25640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f734c06a050 0x7f734c06c510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.139+0000 7f736ee25640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7368160510 0x7f73680b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.139+0000 7f736ee25640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f736815c1f0 0x7f736815f710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.139+0000 7f736ee25640 1 --2- 172.21.15.27:0/3469693937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73680b7d40 0x7f73681f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.139+0000 7f736ee25640 1 -- 172.21.15.27:0/3469693937 >> 172.21.15.27:0/3469693937 conn(0x7f736806a250 msgr2=0x7f73680cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.139+0000 7f736ee25640 1 -- 172.21.15.27:0/3469693937 shutdown_connections 2023-08-11T15:50:55.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.139+0000 7f736ee25640 1 -- 172.21.15.27:0/3469693937 wait complete. 2023-08-11T15:50:55.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T15:50:55.158 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:50:55.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.386+0000 7f608235d640 1 Processor -- start 2023-08-11T15:50:55.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.386+0000 7f608235d640 1 -- start start 2023-08-11T15:50:55.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.388+0000 7f608235d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f607c154ac0 0x7f607c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.388+0000 7f608235d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c140ba0 con 0x7f607c15b6b0 2023-08-11T15:50:55.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.388+0000 7f608235d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c15b220 con 0x7f607c1403a0 2023-08-11T15:50:55.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.388+0000 7f608235d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c071b10 con 0x7f607c154ac0 2023-08-11T15:50:55.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.388+0000 7f607b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f607c15b6b0 0x7f607c13fc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33172/0 (socket says 172.21.15.27:33172) 2023-08-11T15:50:55.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.388+0000 7f607b7fe640 1 -- 172.21.15.27:0/437343788 learned_addr learned my addr 172.21.15.27:0/437343788 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:50:55.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.388+0000 7f607bfff640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f607c154ac0 0x7f607c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.388+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108984178 0 0) 0x7f607c140ba0 con 0x7f607c15b6b0 2023-08-11T15:50:55.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6068003520 con 0x7f607c15b6b0 2023-08-11T15:50:55.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2663453723 0 0) 0x7f6068003520 con 0x7f607c15b6b0 2023-08-11T15:50:55.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f607c154ac0 msgr2=0x7f607c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:55.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f607c154ac0 0x7f607c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 >> v1:172.21.15.119:6789/0 conn(0x7f607c1403a0 legacy=0x7f607c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c1408c0 con 0x7f607c15b6b0 2023-08-11T15:50:55.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f6064001670 con 0x7f607c15b6b0 2023-08-11T15:50:55.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 >> v1:172.21.15.27:6789/0 conn(0x7f607c15b6b0 legacy=0x7f607c13fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60680039a0 0x7f6068003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60680047c0 0x7f6068004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f607c154ac0 0x7f60680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c071b10 con 0x7f60680047c0 2023-08-11T15:50:55.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606800bab0 con 0x7f60680039a0 2023-08-11T15:50:55.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606800bc70 con 0x7f607c154ac0 2023-08-11T15:50:55.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.389+0000 7f607b7fe640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60680047c0 0x7f6068004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.390+0000 7f607bfff640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60680039a0 0x7f6068003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.390+0000 7f607b7fe640 1 -- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f607c154ac0 msgr2=0x7f60680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:55.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.390+0000 7f60808fa640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f607c154ac0 0x7f60680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.390+0000 7f607b7fe640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f607c154ac0 0x7f60680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.390+0000 7f607b7fe640 1 -- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60680039a0 msgr2=0x7f6068003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:55.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.390+0000 7f607b7fe640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60680039a0 0x7f6068003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.390+0000 7f607b7fe640 1 -- 172.21.15.27:0/437343788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6068003520 con 0x7f60680047c0 2023-08-11T15:50:55.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.390+0000 7f607bfff640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60680039a0 0x7f6068003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:50:55.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.390+0000 7f60808fa640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f607c154ac0 0x7f60680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:55.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.391+0000 7f607b7fe640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60680047c0 0x7f6068004c40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f606400c240 tx=0x7f60680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309e270cfbb3a5ed server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:55.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.391+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064005ad0 con 0x7f60680047c0 2023-08-11T15:50:55.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.391+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6064005c50 con 0x7f60680047c0 2023-08-11T15:50:55.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.391+0000 7f607affd640 1 -- 172.21.15.27:0/437343788 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606402c8b0 con 0x7f60680047c0 2023-08-11T15:50:55.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.392+0000 7f608235d640 1 -- 172.21.15.27:0/437343788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60680047c0 msgr2=0x7f6068004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.392+0000 7f608235d640 1 --2- 172.21.15.27:0/437343788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60680047c0 0x7f6068004c40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f606400c240 tx=0x7f60680057f0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.392+0000 7f60808fa640 1 -- 172.21.15.27:0/437343788 reap_dead start 2023-08-11T15:50:55.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.392+0000 7f608235d640 1 -- 172.21.15.27:0/437343788 shutdown_connections 2023-08-11T15:50:55.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.392+0000 7f608235d640 1 -- 172.21.15.27:0/437343788 >> 172.21.15.27:0/437343788 conn(0x7f607c069bf0 msgr2=0x7f607c06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:55.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.392+0000 7f608235d640 1 -- 172.21.15.27:0/437343788 shutdown_connections 2023-08-11T15:50:55.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.392+0000 7f608235d640 1 -- 172.21.15.27:0/437343788 wait complete. 2023-08-11T15:50:55.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.394+0000 7f608235d640 1 Processor -- start 2023-08-11T15:50:55.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.394+0000 7f608235d640 1 -- start start 2023-08-11T15:50:55.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.395+0000 7f608235d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60680047c0 0x7f607c0c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.395+0000 7f608235d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607c153ed0 0x7f607c0c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.395+0000 7f608235d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f607c0c1cf0 0x7f607c0c2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.395+0000 7f608235d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0c2ab0 con 0x7f607c0c1cf0 2023-08-11T15:50:55.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.395+0000 7f608235d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0c2d90 con 0x7f607c153ed0 2023-08-11T15:50:55.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.395+0000 7f608235d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0c3070 con 0x7f60680047c0 2023-08-11T15:50:55.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f60808fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f607c0c1cf0 0x7f607c0c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f60808fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f607c0c1cf0 0x7f607c0c2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49484/0 (socket says 172.21.15.27:49484) 2023-08-11T15:50:55.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f60808fa640 1 -- 172.21.15.27:0/661649113 learned_addr learned my addr 172.21.15.27:0/661649113 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:55.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f607b7fe640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607c153ed0 0x7f607c0c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f607bfff640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60680047c0 0x7f607c0c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f60808fa640 1 -- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60680047c0 msgr2=0x7f607c0c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f60808fa640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60680047c0 0x7f607c0c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f60808fa640 1 -- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607c153ed0 msgr2=0x7f607c0c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f60808fa640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607c153ed0 0x7f607c0c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f60808fa640 1 -- 172.21.15.27:0/661649113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c0c3350 con 0x7f607c0c1cf0 2023-08-11T15:50:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.396+0000 7f60808fa640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f607c0c1cf0 0x7f607c0c2170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f607c144250 tx=0x7f607400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.397+0000 7f6078ff9640 1 -- 172.21.15.27:0/661649113 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400b070 con 0x7f607c0c1cf0 2023-08-11T15:50:55.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.397+0000 7f6078ff9640 1 -- 172.21.15.27:0/661649113 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60740054c0 con 0x7f607c0c1cf0 2023-08-11T15:50:55.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.397+0000 7f6078ff9640 1 -- 172.21.15.27:0/661649113 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074007250 con 0x7f607c0c1cf0 2023-08-11T15:50:55.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.397+0000 7f608235d640 1 -- 172.21.15.27:0/661649113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c0c3630 con 0x7f607c0c1cf0 2023-08-11T15:50:55.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.397+0000 7f608235d640 1 -- 172.21.15.27:0/661649113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c0cc050 con 0x7f607c0c1cf0 2023-08-11T15:50:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.398+0000 7f608235d640 1 -- 172.21.15.27:0/661649113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c04e420 con 0x7f607c0c1cf0 2023-08-11T15:50:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.399+0000 7f6078ff9640 1 -- 172.21.15.27:0/661649113 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f60740073b0 con 0x7f607c0c1cf0 2023-08-11T15:50:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.400+0000 7f6078ff9640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f605c06a050 0x7f605c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.400+0000 7f6078ff9640 1 -- 172.21.15.27:0/661649113 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6074087cf0 con 0x7f607c0c1cf0 2023-08-11T15:50:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.401+0000 7f607bfff640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f605c06a050 0x7f605c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.402+0000 7f607bfff640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f605c06a050 0x7f605c06c510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f606c007520 tx=0x7f606c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.405+0000 7f6078ff9640 1 -- 172.21.15.27:0/661649113 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6074053690 con 0x7f607c0c1cf0 2023-08-11T15:50:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.597+0000 7f608235d640 1 -- 172.21.15.27:0/661649113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f607c06c020 con 0x7f607c0c1cf0 2023-08-11T15:50:55.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.598+0000 7f6078ff9640 1 -- 172.21.15.27:0/661649113 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6074056cb0 con 0x7f607c0c1cf0 2023-08-11T15:50:55.600 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:50:55.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.601+0000 7f608235d640 1 -- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f605c06a050 msgr2=0x7f605c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.601+0000 7f608235d640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f605c06a050 0x7f605c06c510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f606c007520 tx=0x7f606c00b040 comp rx=0 tx=0).stop 2023-08-11T15:50:55.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.601+0000 7f608235d640 1 -- 172.21.15.27:0/661649113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f607c0c1cf0 msgr2=0x7f607c0c2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.601+0000 7f608235d640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f607c0c1cf0 0x7f607c0c2170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f607c144250 tx=0x7f607400d890 comp rx=0 tx=0).stop 2023-08-11T15:50:55.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.602+0000 7f608235d640 1 -- 172.21.15.27:0/661649113 shutdown_connections 2023-08-11T15:50:55.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.602+0000 7f608235d640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f605c06a050 0x7f605c06c510 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.602+0000 7f608235d640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f607c0c1cf0 0x7f607c0c2170 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.602+0000 7f608235d640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f607c153ed0 0x7f607c0c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.602+0000 7f608235d640 1 --2- 172.21.15.27:0/661649113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60680047c0 0x7f607c0c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.602+0000 7f608235d640 1 -- 172.21.15.27:0/661649113 >> 172.21.15.27:0/661649113 conn(0x7f607c069bf0 msgr2=0x7f607c0adab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.602+0000 7f608235d640 1 -- 172.21.15.27:0/661649113 shutdown_connections 2023-08-11T15:50:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.602+0000 7f608235d640 1 -- 172.21.15.27:0/661649113 wait complete. 2023-08-11T15:50:55.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:50:55.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.847+0000 7f58bbfff640 1 Processor -- start 2023-08-11T15:50:55.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.847+0000 7f58bbfff640 1 -- start start 2023-08-11T15:50:55.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.849+0000 7f58bbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58bc13ce80 0x7f58bc13f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.849+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58bc13d260 0x7f58bc13fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.849+0000 7f58bbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f58bc0cc720 0x7f58bc0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.849+0000 7f58bbfff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc140670 con 0x7f58bc13ce80 2023-08-11T15:50:55.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.849+0000 7f58bbfff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc0cd470 con 0x7f58bc0cc720 2023-08-11T15:50:55.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.849+0000 7f58bbfff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0cd750 con 0x7f58bc13d260 2023-08-11T15:50:55.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.850+0000 7f58baffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58bc13ce80 0x7f58bc13f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.850+0000 7f58baffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58bc13ce80 0x7f58bc13f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49498/0 (socket says 172.21.15.27:49498) 2023-08-11T15:50:55.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.850+0000 7f58baffd640 1 -- 172.21.15.27:0/1152595584 learned_addr learned my addr 172.21.15.27:0/1152595584 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:55.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.850+0000 7f58b3fff640 1 --2- 172.21.15.27:0/1152595584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58bc13d260 0x7f58bc13fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.850+0000 7f58bb7fe640 1 --2- 172.21.15.27:0/1152595584 >> v2:172.21.15.119:3300/0 conn(0x7f58bc0cc720 0x7f58bc0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.850+0000 7f58baffd640 1 -- 172.21.15.27:0/1152595584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58bc13d260 msgr2=0x7f58bc13fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:55.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.850+0000 7f58baffd640 1 --2- 172.21.15.27:0/1152595584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58bc13d260 0x7f58bc13fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.850+0000 7f58baffd640 1 -- 172.21.15.27:0/1152595584 >> v2:172.21.15.119:3300/0 conn(0x7f58bc0cc720 msgr2=0x7f58bc0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:55.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.850+0000 7f58baffd640 1 --2- 172.21.15.27:0/1152595584 >> v2:172.21.15.119:3300/0 conn(0x7f58bc0cc720 0x7f58bc0ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.850+0000 7f58baffd640 1 -- 172.21.15.27:0/1152595584 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc0ad7d0 con 0x7f58bc13ce80 2023-08-11T15:50:55.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.851+0000 7f58baffd640 1 --2- 172.21.15.27:0/1152595584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc13ce80 0x7f58bc13f810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0cbfb0 tx=0x7f58ac02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a5539b5d95d3f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:55.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.852+0000 7f58ba7fc640 1 -- 172.21.15.27:0/1152595584 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac031860 con 0x7f58bc13ce80 2023-08-11T15:50:55.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.852+0000 7f58ba7fc640 1 -- 172.21.15.27:0/1152595584 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac0319c0 con 0x7f58bc13ce80 2023-08-11T15:50:55.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.852+0000 7f58ba7fc640 1 -- 172.21.15.27:0/1152595584 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac031d50 con 0x7f58bc13ce80 2023-08-11T15:50:55.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.852+0000 7f58bbfff640 1 -- 172.21.15.27:0/1152595584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc13ce80 msgr2=0x7f58bc13f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.852+0000 7f58bbfff640 1 --2- 172.21.15.27:0/1152595584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc13ce80 0x7f58bc13f810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0cbfb0 tx=0x7f58ac02b900 comp rx=0 tx=0).stop 2023-08-11T15:50:55.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.852+0000 7f58bbfff640 1 -- 172.21.15.27:0/1152595584 shutdown_connections 2023-08-11T15:50:55.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.852+0000 7f58bbfff640 1 --2- 172.21.15.27:0/1152595584 >> v2:172.21.15.119:3300/0 conn(0x7f58bc0cc720 0x7f58bc0ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.852+0000 7f58bbfff640 1 --2- 172.21.15.27:0/1152595584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58bc13d260 0x7f58bc13fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.852+0000 7f58bbfff640 1 --2- 172.21.15.27:0/1152595584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc13ce80 0x7f58bc13f810 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.852+0000 7f58bbfff640 1 -- 172.21.15.27:0/1152595584 >> 172.21.15.27:0/1152595584 conn(0x7f58bc069800 msgr2=0x7f58bc069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:55.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.853+0000 7f58bbfff640 1 -- 172.21.15.27:0/1152595584 shutdown_connections 2023-08-11T15:50:55.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.853+0000 7f58bbfff640 1 -- 172.21.15.27:0/1152595584 wait complete. 2023-08-11T15:50:55.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.854+0000 7f58bbfff640 1 Processor -- start 2023-08-11T15:50:55.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.855+0000 7f58bbfff640 1 -- start start 2023-08-11T15:50:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc0cc720 0x7f58bc15a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58bc13ce80 0x7f58bc0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc13d260 0x7f58bc13b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58bbfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc15d000 con 0x7f58bc0cc720 2023-08-11T15:50:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58bbfff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0b2800 con 0x7f58bc13d260 2023-08-11T15:50:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58bbfff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc15c9b0 con 0x7f58bc13ce80 2023-08-11T15:50:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58baffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc0cc720 0x7f58bc15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58baffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc0cc720 0x7f58bc15a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49500/0 (socket says 172.21.15.27:49500) 2023-08-11T15:50:55.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58baffd640 1 -- 172.21.15.27:0/2357253811 learned_addr learned my addr 172.21.15.27:0/2357253811 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:55.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58b3fff640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58bc13ce80 0x7f58bc0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.856+0000 7f58bb7fe640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc13d260 0x7f58bc13b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.857+0000 7f58baffd640 1 -- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58bc13ce80 msgr2=0x7f58bc0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.857+0000 7f58baffd640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58bc13ce80 0x7f58bc0b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.857+0000 7f58baffd640 1 -- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc13d260 msgr2=0x7f58bc13b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:55.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.857+0000 7f58baffd640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc13d260 0x7f58bc13b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:55.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.857+0000 7f58baffd640 1 -- 172.21.15.27:0/2357253811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac0094d0 con 0x7f58bc0cc720 2023-08-11T15:50:55.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.857+0000 7f58baffd640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc0cc720 0x7f58bc15a990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f58ac02b940 tx=0x7f58ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:55.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.857+0000 7f58b8ff9640 1 -- 172.21.15.27:0/2357253811 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac03ad30 con 0x7f58bc0cc720 2023-08-11T15:50:55.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.857+0000 7f58b8ff9640 1 -- 172.21.15.27:0/2357253811 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac0443f0 con 0x7f58bc0cc720 2023-08-11T15:50:55.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.857+0000 7f58bbfff640 1 -- 172.21.15.27:0/2357253811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc15cc90 con 0x7f58bc0cc720 2023-08-11T15:50:55.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.857+0000 7f58b8ff9640 1 -- 172.21.15.27:0/2357253811 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0447a0 con 0x7f58bc0cc720 2023-08-11T15:50:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.858+0000 7f58bbfff640 1 -- 172.21.15.27:0/2357253811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc15d490 con 0x7f58bc0cc720 2023-08-11T15:50:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.859+0000 7f58bbfff640 1 -- 172.21.15.27:0/2357253811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc15ea20 con 0x7f58bc0cc720 2023-08-11T15:50:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.860+0000 7f58b8ff9640 1 -- 172.21.15.27:0/2357253811 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f58ac04d450 con 0x7f58bc0cc720 2023-08-11T15:50:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.861+0000 7f58b8ff9640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f589c06a050 0x7f589c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.861+0000 7f58b8ff9640 1 -- 172.21.15.27:0/2357253811 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f58ac0aa8f0 con 0x7f58bc0cc720 2023-08-11T15:50:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.861+0000 7f58b3fff640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f589c06a050 0x7f589c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.862+0000 7f58b3fff640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f589c06a050 0x7f589c06c510 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58a4006fd0 tx=0x7f58a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:55.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:55.865+0000 7f58b8ff9640 1 -- 172.21.15.27:0/2357253811 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f58ac076290 con 0x7f58bc0cc720 2023-08-11T15:50:56.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.061+0000 7f58bbfff640 1 -- 172.21.15.27:0/2357253811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f58bc066e80 con 0x7f58bc0cc720 2023-08-11T15:50:56.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.062+0000 7f58b8ff9640 1 -- 172.21.15.27:0/2357253811 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f58ac0350a0 con 0x7f58bc0cc720 2023-08-11T15:50:56.064 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:50:56.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 -- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f589c06a050 msgr2=0x7f589c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f589c06a050 0x7f589c06c510 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58a4006fd0 tx=0x7f58a4008040 comp rx=0 tx=0).stop 2023-08-11T15:50:56.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 -- 172.21.15.27:0/2357253811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc0cc720 msgr2=0x7f58bc15a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc0cc720 0x7f58bc15a990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f58ac02b940 tx=0x7f58ac0046c0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 -- 172.21.15.27:0/2357253811 shutdown_connections 2023-08-11T15:50:56.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f589c06a050 0x7f589c06c510 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58bc13ce80 0x7f58bc0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58bc13d260 0x7f58bc13b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 --2- 172.21.15.27:0/2357253811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58bc0cc720 0x7f58bc15a990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 -- 172.21.15.27:0/2357253811 >> 172.21.15.27:0/2357253811 conn(0x7f58bc069800 msgr2=0x7f58bc158530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:56.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 -- 172.21.15.27:0/2357253811 shutdown_connections 2023-08-11T15:50:56.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.065+0000 7f58bbfff640 1 -- 172.21.15.27:0/2357253811 wait complete. 2023-08-11T15:50:56.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:50:56.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.321+0000 7f3a1dd22640 1 Processor -- start 2023-08-11T15:50:56.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.321+0000 7f3a1dd22640 1 -- start start 2023-08-11T15:50:56.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.323+0000 7f3a1dd22640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a1806f5d0 0x7f3a1806fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.323+0000 7f3a1dd22640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a180c8a30 con 0x7f3a1815a140 2023-08-11T15:50:56.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.323+0000 7f3a1dd22640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1806c0c0 con 0x7f3a180cb430 2023-08-11T15:50:56.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.323+0000 7f3a1dd22640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1806ffa0 con 0x7f3a1806f5d0 2023-08-11T15:50:56.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.323+0000 7f3a177fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3a1815a140 0x7f3a1806b870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33178/0 (socket says 172.21.15.27:33178) 2023-08-11T15:50:56.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.323+0000 7f3a177fe640 1 -- 172.21.15.27:0/1438828569 learned_addr learned my addr 172.21.15.27:0/1438828569 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:50:56.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.323+0000 7f3a17fff640 1 --2- 172.21.15.27:0/1438828569 >> v2:172.21.15.119:3300/0 conn(0x7f3a1806f5d0 0x7f3a1806fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.324+0000 7f3a16ffd640 1 -- 172.21.15.27:0/1438828569 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368371098 0 0) 0x7f3a180c8a30 con 0x7f3a1815a140 2023-08-11T15:50:56.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.324+0000 7f3a16ffd640 1 -- 172.21.15.27:0/1438828569 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a04003520 con 0x7f3a1815a140 2023-08-11T15:50:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.324+0000 7f3a17fff640 1 -- 172.21.15.27:0/1438828569 >> v1:172.21.15.119:6789/0 conn(0x7f3a180cb430 legacy=0x7f3a180b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.324+0000 7f3a16ffd640 1 -- 172.21.15.27:0/1438828569 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2664034081 0 0) 0x7f3a04003520 con 0x7f3a1815a140 2023-08-11T15:50:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.324+0000 7f3a17fff640 1 -- 172.21.15.27:0/1438828569 >> v1:172.21.15.27:6789/0 conn(0x7f3a1815a140 legacy=0x7f3a1806b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.324+0000 7f3a17fff640 1 -- 172.21.15.27:0/1438828569 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1806c3a0 con 0x7f3a1806f5d0 2023-08-11T15:50:56.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.325+0000 7f3a17fff640 1 --2- 172.21.15.27:0/1438828569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1806f5d0 0x7f3a1806fa60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009d90 tx=0x7f3a1002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f810bfafc6f24977 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:56.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.326+0000 7f3a16ffd640 1 -- 172.21.15.27:0/1438828569 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a080017f0 con 0x7f3a1806f5d0 2023-08-11T15:50:56.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.326+0000 7f3a16ffd640 1 -- 172.21.15.27:0/1438828569 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1002c940 con 0x7f3a1806f5d0 2023-08-11T15:50:56.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.326+0000 7f3a16ffd640 1 -- 172.21.15.27:0/1438828569 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1002ccd0 con 0x7f3a1806f5d0 2023-08-11T15:50:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.327+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1438828569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1806f5d0 msgr2=0x7f3a1806fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.327+0000 7f3a1dd22640 1 --2- 172.21.15.27:0/1438828569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1806f5d0 0x7f3a1806fa60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009d90 tx=0x7f3a1002be10 comp rx=0 tx=0).stop 2023-08-11T15:50:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.327+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1438828569 shutdown_connections 2023-08-11T15:50:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.327+0000 7f3a1dd22640 1 --2- 172.21.15.27:0/1438828569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1806f5d0 0x7f3a1806fa60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.327+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1438828569 >> 172.21.15.27:0/1438828569 conn(0x7f3a180afb70 msgr2=0x7f3a180aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:56.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.327+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1438828569 shutdown_connections 2023-08-11T15:50:56.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.327+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1438828569 wait complete. 2023-08-11T15:50:56.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.329+0000 7f3a1dd22640 1 Processor -- start 2023-08-11T15:50:56.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.329+0000 7f3a1dd22640 1 -- start start 2023-08-11T15:50:56.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a1dd22640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1806f5d0 0x7f3a180cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a1dd22640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3a180cb430 0x7f3a181642d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a1dd22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1815a140 0x7f3a180c3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a1dd22640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a18158760 con 0x7f3a1815a140 2023-08-11T15:50:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a1dd22640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a18158a40 con 0x7f3a1806f5d0 2023-08-11T15:50:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a1dd22640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a18158d20 con 0x7f3a180cb430 2023-08-11T15:50:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a17fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1815a140 0x7f3a180c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a17fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1815a140 0x7f3a180c3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49512/0 (socket says 172.21.15.27:49512) 2023-08-11T15:50:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a17fff640 1 -- 172.21.15.27:0/1995080312 learned_addr learned my addr 172.21.15.27:0/1995080312 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a177fe640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1806f5d0 0x7f3a180cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.330+0000 7f3a0effd640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3a180cb430 0x7f3a181642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a17fff640 1 -- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3a180cb430 msgr2=0x7f3a181642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a17fff640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3a180cb430 0x7f3a181642d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a17fff640 1 -- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1806f5d0 msgr2=0x7f3a180cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a17fff640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1806f5d0 0x7f3a180cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a17fff640 1 -- 172.21.15.27:0/1995080312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009a70 con 0x7f3a1815a140 2023-08-11T15:50:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a177fe640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1806f5d0 0x7f3a180cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:50:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a17fff640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1815a140 0x7f3a180c3810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3a10037040 tx=0x7f3a10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a14ff9640 1 -- 172.21.15.27:0/1995080312 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10009450 con 0x7f3a1815a140 2023-08-11T15:50:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a14ff9640 1 -- 172.21.15.27:0/1995080312 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a10003990 con 0x7f3a1815a140 2023-08-11T15:50:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a14ff9640 1 -- 172.21.15.27:0/1995080312 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10003d20 con 0x7f3a1815a140 2023-08-11T15:50:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.331+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1995080312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a18159000 con 0x7f3a1815a140 2023-08-11T15:50:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.332+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1995080312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a180cd650 con 0x7f3a1815a140 2023-08-11T15:50:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.333+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1995080312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a18142e80 con 0x7f3a1815a140 2023-08-11T15:50:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.334+0000 7f3a14ff9640 1 -- 172.21.15.27:0/1995080312 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3a1003b050 con 0x7f3a1815a140 2023-08-11T15:50:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.335+0000 7f3a14ff9640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f39f806a050 0x7f39f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.335+0000 7f3a14ff9640 1 -- 172.21.15.27:0/1995080312 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3a100aa880 con 0x7f3a1815a140 2023-08-11T15:50:56.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.335+0000 7f3a177fe640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f39f806a050 0x7f39f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.337+0000 7f3a177fe640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f39f806a050 0x7f39f806c510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3a08001360 tx=0x7f3a08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.339+0000 7f3a14ff9640 1 -- 172.21.15.27:0/1995080312 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f3a18142e80 con 0x7f3a1815a140 2023-08-11T15:50:56.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.572+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1995080312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a18142e80 con 0x7f3a1815a140 2023-08-11T15:50:56.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.573+0000 7f3a14ff9640 1 -- 172.21.15.27:0/1995080312 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f3a1004de90 con 0x7f3a1815a140 2023-08-11T15:50:56.575 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:50:56.575 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-11T15:50:56.575 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2023-08-11T15:50:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f39f806a050 msgr2=0x7f39f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f39f806a050 0x7f39f806c510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3a08001360 tx=0x7f3a08009040 comp rx=0 tx=0).stop 2023-08-11T15:50:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1995080312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1815a140 msgr2=0x7f3a180c3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1815a140 0x7f3a180c3810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3a10037040 tx=0x7f3a10004a40 comp rx=0 tx=0).stop 2023-08-11T15:50:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1995080312 shutdown_connections 2023-08-11T15:50:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f39f806a050 0x7f39f806c510 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3a180cb430 0x7f3a181642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1815a140 0x7f3a180c3810 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 --2- 172.21.15.27:0/1995080312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1806f5d0 0x7f3a180cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1995080312 >> 172.21.15.27:0/1995080312 conn(0x7f3a180afb70 msgr2=0x7f3a18165710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:56.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1995080312 shutdown_connections 2023-08-11T15:50:56.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.576+0000 7f3a1dd22640 1 -- 172.21.15.27:0/1995080312 wait complete. 2023-08-11T15:50:56.597 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:50:56.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.832+0000 7fa9e228e640 1 Processor -- start 2023-08-11T15:50:56.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.833+0000 7fa9e228e640 1 -- start start 2023-08-11T15:50:56.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.834+0000 7fa9e228e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc0b55c0 0x7fa9dc0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.834+0000 7fa9e228e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa9dc0cf930 0x7fa9dc0d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.834+0000 7fa9e228e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9dc0cda30 con 0x7fa9dc15c620 2023-08-11T15:50:56.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.834+0000 7fa9e228e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9dc0ccc10 con 0x7fa9dc0cf930 2023-08-11T15:50:56.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.834+0000 7fa9e228e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0cfda0 con 0x7fa9dc0b55c0 2023-08-11T15:50:56.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.834+0000 7fa9dbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa9dc15c620 0x7fa9dc0b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33190/0 (socket says 172.21.15.27:33190) 2023-08-11T15:50:56.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.834+0000 7fa9dbfff640 1 -- 172.21.15.27:0/1217610873 learned_addr learned my addr 172.21.15.27:0/1217610873 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:50:56.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.834+0000 7fa9e082b640 1 --2- 172.21.15.27:0/1217610873 >> v2:172.21.15.27:3300/0 conn(0x7fa9dc0cf930 0x7fa9dc0d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.835+0000 7fa9db7fe640 1 --2- 172.21.15.27:0/1217610873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc0b55c0 0x7fa9dc0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.835+0000 7fa9daffd640 1 -- 172.21.15.27:0/1217610873 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358360785 0 0) 0x7fa9dc0cda30 con 0x7fa9dc15c620 2023-08-11T15:50:56.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.835+0000 7fa9daffd640 1 -- 172.21.15.27:0/1217610873 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c4003520 con 0x7fa9dc15c620 2023-08-11T15:50:56.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.835+0000 7fa9e082b640 1 -- 172.21.15.27:0/1217610873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc0b55c0 msgr2=0x7fa9dc0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:56.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.835+0000 7fa9e082b640 1 --2- 172.21.15.27:0/1217610873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc0b55c0 0x7fa9dc0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.835+0000 7fa9e082b640 1 -- 172.21.15.27:0/1217610873 >> v1:172.21.15.27:6789/0 conn(0x7fa9dc15c620 legacy=0x7fa9dc0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.835+0000 7fa9e082b640 1 -- 172.21.15.27:0/1217610873 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc161110 con 0x7fa9dc0cf930 2023-08-11T15:50:56.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.835+0000 7fa9db7fe640 1 --2- 172.21.15.27:0/1217610873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc0b55c0 0x7fa9dc0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:56.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.836+0000 7fa9e082b640 1 --2- 172.21.15.27:0/1217610873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 0x7fa9dc0d0b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097a0 tx=0x7fa9d402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1954d195ba592e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:56.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.836+0000 7fa9daffd640 1 -- 172.21.15.27:0/1217610873 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d402d040 con 0x7fa9dc0cf930 2023-08-11T15:50:56.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.836+0000 7fa9daffd640 1 -- 172.21.15.27:0/1217610873 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4031810 con 0x7fa9dc0cf930 2023-08-11T15:50:56.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.836+0000 7fa9daffd640 1 -- 172.21.15.27:0/1217610873 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4031b80 con 0x7fa9dc0cf930 2023-08-11T15:50:56.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.837+0000 7fa9e228e640 1 -- 172.21.15.27:0/1217610873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 msgr2=0x7fa9dc0d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.837+0000 7fa9e228e640 1 --2- 172.21.15.27:0/1217610873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 0x7fa9dc0d0b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097a0 tx=0x7fa9d402b8b0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.837+0000 7fa9e228e640 1 -- 172.21.15.27:0/1217610873 shutdown_connections 2023-08-11T15:50:56.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.837+0000 7fa9e228e640 1 --2- 172.21.15.27:0/1217610873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 0x7fa9dc0d0b00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.837+0000 7fa9e228e640 1 --2- 172.21.15.27:0/1217610873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc0b55c0 0x7fa9dc0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.837+0000 7fa9e228e640 1 -- 172.21.15.27:0/1217610873 >> 172.21.15.27:0/1217610873 conn(0x7fa9dc071110 msgr2=0x7fa9dc0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:56.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.837+0000 7fa9e228e640 1 -- 172.21.15.27:0/1217610873 shutdown_connections 2023-08-11T15:50:56.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.837+0000 7fa9e228e640 1 -- 172.21.15.27:0/1217610873 wait complete. 2023-08-11T15:50:56.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.839+0000 7fa9e228e640 1 Processor -- start 2023-08-11T15:50:56.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.839+0000 7fa9e228e640 1 -- start start 2023-08-11T15:50:56.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.840+0000 7fa9e228e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc0b55c0 0x7fa9dc14e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.840+0000 7fa9e228e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 0x7fa9dc0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.840+0000 7fa9e228e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc15c620 0x7fa9dc0aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.840+0000 7fa9e228e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0a9ca0 con 0x7fa9dc0cf930 2023-08-11T15:50:56.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.840+0000 7fa9e228e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0a9f80 con 0x7fa9dc0b55c0 2023-08-11T15:50:56.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.840+0000 7fa9e228e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0aa260 con 0x7fa9dc15c620 2023-08-11T15:50:56.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.840+0000 7fa9db7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 0x7fa9dc0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.841+0000 7fa9db7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 0x7fa9dc0cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49524/0 (socket says 172.21.15.27:49524) 2023-08-11T15:50:56.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.841+0000 7fa9db7fe640 1 -- 172.21.15.27:0/908098201 learned_addr learned my addr 172.21.15.27:0/908098201 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:56.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.841+0000 7fa9dbfff640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc0b55c0 0x7fa9dc14e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.841+0000 7fa9e082b640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc15c620 0x7fa9dc0aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.841+0000 7fa9db7fe640 1 -- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc15c620 msgr2=0x7fa9dc0aaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.841+0000 7fa9db7fe640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc15c620 0x7fa9dc0aaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.841+0000 7fa9db7fe640 1 -- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc0b55c0 msgr2=0x7fa9dc14e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:56.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.841+0000 7fa9db7fe640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc0b55c0 0x7fa9dc14e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:56.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.841+0000 7fa9db7fe640 1 -- 172.21.15.27:0/908098201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4009400 con 0x7fa9dc0cf930 2023-08-11T15:50:56.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.841+0000 7fa9db7fe640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 0x7fa9dc0cc1d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000cf60 tx=0x7fa9d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:56.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.842+0000 7fa9d8ff9640 1 -- 172.21.15.27:0/908098201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000d840 con 0x7fa9dc0cf930 2023-08-11T15:50:56.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.842+0000 7fa9d8ff9640 1 -- 172.21.15.27:0/908098201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d000d9a0 con 0x7fa9dc0cf930 2023-08-11T15:50:56.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.842+0000 7fa9e228e640 1 -- 172.21.15.27:0/908098201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc0c9730 con 0x7fa9dc0cf930 2023-08-11T15:50:56.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.842+0000 7fa9d8ff9640 1 -- 172.21.15.27:0/908098201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000dd30 con 0x7fa9dc0cf930 2023-08-11T15:50:56.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.842+0000 7fa9e228e640 1 -- 172.21.15.27:0/908098201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc0cc710 con 0x7fa9dc0cf930 2023-08-11T15:50:56.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.843+0000 7fa9e228e640 1 -- 172.21.15.27:0/908098201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9dc0c9920 con 0x7fa9dc0cf930 2023-08-11T15:50:56.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.844+0000 7fa9d8ff9640 1 -- 172.21.15.27:0/908098201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa9d0015440 con 0x7fa9dc0cf930 2023-08-11T15:50:56.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.845+0000 7fa9d8ff9640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa9c806a050 0x7fa9c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:56.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.845+0000 7fa9d8ff9640 1 -- 172.21.15.27:0/908098201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa9d0088bd0 con 0x7fa9dc0cf930 2023-08-11T15:50:56.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.845+0000 7fa9dbfff640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa9c806a050 0x7fa9c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:56.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.847+0000 7fa9dbfff640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa9c806a050 0x7fa9c806c510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc005880 tx=0x7fa9cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:56.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:56.849+0000 7fa9d8ff9640 1 -- 172.21.15.27:0/908098201 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fa9d00546b0 con 0x7fa9dc0cf930 2023-08-11T15:50:57.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.045+0000 7fa9e228e640 1 -- 172.21.15.27:0/908098201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9dc0a9de0 con 0x7fa9dc0cf930 2023-08-11T15:50:57.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.047+0000 7fa9d8ff9640 1 -- 172.21.15.27:0/908098201 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+14636 (secure 0 0 0) 0x7fa9dc0a9de0 con 0x7fa9dc0cf930 2023-08-11T15:50:57.049 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:50:57.049 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:50:55.656407+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:50:57.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 -- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa9c806a050 msgr2=0x7fa9c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa9c806a050 0x7fa9c806c510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc005880 tx=0x7fa9cc009040 comp rx=0 tx=0).stop 2023-08-11T15:50:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 -- 172.21.15.27:0/908098201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 msgr2=0x7fa9dc0cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 0x7fa9dc0cc1d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000cf60 tx=0x7fa9d000cf90 comp rx=0 tx=0).stop 2023-08-11T15:50:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 -- 172.21.15.27:0/908098201 shutdown_connections 2023-08-11T15:50:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa9c806a050 0x7fa9c806c510 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9dc15c620 0x7fa9dc0aaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9dc0cf930 0x7fa9dc0cc1d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 --2- 172.21.15.27:0/908098201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc0b55c0 0x7fa9dc14e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 -- 172.21.15.27:0/908098201 >> 172.21.15.27:0/908098201 conn(0x7fa9dc071110 msgr2=0x7fa9dc15bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:57.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.050+0000 7fa9e228e640 1 -- 172.21.15.27:0/908098201 shutdown_connections 2023-08-11T15:50:57.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.051+0000 7fa9e228e640 1 -- 172.21.15.27:0/908098201 wait complete. 2023-08-11T15:50:57.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T15:50:57.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:50:57.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.310+0000 7fee5e14f640 1 Processor -- start 2023-08-11T15:50:57.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.310+0000 7fee5e14f640 1 -- start start 2023-08-11T15:50:57.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.312+0000 7fee5e14f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee580cb700 0x7fee5813fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.312+0000 7fee5e14f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee58140ba0 con 0x7fee5815b6b0 2023-08-11T15:50:57.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.312+0000 7fee5e14f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5815b220 con 0x7fee58154ac0 2023-08-11T15:50:57.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.312+0000 7fee5e14f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee58071b10 con 0x7fee580cb700 2023-08-11T15:50:57.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.312+0000 7fee56ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee580cb700 0x7fee5813fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:57.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.312+0000 7fee56ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee580cb700 0x7fee5813fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49528/0 (socket says 172.21.15.27:49528) 2023-08-11T15:50:57.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.312+0000 7fee57fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fee5815b6b0 0x7fee580b2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33194/0 (socket says 172.21.15.27:33194) 2023-08-11T15:50:57.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.312+0000 7fee56ffd640 1 -- 172.21.15.27:0/3345075413 learned_addr learned my addr 172.21.15.27:0/3345075413 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:57.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.313+0000 7fee567fc640 1 -- 172.21.15.27:0/3345075413 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140503963 0 0) 0x7fee58140ba0 con 0x7fee5815b6b0 2023-08-11T15:50:57.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.313+0000 7fee56ffd640 1 -- 172.21.15.27:0/3345075413 >> v1:172.21.15.119:6789/0 conn(0x7fee58154ac0 legacy=0x7fee580caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.313+0000 7fee56ffd640 1 -- 172.21.15.27:0/3345075413 >> v1:172.21.15.27:6789/0 conn(0x7fee5815b6b0 legacy=0x7fee580b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.313+0000 7fee56ffd640 1 -- 172.21.15.27:0/3345075413 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee581408c0 con 0x7fee580cb700 2023-08-11T15:50:57.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.314+0000 7fee56ffd640 1 --2- 172.21.15.27:0/3345075413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee580cb700 0x7fee5813fc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009870 tx=0x7fee4c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9e65fc67fa118c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:57.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.314+0000 7fee567fc640 1 -- 172.21.15.27:0/3345075413 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c030860 con 0x7fee580cb700 2023-08-11T15:50:57.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.314+0000 7fee567fc640 1 -- 172.21.15.27:0/3345075413 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c0309c0 con 0x7fee580cb700 2023-08-11T15:50:57.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.314+0000 7fee567fc640 1 -- 172.21.15.27:0/3345075413 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c030d50 con 0x7fee580cb700 2023-08-11T15:50:57.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.315+0000 7fee5e14f640 1 -- 172.21.15.27:0/3345075413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee580cb700 msgr2=0x7fee5813fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.315+0000 7fee5e14f640 1 --2- 172.21.15.27:0/3345075413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee580cb700 0x7fee5813fc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009870 tx=0x7fee4c02b9a0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.315+0000 7fee5e14f640 1 -- 172.21.15.27:0/3345075413 shutdown_connections 2023-08-11T15:50:57.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.315+0000 7fee5e14f640 1 --2- 172.21.15.27:0/3345075413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee580cb700 0x7fee5813fc90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.315+0000 7fee5e14f640 1 -- 172.21.15.27:0/3345075413 >> 172.21.15.27:0/3345075413 conn(0x7fee58069bf0 msgr2=0x7fee5806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:57.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.315+0000 7fee5e14f640 1 -- 172.21.15.27:0/3345075413 shutdown_connections 2023-08-11T15:50:57.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.315+0000 7fee5e14f640 1 -- 172.21.15.27:0/3345075413 wait complete. 2023-08-11T15:50:57.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.317+0000 7fee5e14f640 1 Processor -- start 2023-08-11T15:50:57.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.317+0000 7fee5e14f640 1 -- start start 2023-08-11T15:50:57.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.318+0000 7fee5e14f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee580cb700 0x7fee581f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.318+0000 7fee5e14f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58154ac0 0x7fee580b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.318+0000 7fee5e14f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee5815b6b0 0x7fee580b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.318+0000 7fee5e14f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee580b1a20 con 0x7fee58154ac0 2023-08-11T15:50:57.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.318+0000 7fee5e14f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee580b0a10 con 0x7fee580cb700 2023-08-11T15:50:57.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.318+0000 7fee5e14f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee580b0e10 con 0x7fee5815b6b0 2023-08-11T15:50:57.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.319+0000 7fee56ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58154ac0 0x7fee580b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:57.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.319+0000 7fee56ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58154ac0 0x7fee580b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49532/0 (socket says 172.21.15.27:49532) 2023-08-11T15:50:57.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.319+0000 7fee56ffd640 1 -- 172.21.15.27:0/4021096577 learned_addr learned my addr 172.21.15.27:0/4021096577 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:57.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.319+0000 7fee577fe640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee580cb700 0x7fee581f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:57.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.319+0000 7fee57fff640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee5815b6b0 0x7fee580b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:57.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.319+0000 7fee56ffd640 1 -- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee5815b6b0 msgr2=0x7fee580b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.319+0000 7fee56ffd640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee5815b6b0 0x7fee580b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.319+0000 7fee56ffd640 1 -- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee580cb700 msgr2=0x7fee581f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.319+0000 7fee56ffd640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee580cb700 0x7fee581f1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.319+0000 7fee56ffd640 1 -- 172.21.15.27:0/4021096577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0094d0 con 0x7fee58154ac0 2023-08-11T15:50:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.320+0000 7fee56ffd640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58154ac0 0x7fee580b04d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009870 tx=0x7fee4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.320+0000 7fee3ffff640 1 -- 172.21.15.27:0/4021096577 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c039d60 con 0x7fee58154ac0 2023-08-11T15:50:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.320+0000 7fee3ffff640 1 -- 172.21.15.27:0/4021096577 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c043400 con 0x7fee58154ac0 2023-08-11T15:50:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.320+0000 7fee3ffff640 1 -- 172.21.15.27:0/4021096577 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0437b0 con 0x7fee58154ac0 2023-08-11T15:50:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.320+0000 7fee5e14f640 1 -- 172.21.15.27:0/4021096577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee580b10f0 con 0x7fee58154ac0 2023-08-11T15:50:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.320+0000 7fee5e14f640 1 -- 172.21.15.27:0/4021096577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee580b16b0 con 0x7fee58154ac0 2023-08-11T15:50:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.322+0000 7fee5e14f640 1 -- 172.21.15.27:0/4021096577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5804e420 con 0x7fee58154ac0 2023-08-11T15:50:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.322+0000 7fee3ffff640 1 -- 172.21.15.27:0/4021096577 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fee4c043910 con 0x7fee58154ac0 2023-08-11T15:50:57.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.323+0000 7fee3ffff640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee4406a0a0 0x7fee4406c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.323+0000 7fee3ffff640 1 -- 172.21.15.27:0/4021096577 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fee4c03e070 con 0x7fee58154ac0 2023-08-11T15:50:57.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.324+0000 7fee577fe640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee4406a0a0 0x7fee4406c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:57.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.325+0000 7fee577fe640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee4406a0a0 0x7fee4406c560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fee48000f60 tx=0x7fee48009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:57.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.327+0000 7fee3ffff640 1 -- 172.21.15.27:0/4021096577 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fee4c077010 con 0x7fee58154ac0 2023-08-11T15:50:57.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.520+0000 7fee5e14f640 1 -- 172.21.15.27:0/4021096577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee580af8a0 con 0x7fee58154ac0 2023-08-11T15:50:57.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.521+0000 7fee3ffff640 1 -- 172.21.15.27:0/4021096577 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee4c07a630 con 0x7fee58154ac0 2023-08-11T15:50:57.522 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:50:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.523+0000 7fee5e14f640 1 -- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee4406a0a0 msgr2=0x7fee4406c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.523+0000 7fee5e14f640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee4406a0a0 0x7fee4406c560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fee48000f60 tx=0x7fee48009040 comp rx=0 tx=0).stop 2023-08-11T15:50:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.523+0000 7fee5e14f640 1 -- 172.21.15.27:0/4021096577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58154ac0 msgr2=0x7fee580b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.523+0000 7fee5e14f640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58154ac0 0x7fee580b04d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009870 tx=0x7fee4c004730 comp rx=0 tx=0).stop 2023-08-11T15:50:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.523+0000 7fee5e14f640 1 -- 172.21.15.27:0/4021096577 shutdown_connections 2023-08-11T15:50:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.523+0000 7fee5e14f640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee4406a0a0 0x7fee4406c560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.523+0000 7fee5e14f640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee5815b6b0 0x7fee580b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.523+0000 7fee5e14f640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58154ac0 0x7fee580b04d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.523+0000 7fee5e14f640 1 --2- 172.21.15.27:0/4021096577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee580cb700 0x7fee581f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.523+0000 7fee5e14f640 1 -- 172.21.15.27:0/4021096577 >> 172.21.15.27:0/4021096577 conn(0x7fee58069bf0 msgr2=0x7fee580c9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.524+0000 7fee5e14f640 1 -- 172.21.15.27:0/4021096577 shutdown_connections 2023-08-11T15:50:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.524+0000 7fee5e14f640 1 -- 172.21.15.27:0/4021096577 wait complete. 2023-08-11T15:50:57.543 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:50:57.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.782+0000 7fe144501640 1 Processor -- start 2023-08-11T15:50:57.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.782+0000 7fe144501640 1 -- start start 2023-08-11T15:50:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.784+0000 7fe144501640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe13c154ac0 0x7fe13c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.784+0000 7fe144501640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe13c1403a0 0x7fe13c0cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.784+0000 7fe144501640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c140ba0 con 0x7fe13c15b6b0 2023-08-11T15:50:57.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.784+0000 7fe144501640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13c071b10 con 0x7fe13c1403a0 2023-08-11T15:50:57.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.784+0000 7fe144501640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13c15b220 con 0x7fe13c154ac0 2023-08-11T15:50:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.784+0000 7fe142a9e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe13c1403a0 0x7fe13c0cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.785+0000 7fe141a9c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe13c15b6b0 0x7fe13c13fc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33202/0 (socket says 172.21.15.27:33202) 2023-08-11T15:50:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.785+0000 7fe141a9c640 1 -- 172.21.15.27:0/3614337129 learned_addr learned my addr 172.21.15.27:0/3614337129 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:50:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.785+0000 7fe14229d640 1 --2- 172.21.15.27:0/3614337129 >> v2:172.21.15.119:3300/0 conn(0x7fe13c154ac0 0x7fe13c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.785+0000 7fe142a9e640 1 -- 172.21.15.27:0/3614337129 >> v2:172.21.15.119:3300/0 conn(0x7fe13c154ac0 msgr2=0x7fe13c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.785+0000 7fe142a9e640 1 --2- 172.21.15.27:0/3614337129 >> v2:172.21.15.119:3300/0 conn(0x7fe13c154ac0 0x7fe13c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.785+0000 7fe142a9e640 1 -- 172.21.15.27:0/3614337129 >> v1:172.21.15.27:6789/0 conn(0x7fe13c15b6b0 legacy=0x7fe13c13fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.785+0000 7fe142a9e640 1 -- 172.21.15.27:0/3614337129 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c1408c0 con 0x7fe13c1403a0 2023-08-11T15:50:57.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.785+0000 7fe14129b640 1 -- 172.21.15.27:0/3614337129 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149855782 0 0) 0x7fe13c140ba0 con 0x7fe13c15b6b0 2023-08-11T15:50:57.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.785+0000 7fe14229d640 1 --2- 172.21.15.27:0/3614337129 >> v2:172.21.15.119:3300/0 conn(0x7fe13c154ac0 0x7fe13c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:50:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.786+0000 7fe142a9e640 1 --2- 172.21.15.27:0/3614337129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 0x7fe13c0cb530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0cbb90 tx=0x7fe13402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650a03e054e26ea2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe14129b640 1 -- 172.21.15.27:0/3614337129 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13402cbe0 con 0x7fe13c1403a0 2023-08-11T15:50:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe14129b640 1 -- 172.21.15.27:0/3614337129 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13402cd40 con 0x7fe13c1403a0 2023-08-11T15:50:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe14129b640 1 -- 172.21.15.27:0/3614337129 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134034660 con 0x7fe13c1403a0 2023-08-11T15:50:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe144501640 1 -- 172.21.15.27:0/3614337129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 msgr2=0x7fe13c0cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe144501640 1 --2- 172.21.15.27:0/3614337129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 0x7fe13c0cb530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0cbb90 tx=0x7fe13402be30 comp rx=0 tx=0).stop 2023-08-11T15:50:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe144501640 1 -- 172.21.15.27:0/3614337129 shutdown_connections 2023-08-11T15:50:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe144501640 1 --2- 172.21.15.27:0/3614337129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 0x7fe13c0cb530 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe144501640 1 --2- 172.21.15.27:0/3614337129 >> v2:172.21.15.119:3300/0 conn(0x7fe13c154ac0 0x7fe13c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe144501640 1 -- 172.21.15.27:0/3614337129 >> 172.21.15.27:0/3614337129 conn(0x7fe13c069bf0 msgr2=0x7fe13c06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe144501640 1 -- 172.21.15.27:0/3614337129 shutdown_connections 2023-08-11T15:50:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.787+0000 7fe144501640 1 -- 172.21.15.27:0/3614337129 wait complete. 2023-08-11T15:50:57.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.789+0000 7fe144501640 1 Processor -- start 2023-08-11T15:50:57.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.789+0000 7fe144501640 1 -- start start 2023-08-11T15:50:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.790+0000 7fe144501640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 0x7fe13c0adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.790+0000 7fe144501640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe13c154ac0 0x7fe13c141760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.790+0000 7fe144501640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe13c15b6b0 0x7fe13c1f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.790+0000 7fe144501640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c142ce0 con 0x7fe13c1403a0 2023-08-11T15:50:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.790+0000 7fe144501640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c1420d0 con 0x7fe13c15b6b0 2023-08-11T15:50:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.790+0000 7fe144501640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c1423b0 con 0x7fe13c154ac0 2023-08-11T15:50:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.791+0000 7fe14229d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 0x7fe13c0adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.791+0000 7fe14229d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 0x7fe13c0adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49552/0 (socket says 172.21.15.27:49552) 2023-08-11T15:50:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.791+0000 7fe14229d640 1 -- 172.21.15.27:0/3463247759 learned_addr learned my addr 172.21.15.27:0/3463247759 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.791+0000 7fe14229d640 1 -- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe13c154ac0 msgr2=0x7fe13c141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.791+0000 7fe14229d640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe13c154ac0 0x7fe13c141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.791+0000 7fe14229d640 1 -- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe13c15b6b0 msgr2=0x7fe13c1f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.791+0000 7fe142a9e640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe13c15b6b0 0x7fe13c1f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.791+0000 7fe14229d640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe13c15b6b0 0x7fe13c1f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.791+0000 7fe14229d640 1 -- 172.21.15.27:0/3463247759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe134009a70 con 0x7fe13c1403a0 2023-08-11T15:50:57.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.791+0000 7fe14229d640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 0x7fe13c0adab0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe13800e7e0 tx=0x7fe13800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:57.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.792+0000 7fe132ffd640 1 -- 172.21.15.27:0/3463247759 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138004030 con 0x7fe13c1403a0 2023-08-11T15:50:57.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.792+0000 7fe132ffd640 1 -- 172.21.15.27:0/3463247759 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138004190 con 0x7fe13c1403a0 2023-08-11T15:50:57.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.792+0000 7fe132ffd640 1 -- 172.21.15.27:0/3463247759 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138004520 con 0x7fe13c1403a0 2023-08-11T15:50:57.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.792+0000 7fe144501640 1 -- 172.21.15.27:0/3463247759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c142690 con 0x7fe13c1403a0 2023-08-11T15:50:57.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.792+0000 7fe144501640 1 -- 172.21.15.27:0/3463247759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c0adff0 con 0x7fe13c1403a0 2023-08-11T15:50:57.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.793+0000 7fe144501640 1 -- 172.21.15.27:0/3463247759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c04e420 con 0x7fe13c1403a0 2023-08-11T15:50:57.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.794+0000 7fe132ffd640 1 -- 172.21.15.27:0/3463247759 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe13800c6d0 con 0x7fe13c1403a0 2023-08-11T15:50:57.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.795+0000 7fe132ffd640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe12006a050 0x7fe12006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:57.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.795+0000 7fe132ffd640 1 -- 172.21.15.27:0/3463247759 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe138087fe0 con 0x7fe13c1403a0 2023-08-11T15:50:57.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.796+0000 7fe141a9c640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe12006a050 0x7fe12006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:57.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.797+0000 7fe141a9c640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe12006a050 0x7fe12006c510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe12c007fd0 tx=0x7fe12c007960 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:57.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.799+0000 7fe132ffd640 1 -- 172.21.15.27:0/3463247759 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe1380539a0 con 0x7fe13c1403a0 2023-08-11T15:50:57.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.995+0000 7fe144501640 1 -- 172.21.15.27:0/3463247759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe13c06c020 con 0x7fe13c1403a0 2023-08-11T15:50:57.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.996+0000 7fe132ffd640 1 -- 172.21.15.27:0/3463247759 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe138056fc0 con 0x7fe13c1403a0 2023-08-11T15:50:57.998 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:50:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 -- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe12006a050 msgr2=0x7fe12006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe12006a050 0x7fe12006c510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe12c007fd0 tx=0x7fe12c007960 comp rx=0 tx=0).stop 2023-08-11T15:50:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 -- 172.21.15.27:0/3463247759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 msgr2=0x7fe13c0adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 0x7fe13c0adab0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe13800e7e0 tx=0x7fe13800eca0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 -- 172.21.15.27:0/3463247759 shutdown_connections 2023-08-11T15:50:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe12006a050 0x7fe12006c510 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe13c154ac0 0x7fe13c141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe13c15b6b0 0x7fe13c1f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 --2- 172.21.15.27:0/3463247759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe13c1403a0 0x7fe13c0adab0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 -- 172.21.15.27:0/3463247759 >> 172.21.15.27:0/3463247759 conn(0x7fe13c069bf0 msgr2=0x7fe13c0ace30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 -- 172.21.15.27:0/3463247759 shutdown_connections 2023-08-11T15:50:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:57.999+0000 7fe144501640 1 -- 172.21.15.27:0/3463247759 wait complete. 2023-08-11T15:50:58.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-11T15:50:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.256+0000 7f2ffed24640 1 Processor -- start 2023-08-11T15:50:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.256+0000 7f2ffed24640 1 -- start start 2023-08-11T15:50:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.257+0000 7f2ffed24640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2ff806fdf0 0x7f2ff815df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.258+0000 7f2ffed24640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80cb520 con 0x7f2ff815a3b0 2023-08-11T15:50:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.258+0000 7f2ffed24640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff815a790 con 0x7f2ff8071690 2023-08-11T15:50:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.258+0000 7f2ffed24640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff815eea0 con 0x7f2ff806fdf0 2023-08-11T15:50:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.258+0000 7f2ffcac0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2ff815a3b0 0x7f2ff80b0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33210/0 (socket says 172.21.15.27:33210) 2023-08-11T15:50:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.258+0000 7f2ffcac0640 1 -- 172.21.15.27:0/3690178097 learned_addr learned my addr 172.21.15.27:0/3690178097 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:50:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.258+0000 7f2ffd2c1640 1 --2- 172.21.15.27:0/3690178097 >> v2:172.21.15.119:3300/0 conn(0x7f2ff806fdf0 0x7f2ff815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.258+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084698031 0 0) 0x7f2ff80cb520 con 0x7f2ff815a3b0 2023-08-11T15:50:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe8003520 con 0x7f2ff815a3b0 2023-08-11T15:50:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3112606208 0 0) 0x7f2fe8003520 con 0x7f2ff815a3b0 2023-08-11T15:50:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 >> v2:172.21.15.119:3300/0 conn(0x7f2ff806fdf0 msgr2=0x7f2ff815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 --2- 172.21.15.27:0/3690178097 >> v2:172.21.15.119:3300/0 conn(0x7f2ff806fdf0 0x7f2ff815df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 >> v1:172.21.15.119:6789/0 conn(0x7f2ff8071690 legacy=0x7f2ff806f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff815e600 con 0x7f2ff815a3b0 2023-08-11T15:50:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f2fec001670 con 0x7f2ff815a3b0 2023-08-11T15:50:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 >> v1:172.21.15.27:6789/0 conn(0x7f2ff815a3b0 legacy=0x7f2ff80b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8003970 0x7f2fe8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2fe8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2fe8005180 0x7f2fe800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff815eea0 con 0x7f2fe80047c0 2023-08-11T15:50:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe800c020 con 0x7f2fe8003970 2023-08-11T15:50:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe800c1e0 con 0x7f2fe8005180 2023-08-11T15:50:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.259+0000 7f2ff7fff640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2fe8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.260+0000 7f2ffcac0640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8003970 0x7f2fe8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.260+0000 7f2ff7fff640 1 -- 172.21.15.27:0/3690178097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2fe8005180 msgr2=0x7f2fe800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.260+0000 7f2ff7fff640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2fe8005180 0x7f2fe800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.260+0000 7f2ff7fff640 1 -- 172.21.15.27:0/3690178097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8003970 msgr2=0x7f2fe8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.260+0000 7f2ff7fff640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8003970 0x7f2fe8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.260+0000 7f2ff7fff640 1 -- 172.21.15.27:0/3690178097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe8003520 con 0x7f2fe80047c0 2023-08-11T15:50:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.260+0000 7f2ffd2c1640 1 -- 172.21.15.27:0/3690178097 reap_dead start 2023-08-11T15:50:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.260+0000 7f2ffcac0640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8003970 0x7f2fe8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.261+0000 7f2ff7fff640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2fe8004c40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4001750 tx=0x7f2fe80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e94c5a0f8587e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.261+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40052d0 con 0x7f2fe80047c0 2023-08-11T15:50:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.261+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe402e050 con 0x7f2fe80047c0 2023-08-11T15:50:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.261+0000 7f2ff77fe640 1 -- 172.21.15.27:0/3690178097 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4003830 con 0x7f2fe80047c0 2023-08-11T15:50:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.262+0000 7f2ffed24640 1 -- 172.21.15.27:0/3690178097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 msgr2=0x7f2fe8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.262+0000 7f2ffed24640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2fe8004c40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4001750 tx=0x7f2fe80057f0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.262+0000 7f2ffed24640 1 -- 172.21.15.27:0/3690178097 shutdown_connections 2023-08-11T15:50:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.262+0000 7f2ffed24640 1 --2- 172.21.15.27:0/3690178097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2fe8004c40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.262+0000 7f2ffed24640 1 -- 172.21.15.27:0/3690178097 >> 172.21.15.27:0/3690178097 conn(0x7f2ff8069e90 msgr2=0x7f2ff80afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.262+0000 7f2ffed24640 1 -- 172.21.15.27:0/3690178097 shutdown_connections 2023-08-11T15:50:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.262+0000 7f2ffed24640 1 -- 172.21.15.27:0/3690178097 wait complete. 2023-08-11T15:50:58.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.264+0000 7f2ffed24640 1 Processor -- start 2023-08-11T15:50:58.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.264+0000 7f2ffed24640 1 -- start start 2023-08-11T15:50:58.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ffed24640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2ff80c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ffed24640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ff80af560 0x7f2ff814e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ffed24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ff80af9e0 0x7f2ff814e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ffed24640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff815b970 con 0x7f2fe80047c0 2023-08-11T15:50:58.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ffed24640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff815b690 con 0x7f2ff80af9e0 2023-08-11T15:50:58.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ffed24640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff815c370 con 0x7f2ff80af560 2023-08-11T15:50:58.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ffcac0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2ff80c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ffcac0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2ff80c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49562/0 (socket says 172.21.15.27:49562) 2023-08-11T15:50:58.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ffcac0640 1 -- 172.21.15.27:0/1758739681 learned_addr learned my addr 172.21.15.27:0/1758739681 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:58.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ffd2c1640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ff80af9e0 0x7f2ff814e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.265+0000 7f2ff7fff640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ff80af560 0x7f2ff814e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ffcac0640 1 -- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ff80af560 msgr2=0x7f2ff814e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ffcac0640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ff80af560 0x7f2ff814e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ffcac0640 1 -- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ff80af9e0 msgr2=0x7f2ff814e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ffcac0640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ff80af9e0 0x7f2ff814e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ffcac0640 1 -- 172.21.15.27:0/1758739681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff815c650 con 0x7f2fe80047c0 2023-08-11T15:50:58.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ffcac0640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2ff80c50e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00d7a0 tx=0x7f2fec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ff57fa640 1 -- 172.21.15.27:0/1758739681 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0074e0 con 0x7f2fe80047c0 2023-08-11T15:50:58.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ff57fa640 1 -- 172.21.15.27:0/1758739681 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec007660 con 0x7f2fe80047c0 2023-08-11T15:50:58.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ff57fa640 1 -- 172.21.15.27:0/1758739681 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec004a20 con 0x7f2fe80047c0 2023-08-11T15:50:58.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ffed24640 1 -- 172.21.15.27:0/1758739681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff815c930 con 0x7f2fe80047c0 2023-08-11T15:50:58.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.266+0000 7f2ffed24640 1 -- 172.21.15.27:0/1758739681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff80c5800 con 0x7f2fe80047c0 2023-08-11T15:50:58.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.267+0000 7f2ffed24640 1 -- 172.21.15.27:0/1758739681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff8073990 con 0x7f2fe80047c0 2023-08-11T15:50:58.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.268+0000 7f2ff57fa640 1 -- 172.21.15.27:0/1758739681 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2fec004b80 con 0x7f2fe80047c0 2023-08-11T15:50:58.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.270+0000 7f2ff57fa640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2fdc06a050 0x7f2fdc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.270+0000 7f2ff57fa640 1 -- 172.21.15.27:0/1758739681 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2fec087a40 con 0x7f2fe80047c0 2023-08-11T15:50:58.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.270+0000 7f2ff7fff640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2fdc06a050 0x7f2fdc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.272+0000 7f2ff7fff640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2fdc06a050 0x7f2fdc06c510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4033000 tx=0x7f2fe4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.274+0000 7f2ff57fa640 1 -- 172.21.15.27:0/1758739681 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2fec089e60 con 0x7f2fe80047c0 2023-08-11T15:50:58.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.469+0000 7f2ffed24640 1 -- 172.21.15.27:0/1758739681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2ff80b07b0 con 0x7f2fe80047c0 2023-08-11T15:50:58.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.470+0000 7f2ff57fa640 1 -- 172.21.15.27:0/1758739681 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2fec019020 con 0x7f2fe80047c0 2023-08-11T15:50:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:50:58.471 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-08-11T15:50:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:50:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-08-11T15:50:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-08-11T15:50:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:50:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:50:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 2, 2023-08-11T15:50:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": "rbd" 2023-08-11T15:50:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:50:58.473 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-08-11T15:50:58.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.472+0000 7f2ffed24640 1 -- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2fdc06a050 msgr2=0x7f2fdc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.472+0000 7f2ffed24640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2fdc06a050 0x7f2fdc06c510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4033000 tx=0x7f2fe4009030 comp rx=0 tx=0).stop 2023-08-11T15:50:58.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.472+0000 7f2ffed24640 1 -- 172.21.15.27:0/1758739681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 msgr2=0x7f2ff80c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.472+0000 7f2ffed24640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2ff80c50e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00d7a0 tx=0x7f2fec00dc60 comp rx=0 tx=0).stop 2023-08-11T15:50:58.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.473+0000 7f2ffed24640 1 -- 172.21.15.27:0/1758739681 shutdown_connections 2023-08-11T15:50:58.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.473+0000 7f2ffed24640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2fdc06a050 0x7f2fdc06c510 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.473+0000 7f2ffed24640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ff80af560 0x7f2ff814e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.473+0000 7f2ffed24640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ff80af9e0 0x7f2ff814e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.473+0000 7f2ffed24640 1 --2- 172.21.15.27:0/1758739681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe80047c0 0x7f2ff80c50e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.473+0000 7f2ffed24640 1 -- 172.21.15.27:0/1758739681 >> 172.21.15.27:0/1758739681 conn(0x7f2ff8069e90 msgr2=0x7f2ff80c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.473+0000 7f2ffed24640 1 -- 172.21.15.27:0/1758739681 shutdown_connections 2023-08-11T15:50:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.473+0000 7f2ffed24640 1 -- 172.21.15.27:0/1758739681 wait complete. 2023-08-11T15:50:58.493 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-11T15:50:58.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-11T15:50:58.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.736+0000 7fb4e5bb5640 1 Processor -- start 2023-08-11T15:50:58.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.737+0000 7fb4e5bb5640 1 -- start start 2023-08-11T15:50:58.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.738+0000 7fb4e5bb5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb4e00b55c0 0x7fb4e00cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.738+0000 7fb4e5bb5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb4e00cf930 0x7fb4e00d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.739+0000 7fb4e5bb5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e00cfd80 con 0x7fb4e015c620 2023-08-11T15:50:58.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.739+0000 7fb4e5bb5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e00ccc10 con 0x7fb4e00b55c0 2023-08-11T15:50:58.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.739+0000 7fb4e5bb5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e00cda30 con 0x7fb4e00cf930 2023-08-11T15:50:58.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.739+0000 7fb4df7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb4e015c620 0x7fb4e00b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33214/0 (socket says 172.21.15.27:33214) 2023-08-11T15:50:58.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.739+0000 7fb4df7fe640 1 -- 172.21.15.27:0/1759316937 learned_addr learned my addr 172.21.15.27:0/1759316937 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:50:58.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.739+0000 7fb4deffd640 1 --2- 172.21.15.27:0/1759316937 >> v2:172.21.15.119:3300/0 conn(0x7fb4e00b55c0 0x7fb4e00cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.739+0000 7fb4dffff640 1 --2- 172.21.15.27:0/1759316937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb4e00cf930 0x7fb4e00d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.739+0000 7fb4de7fc640 1 -- 172.21.15.27:0/1759316937 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913407076 0 0) 0x7fb4d0001490 con 0x7fb4e015c620 2023-08-11T15:50:58.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.740+0000 7fb4de7fc640 1 -- 172.21.15.27:0/1759316937 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4c8003520 con 0x7fb4e015c620 2023-08-11T15:50:58.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.740+0000 7fb4deffd640 1 -- 172.21.15.27:0/1759316937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb4e00cf930 msgr2=0x7fb4e00d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.740+0000 7fb4deffd640 1 --2- 172.21.15.27:0/1759316937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb4e00cf930 0x7fb4e00d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.740+0000 7fb4de7fc640 1 -- 172.21.15.27:0/1759316937 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4028463915 0 0) 0x7fb4c8003520 con 0x7fb4e015c620 2023-08-11T15:50:58.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.740+0000 7fb4deffd640 1 -- 172.21.15.27:0/1759316937 >> v1:172.21.15.27:6789/0 conn(0x7fb4e015c620 legacy=0x7fb4e00b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.740+0000 7fb4deffd640 1 -- 172.21.15.27:0/1759316937 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0161110 con 0x7fb4e00b55c0 2023-08-11T15:50:58.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.741+0000 7fb4deffd640 1 --2- 172.21.15.27:0/1759316937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4e00b55c0 0x7fb4e00cf3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40097a0 tx=0x7fb4d402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=366f6e31c19a2b97 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.742+0000 7fb4de7fc640 1 -- 172.21.15.27:0/1759316937 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0001b80 con 0x7fb4e00b55c0 2023-08-11T15:50:58.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.742+0000 7fb4de7fc640 1 -- 172.21.15.27:0/1759316937 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d402e050 con 0x7fb4e00b55c0 2023-08-11T15:50:58.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.742+0000 7fb4de7fc640 1 -- 172.21.15.27:0/1759316937 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4030710 con 0x7fb4e00b55c0 2023-08-11T15:50:58.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.742+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1759316937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4e00b55c0 msgr2=0x7fb4e00cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.742+0000 7fb4e5bb5640 1 --2- 172.21.15.27:0/1759316937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4e00b55c0 0x7fb4e00cf3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40097a0 tx=0x7fb4d402b790 comp rx=0 tx=0).stop 2023-08-11T15:50:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.742+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1759316937 shutdown_connections 2023-08-11T15:50:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.742+0000 7fb4e5bb5640 1 --2- 172.21.15.27:0/1759316937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb4e00cf930 0x7fb4e00d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.742+0000 7fb4e5bb5640 1 --2- 172.21.15.27:0/1759316937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4e00b55c0 0x7fb4e00cf3f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.742+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1759316937 >> 172.21.15.27:0/1759316937 conn(0x7fb4e0071110 msgr2=0x7fb4e00714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.742+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1759316937 shutdown_connections 2023-08-11T15:50:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.743+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1759316937 wait complete. 2023-08-11T15:50:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.744+0000 7fb4e5bb5640 1 Processor -- start 2023-08-11T15:50:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.744+0000 7fb4e5bb5640 1 -- start start 2023-08-11T15:50:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.745+0000 7fb4e5bb5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb4e00b55c0 0x7fb4e0159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.745+0000 7fb4e5bb5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4e00cf930 0x7fb4e00a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.745+0000 7fb4e5bb5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4e015c620 0x7fb4e00ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.745+0000 7fb4e5bb5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e00aac40 con 0x7fb4e00cf930 2023-08-11T15:50:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.745+0000 7fb4e5bb5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e00aa3a0 con 0x7fb4e015c620 2023-08-11T15:50:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.745+0000 7fb4e5bb5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e00aa680 con 0x7fb4e00b55c0 2023-08-11T15:50:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4deffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4e00cf930 0x7fb4e00a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4deffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4e00cf930 0x7fb4e00a99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49564/0 (socket says 172.21.15.27:49564) 2023-08-11T15:50:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4deffd640 1 -- 172.21.15.27:0/1532026508 learned_addr learned my addr 172.21.15.27:0/1532026508 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4df7fe640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb4e00b55c0 0x7fb4e0159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4dffff640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4e015c620 0x7fb4e00ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4deffd640 1 -- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb4e00b55c0 msgr2=0x7fb4e0159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4deffd640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb4e00b55c0 0x7fb4e0159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4deffd640 1 -- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4e015c620 msgr2=0x7fb4e00ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4deffd640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4e015c620 0x7fb4e00ab7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4deffd640 1 -- 172.21.15.27:0/1532026508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d4009400 con 0x7fb4e00cf930 2023-08-11T15:50:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.746+0000 7fb4deffd640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4e00cf930 0x7fb4e00a99c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4d402b7d0 tx=0x7fb4d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.747+0000 7fb4c7fff640 1 -- 172.21.15.27:0/1532026508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4030710 con 0x7fb4e00cf930 2023-08-11T15:50:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.747+0000 7fb4c7fff640 1 -- 172.21.15.27:0/1532026508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d4030870 con 0x7fb4e00cf930 2023-08-11T15:50:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.747+0000 7fb4c7fff640 1 -- 172.21.15.27:0/1532026508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4030c00 con 0x7fb4e00cf930 2023-08-11T15:50:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.747+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1532026508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e00aa960 con 0x7fb4e00cf930 2023-08-11T15:50:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.747+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1532026508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e0159dd0 con 0x7fb4e00cf930 2023-08-11T15:50:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.748+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1532026508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e01f1fd0 con 0x7fb4e00cf930 2023-08-11T15:50:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.749+0000 7fb4c7fff640 1 -- 172.21.15.27:0/1532026508 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb4d4039050 con 0x7fb4e00cf930 2023-08-11T15:50:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.750+0000 7fb4c7fff640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb4cc06a080 0x7fb4cc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.750+0000 7fb4c7fff640 1 -- 172.21.15.27:0/1532026508 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb4d40a9c50 con 0x7fb4e00cf930 2023-08-11T15:50:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.751+0000 7fb4df7fe640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb4cc06a080 0x7fb4cc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.752+0000 7fb4df7fe640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb4cc06a080 0x7fb4cc06c540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00013d0 tx=0x7fb4d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.755+0000 7fb4c7fff640 1 -- 172.21.15.27:0/1532026508 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fb4e01f1fd0 con 0x7fb4e00cf930 2023-08-11T15:50:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:58.949+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1532026508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb4e01f1fd0 con 0x7fb4e00cf930 2023-08-11T15:50:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.700+0000 7fb4c7fff640 1 -- 172.21.15.27:0/1532026508 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb4e01f1fd0 con 0x7fb4e00cf930 2023-08-11T15:50:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-08-11T15:50:59.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.706+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb4cc06a080 msgr2=0x7fb4cc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:59.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.706+0000 7fb4e5bb5640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb4cc06a080 0x7fb4cc06c540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00013d0 tx=0x7fb4d0009040 comp rx=0 tx=0).stop 2023-08-11T15:50:59.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.706+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1532026508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4e00cf930 msgr2=0x7fb4e00a99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:59.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.706+0000 7fb4e5bb5640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4e00cf930 0x7fb4e00a99c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4d402b7d0 tx=0x7fb4d40046c0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.708+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1532026508 shutdown_connections 2023-08-11T15:50:59.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.708+0000 7fb4e5bb5640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb4cc06a080 0x7fb4cc06c540 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.708+0000 7fb4e5bb5640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4e015c620 0x7fb4e00ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.708+0000 7fb4e5bb5640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4e00cf930 0x7fb4e00a99c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.708+0000 7fb4e5bb5640 1 --2- 172.21.15.27:0/1532026508 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb4e00b55c0 0x7fb4e0159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.708+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1532026508 >> 172.21.15.27:0/1532026508 conn(0x7fb4e0071110 msgr2=0x7fb4e015f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.708+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1532026508 shutdown_connections 2023-08-11T15:50:59.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.709+0000 7fb4e5bb5640 1 -- 172.21.15.27:0/1532026508 wait complete. 2023-08-11T15:50:59.733 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-08-11T15:50:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.961+0000 7f9179d42640 1 Processor -- start 2023-08-11T15:50:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.962+0000 7f9179d42640 1 -- start start 2023-08-11T15:50:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.963+0000 7f9179d42640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f91740b3090 0x7f91740d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.963+0000 7f9179d42640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9174161210 0x7f91740b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.963+0000 7f9179d42640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91740af250 con 0x7f91740b38f0 2023-08-11T15:50:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.963+0000 7f9179d42640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91740b0190 con 0x7f9174161210 2023-08-11T15:50:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.963+0000 7f9179d42640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91740af510 con 0x7f91740b3090 2023-08-11T15:50:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.964+0000 7f9173fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9174161210 0x7f91740b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.964+0000 7f9173fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9174161210 0x7f91740b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54210/0 (socket says 172.21.15.27:54210) 2023-08-11T15:50:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.964+0000 7f9173fff640 1 -- 172.21.15.27:0/1492039788 learned_addr learned my addr 172.21.15.27:0/1492039788 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.964+0000 7f916bfff640 1 --2- 172.21.15.27:0/1492039788 >> v2:172.21.15.119:3300/0 conn(0x7f91740b3090 0x7f91740d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.964+0000 7f9173fff640 1 -- 172.21.15.27:0/1492039788 >> v2:172.21.15.119:3300/0 conn(0x7f91740b3090 msgr2=0x7f91740d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.964+0000 7f9173fff640 1 --2- 172.21.15.27:0/1492039788 >> v2:172.21.15.119:3300/0 conn(0x7f91740b3090 0x7f91740d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.964+0000 7f9173fff640 1 -- 172.21.15.27:0/1492039788 >> v1:172.21.15.119:6789/0 conn(0x7f91740b38f0 legacy=0x7f9174160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.965+0000 7f9173fff640 1 -- 172.21.15.27:0/1492039788 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91740af7f0 con 0x7f9174161210 2023-08-11T15:50:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.966+0000 7f9173fff640 1 --2- 172.21.15.27:0/1492039788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 0x7f91740b05b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91740c6230 tx=0x7f916c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde580ca5542b09b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.966+0000 7f9172ffd640 1 -- 172.21.15.27:0/1492039788 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c02cb20 con 0x7f9174161210 2023-08-11T15:50:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.966+0000 7f9172ffd640 1 -- 172.21.15.27:0/1492039788 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c02cc80 con 0x7f9174161210 2023-08-11T15:50:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.966+0000 7f9172ffd640 1 -- 172.21.15.27:0/1492039788 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0343f0 con 0x7f9174161210 2023-08-11T15:50:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.967+0000 7f9179d42640 1 -- 172.21.15.27:0/1492039788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 msgr2=0x7f91740b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.967+0000 7f9179d42640 1 --2- 172.21.15.27:0/1492039788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 0x7f91740b05b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91740c6230 tx=0x7f916c02bf80 comp rx=0 tx=0).stop 2023-08-11T15:50:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.967+0000 7f9179d42640 1 -- 172.21.15.27:0/1492039788 shutdown_connections 2023-08-11T15:50:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.967+0000 7f9179d42640 1 --2- 172.21.15.27:0/1492039788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 0x7f91740b05b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.967+0000 7f9179d42640 1 --2- 172.21.15.27:0/1492039788 >> v2:172.21.15.119:3300/0 conn(0x7f91740b3090 0x7f91740d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.967+0000 7f9179d42640 1 -- 172.21.15.27:0/1492039788 >> 172.21.15.27:0/1492039788 conn(0x7f917409b960 msgr2=0x7f91740d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.967+0000 7f9179d42640 1 -- 172.21.15.27:0/1492039788 shutdown_connections 2023-08-11T15:50:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.967+0000 7f9179d42640 1 -- 172.21.15.27:0/1492039788 wait complete. 2023-08-11T15:50:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.969+0000 7f9179d42640 1 Processor -- start 2023-08-11T15:50:59.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.969+0000 7f9179d42640 1 -- start start 2023-08-11T15:50:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.970+0000 7f9179d42640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91740b3090 0x7f91741f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.970+0000 7f9179d42640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f91740b38f0 0x7f91740ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.970+0000 7f9179d42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 0x7f91740c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.970+0000 7f9179d42640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91741f30b0 con 0x7f9174161210 2023-08-11T15:50:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.970+0000 7f9179d42640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91740cafe0 con 0x7f91740b3090 2023-08-11T15:50:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.970+0000 7f9179d42640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91741f5c30 con 0x7f91740b38f0 2023-08-11T15:50:59.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f9173fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 0x7f91740c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:59.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f9173fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 0x7f91740c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54220/0 (socket says 172.21.15.27:54220) 2023-08-11T15:50:59.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f9173fff640 1 -- 172.21.15.27:0/442203701 learned_addr learned my addr 172.21.15.27:0/442203701 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:50:59.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f916bfff640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f91740b38f0 0x7f91740ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:59.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f91737fe640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91740b3090 0x7f91741f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:59.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f9173fff640 1 -- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f91740b38f0 msgr2=0x7f91740ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:59.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f9173fff640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f91740b38f0 0x7f91740ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f9173fff640 1 -- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91740b3090 msgr2=0x7f91741f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f9173fff640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91740b3090 0x7f91741f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f9173fff640 1 -- 172.21.15.27:0/442203701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c009b30 con 0x7f9174161210 2023-08-11T15:50:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f91737fe640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91740b3090 0x7f91741f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.971+0000 7f9173fff640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 0x7f91740c9800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f916c036040 tx=0x7f916c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.972+0000 7f9170ff9640 1 -- 172.21.15.27:0/442203701 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c009510 con 0x7f9174161210 2023-08-11T15:50:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.972+0000 7f9170ff9640 1 -- 172.21.15.27:0/442203701 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c02c920 con 0x7f9174161210 2023-08-11T15:50:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.972+0000 7f9170ff9640 1 -- 172.21.15.27:0/442203701 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c03e670 con 0x7f9174161210 2023-08-11T15:50:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.972+0000 7f9179d42640 1 -- 172.21.15.27:0/442203701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91741f5f10 con 0x7f9174161210 2023-08-11T15:50:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.972+0000 7f9179d42640 1 -- 172.21.15.27:0/442203701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9174149570 con 0x7f9174161210 2023-08-11T15:50:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.973+0000 7f9179d42640 1 -- 172.21.15.27:0/442203701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917404e420 con 0x7f9174161210 2023-08-11T15:50:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.974+0000 7f9170ff9640 1 -- 172.21.15.27:0/442203701 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f916c03e7d0 con 0x7f9174161210 2023-08-11T15:50:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.976+0000 7f9170ff9640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f915406a050 0x7f915406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.976+0000 7f9170ff9640 1 -- 172.21.15.27:0/442203701 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f916c0aba50 con 0x7f9174161210 2023-08-11T15:50:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.977+0000 7f91737fe640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f915406a050 0x7f915406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.978+0000 7f91737fe640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f915406a050 0x7f915406c510 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9164000fb0 tx=0x7f9164009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:59.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:50:59.980+0000 7f9170ff9640 1 -- 172.21.15.27:0/442203701 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f916c077300 con 0x7f9174161210 2023-08-11T15:51:00.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.178+0000 7f9179d42640 1 -- 172.21.15.27:0/442203701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f91740b86b0 con 0x7f9174161210 2023-08-11T15:51:00.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.714+0000 7f9170ff9640 1 -- 172.21.15.27:0/442203701 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v22) v1 ==== 171+0+0 (secure 0 0 0) 0x7f916c07a920 con 0x7f9174161210 2023-08-11T15:51:00.721 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-08-11T15:51:00.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.738+0000 7f9179d42640 1 -- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f915406a050 msgr2=0x7f915406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:00.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.738+0000 7f9179d42640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f915406a050 0x7f915406c510 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9164000fb0 tx=0x7f9164009040 comp rx=0 tx=0).stop 2023-08-11T15:51:00.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.738+0000 7f9179d42640 1 -- 172.21.15.27:0/442203701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 msgr2=0x7f91740c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:00.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.738+0000 7f9179d42640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 0x7f91740c9800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f916c036040 tx=0x7f916c004a40 comp rx=0 tx=0).stop 2023-08-11T15:51:00.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.739+0000 7f9179d42640 1 -- 172.21.15.27:0/442203701 shutdown_connections 2023-08-11T15:51:00.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.739+0000 7f9179d42640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f915406a050 0x7f915406c510 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.739+0000 7f9179d42640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f91740b38f0 0x7f91740ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.739+0000 7f9179d42640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9174161210 0x7f91740c9800 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.739+0000 7f9179d42640 1 --2- 172.21.15.27:0/442203701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91740b3090 0x7f91741f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.739+0000 7f9179d42640 1 -- 172.21.15.27:0/442203701 >> 172.21.15.27:0/442203701 conn(0x7f917409b960 msgr2=0x7f917415ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.740+0000 7f9179d42640 1 -- 172.21.15.27:0/442203701 shutdown_connections 2023-08-11T15:51:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.740+0000 7f9179d42640 1 -- 172.21.15.27:0/442203701 wait complete. 2023-08-11T15:51:00.766 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-11T15:51:00.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.964+0000 7ffabbc0d640 1 Processor -- start 2023-08-11T15:51:00.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.964+0000 7ffabbc0d640 1 -- start start 2023-08-11T15:51:00.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.965+0000 7ffabbc0d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffab4151220 0x7ffab406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:00.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.965+0000 7ffabbc0d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffab406bdf0 0x7ffab415df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:00.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.965+0000 7ffabbc0d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab406c850 con 0x7ffab4151600 2023-08-11T15:51:00.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.965+0000 7ffabbc0d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab406c290 con 0x7ffab406bdf0 2023-08-11T15:51:00.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.965+0000 7ffabbc0d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab406c570 con 0x7ffab4151220 2023-08-11T15:51:00.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.966+0000 7ffab91a8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffab406bdf0 0x7ffab415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:00.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.966+0000 7ffab91a8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffab406bdf0 0x7ffab415df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54226/0 (socket says 172.21.15.27:54226) 2023-08-11T15:51:00.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.966+0000 7ffab91a8640 1 -- 172.21.15.27:0/273977829 learned_addr learned my addr 172.21.15.27:0/273977829 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:00.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.966+0000 7ffab99a9640 1 --2- 172.21.15.27:0/273977829 >> v2:172.21.15.119:3300/0 conn(0x7ffab4151220 0x7ffab406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:00.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.966+0000 7ffab91a8640 1 -- 172.21.15.27:0/273977829 >> v2:172.21.15.119:3300/0 conn(0x7ffab4151220 msgr2=0x7ffab406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:00.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.966+0000 7ffab91a8640 1 --2- 172.21.15.27:0/273977829 >> v2:172.21.15.119:3300/0 conn(0x7ffab4151220 0x7ffab406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:00.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.966+0000 7ffab91a8640 1 -- 172.21.15.27:0/273977829 >> v1:172.21.15.119:6789/0 conn(0x7ffab4151600 legacy=0x7ffab40ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:00.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.966+0000 7ffab91a8640 1 -- 172.21.15.27:0/273977829 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40ca050 con 0x7ffab406bdf0 2023-08-11T15:51:00.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.968+0000 7ffab91a8640 1 --2- 172.21.15.27:0/273977829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab406bdf0 0x7ffab415df70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffab4070e50 tx=0x7ffaa402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eaecd105180d81 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:00.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.968+0000 7ffab89a7640 1 -- 172.21.15.27:0/273977829 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4031750 con 0x7ffab406bdf0 2023-08-11T15:51:00.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.968+0000 7ffab89a7640 1 -- 172.21.15.27:0/273977829 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa40318b0 con 0x7ffab406bdf0 2023-08-11T15:51:00.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.968+0000 7ffab89a7640 1 -- 172.21.15.27:0/273977829 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4031c40 con 0x7ffab406bdf0 2023-08-11T15:51:00.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.969+0000 7ffabbc0d640 1 -- 172.21.15.27:0/273977829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab406bdf0 msgr2=0x7ffab415df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:00.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.969+0000 7ffabbc0d640 1 --2- 172.21.15.27:0/273977829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab406bdf0 0x7ffab415df70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffab4070e50 tx=0x7ffaa402b7f0 comp rx=0 tx=0).stop 2023-08-11T15:51:00.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.969+0000 7ffabbc0d640 1 -- 172.21.15.27:0/273977829 shutdown_connections 2023-08-11T15:51:00.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.969+0000 7ffabbc0d640 1 --2- 172.21.15.27:0/273977829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab406bdf0 0x7ffab415df70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:00.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.969+0000 7ffabbc0d640 1 --2- 172.21.15.27:0/273977829 >> v2:172.21.15.119:3300/0 conn(0x7ffab4151220 0x7ffab406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:00.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.969+0000 7ffabbc0d640 1 -- 172.21.15.27:0/273977829 >> 172.21.15.27:0/273977829 conn(0x7ffab40afbe0 msgr2=0x7ffab40affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:00.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.969+0000 7ffabbc0d640 1 -- 172.21.15.27:0/273977829 shutdown_connections 2023-08-11T15:51:00.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.969+0000 7ffabbc0d640 1 -- 172.21.15.27:0/273977829 wait complete. 2023-08-11T15:51:00.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.971+0000 7ffabbc0d640 1 Processor -- start 2023-08-11T15:51:00.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.971+0000 7ffabbc0d640 1 -- start start 2023-08-11T15:51:00.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.972+0000 7ffabbc0d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab406bdf0 0x7ffab40c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:00.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.972+0000 7ffabbc0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab4151220 0x7ffab4150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:00.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.972+0000 7ffabbc0d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab4151600 0x7ffab40c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:00.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.972+0000 7ffabbc0d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab41f1b40 con 0x7ffab4151600 2023-08-11T15:51:00.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.972+0000 7ffabbc0d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab4150a60 con 0x7ffab4151220 2023-08-11T15:51:00.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.972+0000 7ffabbc0d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab41f0f30 con 0x7ffab406bdf0 2023-08-11T15:51:00.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffaba1aa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab4151600 0x7ffab40c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:00.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffaba1aa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab4151600 0x7ffab40c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54238/0 (socket says 172.21.15.27:54238) 2023-08-11T15:51:00.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffaba1aa640 1 -- 172.21.15.27:0/4151024762 learned_addr learned my addr 172.21.15.27:0/4151024762 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:00.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffab91a8640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab4151220 0x7ffab4150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:00.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffab99a9640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab406bdf0 0x7ffab40c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:00.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffaba1aa640 1 -- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab406bdf0 msgr2=0x7ffab40c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:00.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffaba1aa640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab406bdf0 0x7ffab40c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:00.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffaba1aa640 1 -- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab4151220 msgr2=0x7ffab4150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:00.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffaba1aa640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab4151220 0x7ffab4150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:00.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffaba1aa640 1 -- 172.21.15.27:0/4151024762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40094d0 con 0x7ffab4151600 2023-08-11T15:51:00.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffab91a8640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab4151220 0x7ffab4150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:00.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.973+0000 7ffab99a9640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab406bdf0 0x7ffab40c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:00.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.974+0000 7ffaba1aa640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab4151600 0x7ffab40c84c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b9e0 tx=0x7ffaac00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:00.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.974+0000 7ffaaa7fc640 1 -- 172.21.15.27:0/4151024762 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac003b70 con 0x7ffab4151600 2023-08-11T15:51:00.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.974+0000 7ffaaa7fc640 1 -- 172.21.15.27:0/4151024762 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac003cd0 con 0x7ffab4151600 2023-08-11T15:51:00.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.974+0000 7ffaaa7fc640 1 -- 172.21.15.27:0/4151024762 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac00cb60 con 0x7ffab4151600 2023-08-11T15:51:00.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.974+0000 7ffabbc0d640 1 -- 172.21.15.27:0/4151024762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab41f1210 con 0x7ffab4151600 2023-08-11T15:51:00.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.974+0000 7ffabbc0d640 1 -- 172.21.15.27:0/4151024762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab41f17d0 con 0x7ffab4151600 2023-08-11T15:51:00.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.975+0000 7ffabbc0d640 1 -- 172.21.15.27:0/4151024762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab404e420 con 0x7ffab4151600 2023-08-11T15:51:00.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.976+0000 7ffaaa7fc640 1 -- 172.21.15.27:0/4151024762 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffaac00ccc0 con 0x7ffab4151600 2023-08-11T15:51:00.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.977+0000 7ffaaa7fc640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa9806a050 0x7ffa9806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:00.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.977+0000 7ffaaa7fc640 1 -- 172.21.15.27:0/4151024762 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5962+0+0 (secure 0 0 0) 0x7ffaac0883b0 con 0x7ffab4151600 2023-08-11T15:51:00.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.978+0000 7ffab99a9640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa9806a050 0x7ffa9806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:00.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.979+0000 7ffab99a9640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa9806a050 0x7ffa9806c510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffab0006fd0 tx=0x7ffab0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:00.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:00.984+0000 7ffaaa7fc640 1 -- 172.21.15.27:0/4151024762 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ffaac053a60 con 0x7ffab4151600 2023-08-11T15:51:01.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.219+0000 7ffabbc0d640 1 -- 172.21.15.27:0/4151024762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ffab41f1c80 con 0x7ffab4151600 2023-08-11T15:51:01.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.754+0000 7ffaaa7fc640 1 -- 172.21.15.27:0/4151024762 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7ffaac057080 con 0x7ffab4151600 2023-08-11T15:51:01.756 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 3 and data pool 4 2023-08-11T15:51:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.760+0000 7ffabbc0d640 1 -- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa9806a050 msgr2=0x7ffa9806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.760+0000 7ffabbc0d640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa9806a050 0x7ffa9806c510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffab0006fd0 tx=0x7ffab0008040 comp rx=0 tx=0).stop 2023-08-11T15:51:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.760+0000 7ffabbc0d640 1 -- 172.21.15.27:0/4151024762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab4151600 msgr2=0x7ffab40c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.760+0000 7ffabbc0d640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab4151600 0x7ffab40c84c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b9e0 tx=0x7ffaac00bea0 comp rx=0 tx=0).stop 2023-08-11T15:51:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.761+0000 7ffabbc0d640 1 -- 172.21.15.27:0/4151024762 shutdown_connections 2023-08-11T15:51:01.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.761+0000 7ffabbc0d640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa9806a050 0x7ffa9806c510 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:01.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.761+0000 7ffabbc0d640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffab4151600 0x7ffab40c84c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:01.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.761+0000 7ffabbc0d640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab4151220 0x7ffab4150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:01.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.761+0000 7ffabbc0d640 1 --2- 172.21.15.27:0/4151024762 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab406bdf0 0x7ffab40c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:01.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.761+0000 7ffabbc0d640 1 -- 172.21.15.27:0/4151024762 >> 172.21.15.27:0/4151024762 conn(0x7ffab40afbe0 msgr2=0x7ffab415d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.761+0000 7ffabbc0d640 1 -- 172.21.15.27:0/4151024762 shutdown_connections 2023-08-11T15:51:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.761+0000 7ffabbc0d640 1 -- 172.21.15.27:0/4151024762 wait complete. 2023-08-11T15:51:01.784 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-08-11T15:51:01.784 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-08-11T15:51:01.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.989+0000 7f613166a640 1 Processor -- start 2023-08-11T15:51:01.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.990+0000 7f613166a640 1 -- start start 2023-08-11T15:51:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.991+0000 7f613166a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f612c0cb0c0 0x7f612c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.991+0000 7f613166a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c0cbb70 0x7f612c0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.991+0000 7f613166a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612c0cc020 con 0x7f612c156e50 2023-08-11T15:51:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.991+0000 7f613166a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612c0ca460 con 0x7f612c0cb0c0 2023-08-11T15:51:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.991+0000 7f613166a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c070470 con 0x7f612c0cbb70 2023-08-11T15:51:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.992+0000 7f6123fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f612c0cb0c0 0x7f612c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.992+0000 7f6123fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f612c0cb0c0 0x7f612c15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54248/0 (socket says 172.21.15.27:54248) 2023-08-11T15:51:01.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.992+0000 7f6123fff640 1 -- 172.21.15.27:0/1057642941 learned_addr learned my addr 172.21.15.27:0/1057642941 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:01.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.992+0000 7f612b7fe640 1 --2- 172.21.15.27:0/1057642941 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c0cbb70 0x7f612c0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:01.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.992+0000 7f6123fff640 1 -- 172.21.15.27:0/1057642941 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c0cbb70 msgr2=0x7f612c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:01.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.992+0000 7f6123fff640 1 --2- 172.21.15.27:0/1057642941 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c0cbb70 0x7f612c0708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:01.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.992+0000 7f6123fff640 1 -- 172.21.15.27:0/1057642941 >> v1:172.21.15.119:6789/0 conn(0x7f612c156e50 legacy=0x7f612c0ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:01.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.992+0000 7f6123fff640 1 -- 172.21.15.27:0/1057642941 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0cc270 con 0x7f612c0cb0c0 2023-08-11T15:51:01.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.994+0000 7f6123fff640 1 --2- 172.21.15.27:0/1057642941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cb0c0 0x7f612c15a180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6114009870 tx=0x7f611402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d751356ccbd6b9b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:01.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.994+0000 7f612a7fc640 1 -- 172.21.15.27:0/1057642941 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114030840 con 0x7f612c0cb0c0 2023-08-11T15:51:01.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.994+0000 7f612a7fc640 1 -- 172.21.15.27:0/1057642941 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61140309a0 con 0x7f612c0cb0c0 2023-08-11T15:51:01.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.994+0000 7f612a7fc640 1 -- 172.21.15.27:0/1057642941 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114030d30 con 0x7f612c0cb0c0 2023-08-11T15:51:01.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.995+0000 7f613166a640 1 -- 172.21.15.27:0/1057642941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cb0c0 msgr2=0x7f612c15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:01.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.995+0000 7f613166a640 1 --2- 172.21.15.27:0/1057642941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cb0c0 0x7f612c15a180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6114009870 tx=0x7f611402b980 comp rx=0 tx=0).stop 2023-08-11T15:51:01.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.995+0000 7f613166a640 1 -- 172.21.15.27:0/1057642941 shutdown_connections 2023-08-11T15:51:01.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.995+0000 7f613166a640 1 --2- 172.21.15.27:0/1057642941 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c0cbb70 0x7f612c0708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:01.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.995+0000 7f613166a640 1 --2- 172.21.15.27:0/1057642941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cb0c0 0x7f612c15a180 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.995+0000 7f613166a640 1 -- 172.21.15.27:0/1057642941 >> 172.21.15.27:0/1057642941 conn(0x7f612c0afb70 msgr2=0x7f612c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.995+0000 7f613166a640 1 -- 172.21.15.27:0/1057642941 shutdown_connections 2023-08-11T15:51:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.995+0000 7f613166a640 1 -- 172.21.15.27:0/1057642941 wait complete. 2023-08-11T15:51:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.997+0000 7f613166a640 1 Processor -- start 2023-08-11T15:51:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.997+0000 7f613166a640 1 -- start start 2023-08-11T15:51:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f613166a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f612c0cb0c0 0x7f612c0c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f613166a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cbb70 0x7f612c0c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f613166a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c156e50 0x7f612c0af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f613166a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0c30a0 con 0x7f612c0cbb70 2023-08-11T15:51:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f613166a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0c3a00 con 0x7f612c0cb0c0 2023-08-11T15:51:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f613166a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0c2420 con 0x7f612c156e50 2023-08-11T15:51:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f6123fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cbb70 0x7f612c0c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f6123fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cbb70 0x7f612c0c1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54252/0 (socket says 172.21.15.27:54252) 2023-08-11T15:51:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f6123fff640 1 -- 172.21.15.27:0/1674000894 learned_addr learned my addr 172.21.15.27:0/1674000894 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f612affd640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f612c0cb0c0 0x7f612c0c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.998+0000 7f612b7fe640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c156e50 0x7f612c0af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.999+0000 7f6123fff640 1 -- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c156e50 msgr2=0x7f612c0af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.999+0000 7f6123fff640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c156e50 0x7f612c0af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.999+0000 7f6123fff640 1 -- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f612c0cb0c0 msgr2=0x7f612c0c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.999+0000 7f6123fff640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f612c0cb0c0 0x7f612c0c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.999+0000 7f6123fff640 1 -- 172.21.15.27:0/1674000894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61140094d0 con 0x7f612c0cbb70 2023-08-11T15:51:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.999+0000 7f612b7fe640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c156e50 0x7f612c0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:02.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:01.999+0000 7f6123fff640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cbb70 0x7f612c0c1ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6114000c00 tx=0x7f6114004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:02.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.000+0000 7f61237fe640 1 -- 172.21.15.27:0/1674000894 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114039a90 con 0x7f612c0cbb70 2023-08-11T15:51:02.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.000+0000 7f61237fe640 1 -- 172.21.15.27:0/1674000894 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6114039bf0 con 0x7f612c0cbb70 2023-08-11T15:51:02.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.000+0000 7f61237fe640 1 -- 172.21.15.27:0/1674000894 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114043410 con 0x7f612c0cbb70 2023-08-11T15:51:02.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.000+0000 7f613166a640 1 -- 172.21.15.27:0/1674000894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c0c2700 con 0x7f612c0cbb70 2023-08-11T15:51:02.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.000+0000 7f613166a640 1 -- 172.21.15.27:0/1674000894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c0c2cc0 con 0x7f612c0cbb70 2023-08-11T15:51:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.001+0000 7f613166a640 1 -- 172.21.15.27:0/1674000894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612c0c31e0 con 0x7f612c0cbb70 2023-08-11T15:51:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.002+0000 7f61237fe640 1 -- 172.21.15.27:0/1674000894 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f611402f390 con 0x7f612c0cbb70 2023-08-11T15:51:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.003+0000 7f61237fe640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a050 0x7f610c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:02.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.003+0000 7f61237fe640 1 -- 172.21.15.27:0/1674000894 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61140aa700 con 0x7f612c0cbb70 2023-08-11T15:51:02.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.004+0000 7f612affd640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a050 0x7f610c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:02.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.005+0000 7f612affd640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a050 0x7f610c06c510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f611c007fd0 tx=0x7f611c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:02.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.008+0000 7f61237fe640 1 -- 172.21.15.27:0/1674000894 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f612c0c31e0 con 0x7f612c0cbb70 2023-08-11T15:51:02.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.210+0000 7f613166a640 1 -- 172.21.15.27:0/1674000894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f612c0c31e0 con 0x7f612c0cbb70 2023-08-11T15:51:02.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.713+0000 7f61237fe640 1 -- 172.21.15.27:0/1674000894 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v24) v1 ==== 157+0+0 (secure 0 0 0) 0x7f612c0c31e0 con 0x7f612c0cbb70 2023-08-11T15:51:02.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.717+0000 7f613166a640 1 -- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a050 msgr2=0x7f610c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:02.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.717+0000 7f613166a640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a050 0x7f610c06c510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f611c007fd0 tx=0x7f611c009040 comp rx=0 tx=0).stop 2023-08-11T15:51:02.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.717+0000 7f613166a640 1 -- 172.21.15.27:0/1674000894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cbb70 msgr2=0x7f612c0c1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:02.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.717+0000 7f613166a640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cbb70 0x7f612c0c1ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6114000c00 tx=0x7f6114004730 comp rx=0 tx=0).stop 2023-08-11T15:51:02.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.717+0000 7f613166a640 1 -- 172.21.15.27:0/1674000894 shutdown_connections 2023-08-11T15:51:02.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.717+0000 7f613166a640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a050 0x7f610c06c510 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.717+0000 7f613166a640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612c156e50 0x7f612c0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.717+0000 7f613166a640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c0cbb70 0x7f612c0c1ad0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.717+0000 7f613166a640 1 --2- 172.21.15.27:0/1674000894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f612c0cb0c0 0x7f612c0c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.717+0000 7f613166a640 1 -- 172.21.15.27:0/1674000894 >> 172.21.15.27:0/1674000894 conn(0x7f612c0afb70 msgr2=0x7f612c0708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:02.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.718+0000 7f613166a640 1 -- 172.21.15.27:0/1674000894 shutdown_connections 2023-08-11T15:51:02.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.718+0000 7f613166a640 1 -- 172.21.15.27:0/1674000894 wait complete. 2023-08-11T15:51:02.743 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-08-11T15:51:02.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.981+0000 7fdd18bd5640 1 Processor -- start 2023-08-11T15:51:02.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.981+0000 7fdd18bd5640 1 -- start start 2023-08-11T15:51:02.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.983+0000 7fdd18bd5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd1415c6d0 0x7fdd1406b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:02.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.983+0000 7fdd18bd5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdd140d0d20 0x7fdd1406b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:02.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.983+0000 7fdd18bd5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd140af5f0 con 0x7fdd140cad70 2023-08-11T15:51:02.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.983+0000 7fdd18bd5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd140ca8e0 con 0x7fdd140d0d20 2023-08-11T15:51:02.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.983+0000 7fdd18bd5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140af310 con 0x7fdd1415c6d0 2023-08-11T15:51:02.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.983+0000 7fdd11d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdd140d0d20 0x7fdd1406b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:02.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.983+0000 7fdd11d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdd140d0d20 0x7fdd1406b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54268/0 (socket says 172.21.15.27:54268) 2023-08-11T15:51:02.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.983+0000 7fdd11d9b640 1 -- 172.21.15.27:0/3741646879 learned_addr learned my addr 172.21.15.27:0/3741646879 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:02.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.984+0000 7fdd1259c640 1 --2- 172.21.15.27:0/3741646879 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd1415c6d0 0x7fdd1406b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:02.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.984+0000 7fdd11d9b640 1 -- 172.21.15.27:0/3741646879 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd1415c6d0 msgr2=0x7fdd1406b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:02.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.984+0000 7fdd11d9b640 1 --2- 172.21.15.27:0/3741646879 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd1415c6d0 0x7fdd1406b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.984+0000 7fdd11d9b640 1 -- 172.21.15.27:0/3741646879 >> v1:172.21.15.119:6789/0 conn(0x7fdd140cad70 legacy=0x7fdd140cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:02.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.984+0000 7fdd11d9b640 1 -- 172.21.15.27:0/3741646879 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1406ac50 con 0x7fdd140d0d20 2023-08-11T15:51:02.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.985+0000 7fdd11d9b640 1 --2- 172.21.15.27:0/3741646879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 0x7fdd1406b7d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009870 tx=0x7fdd0802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25c71dc7d001718 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:02.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.985+0000 7fdd1159a640 1 -- 172.21.15.27:0/3741646879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08031640 con 0x7fdd140d0d20 2023-08-11T15:51:02.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.985+0000 7fdd1159a640 1 -- 172.21.15.27:0/3741646879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd080317a0 con 0x7fdd140d0d20 2023-08-11T15:51:02.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.985+0000 7fdd1159a640 1 -- 172.21.15.27:0/3741646879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08031b30 con 0x7fdd140d0d20 2023-08-11T15:51:02.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.986+0000 7fdd18bd5640 1 -- 172.21.15.27:0/3741646879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 msgr2=0x7fdd1406b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:02.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.986+0000 7fdd18bd5640 1 --2- 172.21.15.27:0/3741646879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 0x7fdd1406b7d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009870 tx=0x7fdd0802b8f0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.986+0000 7fdd18bd5640 1 -- 172.21.15.27:0/3741646879 shutdown_connections 2023-08-11T15:51:02.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.986+0000 7fdd18bd5640 1 --2- 172.21.15.27:0/3741646879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 0x7fdd1406b7d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.986+0000 7fdd18bd5640 1 --2- 172.21.15.27:0/3741646879 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd1415c6d0 0x7fdd1406b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.986+0000 7fdd18bd5640 1 -- 172.21.15.27:0/3741646879 >> 172.21.15.27:0/3741646879 conn(0x7fdd140d0200 msgr2=0x7fdd140d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:02.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.986+0000 7fdd18bd5640 1 -- 172.21.15.27:0/3741646879 shutdown_connections 2023-08-11T15:51:02.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.986+0000 7fdd18bd5640 1 -- 172.21.15.27:0/3741646879 wait complete. 2023-08-11T15:51:02.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.988+0000 7fdd18bd5640 1 Processor -- start 2023-08-11T15:51:02.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.988+0000 7fdd18bd5640 1 -- start start 2023-08-11T15:51:02.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.989+0000 7fdd18bd5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd140cad70 0x7fdd140b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:02.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.989+0000 7fdd18bd5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 0x7fdd140b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:02.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.989+0000 7fdd18bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd1415c6d0 0x7fdd140b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:02.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.989+0000 7fdd18bd5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd14150e10 con 0x7fdd140d0d20 2023-08-11T15:51:02.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.989+0000 7fdd18bd5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140b3920 con 0x7fdd1415c6d0 2023-08-11T15:51:02.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.989+0000 7fdd18bd5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140b3c00 con 0x7fdd140cad70 2023-08-11T15:51:02.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 0x7fdd140b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:02.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd11d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 0x7fdd140b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54278/0 (socket says 172.21.15.27:54278) 2023-08-11T15:51:02.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd11d9b640 1 -- 172.21.15.27:0/1029872258 learned_addr learned my addr 172.21.15.27:0/1029872258 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:02.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd1259c640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd140cad70 0x7fdd140b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:02.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd12d9d640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd1415c6d0 0x7fdd140b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:02.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd11d9b640 1 -- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd140cad70 msgr2=0x7fdd140b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:02.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd11d9b640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd140cad70 0x7fdd140b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd11d9b640 1 -- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd1415c6d0 msgr2=0x7fdd140b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:02.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd11d9b640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd1415c6d0 0x7fdd140b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:02.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd11d9b640 1 -- 172.21.15.27:0/1029872258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd080094d0 con 0x7fdd140d0d20 2023-08-11T15:51:02.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd12d9d640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd1415c6d0 0x7fdd140b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:02.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.990+0000 7fdd11d9b640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 0x7fdd140b49c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009870 tx=0x7fdd080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:02.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.991+0000 7fdd02ffd640 1 -- 172.21.15.27:0/1029872258 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0803ac60 con 0x7fdd140d0d20 2023-08-11T15:51:02.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.991+0000 7fdd02ffd640 1 -- 172.21.15.27:0/1029872258 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0803ade0 con 0x7fdd140d0d20 2023-08-11T15:51:02.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.991+0000 7fdd02ffd640 1 -- 172.21.15.27:0/1029872258 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08044630 con 0x7fdd140d0d20 2023-08-11T15:51:02.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.991+0000 7fdd18bd5640 1 -- 172.21.15.27:0/1029872258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd140b3ee0 con 0x7fdd140d0d20 2023-08-11T15:51:02.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.991+0000 7fdd18bd5640 1 -- 172.21.15.27:0/1029872258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd140b44a0 con 0x7fdd140d0d20 2023-08-11T15:51:02.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.992+0000 7fdd18bd5640 1 -- 172.21.15.27:0/1029872258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd14073990 con 0x7fdd140d0d20 2023-08-11T15:51:02.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.993+0000 7fdd02ffd640 1 -- 172.21.15.27:0/1029872258 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdd0802f050 con 0x7fdd140d0d20 2023-08-11T15:51:02.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.994+0000 7fdd02ffd640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdcf406a050 0x7fdcf406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:02.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.994+0000 7fdd02ffd640 1 -- 172.21.15.27:0/1029872258 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fdd080aa780 con 0x7fdd140d0d20 2023-08-11T15:51:02.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.995+0000 7fdd1259c640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdcf406a050 0x7fdcf406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:02.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.996+0000 7fdd1259c640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdcf406a050 0x7fdcf406c510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdd040070b0 tx=0x7fdd04007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:03.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:02.999+0000 7fdd02ffd640 1 -- 172.21.15.27:0/1029872258 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fdd08075f40 con 0x7fdd140d0d20 2023-08-11T15:51:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:03.203+0000 7fdd18bd5640 1 -- 172.21.15.27:0/1029872258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdd140b2230 con 0x7fdd140d0d20 2023-08-11T15:51:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.805+0000 7fdd02ffd640 1 -- 172.21.15.27:0/1029872258 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v26) v1 ==== 228+0+0 (secure 0 0 0) 0x7fdd08079560 con 0x7fdd140d0d20 2023-08-11T15:51:04.807 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' created 2023-08-11T15:51:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.811+0000 7fdd18bd5640 1 -- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdcf406a050 msgr2=0x7fdcf406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.811+0000 7fdd18bd5640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdcf406a050 0x7fdcf406c510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdd040070b0 tx=0x7fdd04007040 comp rx=0 tx=0).stop 2023-08-11T15:51:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.811+0000 7fdd18bd5640 1 -- 172.21.15.27:0/1029872258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 msgr2=0x7fdd140b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.811+0000 7fdd18bd5640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 0x7fdd140b49c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009870 tx=0x7fdd080046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.812+0000 7fdd18bd5640 1 -- 172.21.15.27:0/1029872258 shutdown_connections 2023-08-11T15:51:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.812+0000 7fdd18bd5640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdcf406a050 0x7fdcf406c510 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.812+0000 7fdd18bd5640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd1415c6d0 0x7fdd140b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.812+0000 7fdd18bd5640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd140d0d20 0x7fdd140b49c0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009870 tx=0x7fdd080046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.812+0000 7fdd18bd5640 1 --2- 172.21.15.27:0/1029872258 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd140cad70 0x7fdd140b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.812+0000 7fdd18bd5640 1 -- 172.21.15.27:0/1029872258 >> 172.21.15.27:0/1029872258 conn(0x7fdd140d0200 msgr2=0x7fdd140cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.813+0000 7fdd18bd5640 1 -- 172.21.15.27:0/1029872258 shutdown_connections 2023-08-11T15:51:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:04.813+0000 7fdd18bd5640 1 -- 172.21.15.27:0/1029872258 wait complete. 2023-08-11T15:51:04.843 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-08-11T15:51:05.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.053+0000 7f025a70c640 1 Processor -- start 2023-08-11T15:51:05.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.053+0000 7f025a70c640 1 -- start start 2023-08-11T15:51:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.055+0000 7f025a70c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f02540cc640 0x7f02540ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.055+0000 7f025a70c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540cadd0 0x7f02540ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.055+0000 7f025a70c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02540cb5a0 con 0x7f02540b1d60 2023-08-11T15:51:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.055+0000 7f025a70c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02540cb880 con 0x7f02540cc640 2023-08-11T15:51:05.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.055+0000 7f025a70c640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02540cbb60 con 0x7f02540cadd0 2023-08-11T15:51:05.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.055+0000 7f024bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f02540cc640 0x7f02540ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.055+0000 7f024bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f02540cc640 0x7f02540ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54294/0 (socket says 172.21.15.27:54294) 2023-08-11T15:51:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.055+0000 7f024bfff640 1 -- 172.21.15.27:0/4096373764 learned_addr learned my addr 172.21.15.27:0/4096373764 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.055+0000 7f0258ca9640 1 --2- 172.21.15.27:0/4096373764 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540cadd0 0x7f02540ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.056+0000 7f024bfff640 1 -- 172.21.15.27:0/4096373764 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540cadd0 msgr2=0x7f02540ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.056+0000 7f024bfff640 1 --2- 172.21.15.27:0/4096373764 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540cadd0 0x7f02540ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.056+0000 7f024bfff640 1 -- 172.21.15.27:0/4096373764 >> v1:172.21.15.27:6789/0 conn(0x7f02540b1d60 legacy=0x7f02540b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.056+0000 7f024bfff640 1 -- 172.21.15.27:0/4096373764 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02540cbe40 con 0x7f02540cc640 2023-08-11T15:51:05.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.057+0000 7f024bfff640 1 --2- 172.21.15.27:0/4096373764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cc640 0x7f02540ca890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f023c009870 tx=0x7f023c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870bc81790ce335c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:05.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.057+0000 7f02537fe640 1 -- 172.21.15.27:0/4096373764 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c031640 con 0x7f02540cc640 2023-08-11T15:51:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.057+0000 7f02537fe640 1 -- 172.21.15.27:0/4096373764 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c0317a0 con 0x7f02540cc640 2023-08-11T15:51:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.057+0000 7f02537fe640 1 -- 172.21.15.27:0/4096373764 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c031b30 con 0x7f02540cc640 2023-08-11T15:51:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.058+0000 7f025a70c640 1 -- 172.21.15.27:0/4096373764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cc640 msgr2=0x7f02540ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.058+0000 7f025a70c640 1 --2- 172.21.15.27:0/4096373764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cc640 0x7f02540ca890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f023c009870 tx=0x7f023c02b8f0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.058+0000 7f025a70c640 1 -- 172.21.15.27:0/4096373764 shutdown_connections 2023-08-11T15:51:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.058+0000 7f025a70c640 1 --2- 172.21.15.27:0/4096373764 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540cadd0 0x7f02540ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.058+0000 7f025a70c640 1 --2- 172.21.15.27:0/4096373764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cc640 0x7f02540ca890 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.058+0000 7f025a70c640 1 -- 172.21.15.27:0/4096373764 >> 172.21.15.27:0/4096373764 conn(0x7f025406f440 msgr2=0x7f025406f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.058+0000 7f025a70c640 1 -- 172.21.15.27:0/4096373764 shutdown_connections 2023-08-11T15:51:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.058+0000 7f025a70c640 1 -- 172.21.15.27:0/4096373764 wait complete. 2023-08-11T15:51:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.060+0000 7f025a70c640 1 Processor -- start 2023-08-11T15:51:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.060+0000 7f025a70c640 1 -- start start 2023-08-11T15:51:05.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.061+0000 7f025a70c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540b1d60 0x7f0254160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:05.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.061+0000 7f025a70c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cadd0 0x7f02540c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:05.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.061+0000 7f025a70c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540cc640 0x7f02540b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:05.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.061+0000 7f025a70c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541eaa00 con 0x7f02540cadd0 2023-08-11T15:51:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.061+0000 7f025a70c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541eace0 con 0x7f02540cc640 2023-08-11T15:51:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.061+0000 7f025a70c640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02541eafc0 con 0x7f02540b1d60 2023-08-11T15:51:05.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.061+0000 7f024bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cadd0 0x7f02540c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:05.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.061+0000 7f024bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cadd0 0x7f02540c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54308/0 (socket says 172.21.15.27:54308) 2023-08-11T15:51:05.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.061+0000 7f024bfff640 1 -- 172.21.15.27:0/1704091923 learned_addr learned my addr 172.21.15.27:0/1704091923 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:05.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.061+0000 7f0253fff640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540b1d60 0x7f0254160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f0258ca9640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540cc640 0x7f02540b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f024bfff640 1 -- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540b1d60 msgr2=0x7f0254160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f024bfff640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540b1d60 0x7f0254160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f024bfff640 1 -- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540cc640 msgr2=0x7f02540b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f024bfff640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540cc640 0x7f02540b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f024bfff640 1 -- 172.21.15.27:0/1704091923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023c0094d0 con 0x7f02540cadd0 2023-08-11T15:51:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f0258ca9640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540cc640 0x7f02540b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f024bfff640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cadd0 0x7f02540c35b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f023c02b930 tx=0x7f023c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f02517fa640 1 -- 172.21.15.27:0/1704091923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c03bc70 con 0x7f02540cadd0 2023-08-11T15:51:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f02517fa640 1 -- 172.21.15.27:0/1704091923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c03be10 con 0x7f02540cadd0 2023-08-11T15:51:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f02517fa640 1 -- 172.21.15.27:0/1704091923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c045630 con 0x7f02540cadd0 2023-08-11T15:51:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.062+0000 7f025a70c640 1 -- 172.21.15.27:0/1704091923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02540b4920 con 0x7f02540cadd0 2023-08-11T15:51:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.063+0000 7f025a70c640 1 -- 172.21.15.27:0/1704091923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02540c3af0 con 0x7f02540cadd0 2023-08-11T15:51:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.064+0000 7f025a70c640 1 -- 172.21.15.27:0/1704091923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025404e420 con 0x7f02540cadd0 2023-08-11T15:51:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.065+0000 7f02517fa640 1 -- 172.21.15.27:0/1704091923 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f023c02f050 con 0x7f02540cadd0 2023-08-11T15:51:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.066+0000 7f02517fa640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f023406a050 0x7f023406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.066+0000 7f02517fa640 1 -- 172.21.15.27:0/1704091923 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f023c0aa940 con 0x7f02540cadd0 2023-08-11T15:51:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.066+0000 7f0253fff640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f023406a050 0x7f023406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.067+0000 7f0253fff640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f023406a050 0x7f023406c510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0254163e30 tx=0x7f0244009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:05.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.071+0000 7f02517fa640 1 -- 172.21.15.27:0/1704091923 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f023c075d50 con 0x7f02540cadd0 2023-08-11T15:51:05.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.273+0000 7f025a70c640 1 -- 172.21.15.27:0/1704091923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f02541eab40 con 0x7f02540cadd0 2023-08-11T15:51:05.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.809+0000 7f02517fa640 1 -- 172.21.15.27:0/1704091923 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f02541eab40 con 0x7f02540cadd0 2023-08-11T15:51:05.811 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 5 allow_ec_overwrites to true 2023-08-11T15:51:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.821+0000 7f025a70c640 1 -- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f023406a050 msgr2=0x7f023406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.821+0000 7f025a70c640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f023406a050 0x7f023406c510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0254163e30 tx=0x7f0244009040 comp rx=0 tx=0).stop 2023-08-11T15:51:05.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.821+0000 7f025a70c640 1 -- 172.21.15.27:0/1704091923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cadd0 msgr2=0x7f02540c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:05.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.821+0000 7f025a70c640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cadd0 0x7f02540c35b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f023c02b930 tx=0x7f023c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.828+0000 7f025a70c640 1 -- 172.21.15.27:0/1704091923 shutdown_connections 2023-08-11T15:51:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.828+0000 7f025a70c640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f023406a050 0x7f023406c510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.828+0000 7f025a70c640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540cc640 0x7f02540b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.828+0000 7f025a70c640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02540cadd0 0x7f02540c35b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.828+0000 7f025a70c640 1 --2- 172.21.15.27:0/1704091923 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540b1d60 0x7f0254160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.828+0000 7f025a70c640 1 -- 172.21.15.27:0/1704091923 >> 172.21.15.27:0/1704091923 conn(0x7f025406f440 msgr2=0x7f02540cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:05.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.829+0000 7f025a70c640 1 -- 172.21.15.27:0/1704091923 shutdown_connections 2023-08-11T15:51:05.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:05.829+0000 7f025a70c640 1 -- 172.21.15.27:0/1704091923 wait complete. 2023-08-11T15:51:05.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-08-11T15:51:05.888 INFO:tasks.ceph.mds.b.smithi119.stderr:2023-08-11T15:51:05.885+0000 7f599d7c5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-11T15:51:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.065+0000 7feb4efb5640 1 Processor -- start 2023-08-11T15:51:06.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.065+0000 7feb4efb5640 1 -- start start 2023-08-11T15:51:06.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.066+0000 7feb4efb5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feb48160d20 0x7feb480cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:06.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.066+0000 7feb4efb5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb4815d3b0 0x7feb480c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:06.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.066+0000 7feb4efb5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4815da30 con 0x7feb480cc350 2023-08-11T15:51:06.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.066+0000 7feb4efb5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb480cc800 con 0x7feb48160d20 2023-08-11T15:51:06.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.066+0000 7feb4efb5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4815d840 con 0x7feb4815d3b0 2023-08-11T15:51:06.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.067+0000 7feb4cd51640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feb48160d20 0x7feb480cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:06.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.067+0000 7feb4cd51640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feb48160d20 0x7feb480cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54324/0 (socket says 172.21.15.27:54324) 2023-08-11T15:51:06.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.067+0000 7feb4cd51640 1 -- 172.21.15.27:0/452783256 learned_addr learned my addr 172.21.15.27:0/452783256 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:06.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.067+0000 7feb4cd51640 1 -- 172.21.15.27:0/452783256 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb4815d3b0 msgr2=0x7feb480c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:06.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.067+0000 7feb4cd51640 1 --2- 172.21.15.27:0/452783256 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb4815d3b0 0x7feb480c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:06.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.067+0000 7feb4cd51640 1 -- 172.21.15.27:0/452783256 >> v1:172.21.15.119:6789/0 conn(0x7feb480cc350 legacy=0x7feb4815cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:06.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.067+0000 7feb4cd51640 1 -- 172.21.15.27:0/452783256 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb480ca570 con 0x7feb48160d20 2023-08-11T15:51:06.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.068+0000 7feb4cd51640 1 --2- 172.21.15.27:0/452783256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb48160d20 0x7feb480cbe10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb38009850 tx=0x7feb3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba860baa82d0bac server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:06.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.069+0000 7feb477fe640 1 -- 172.21.15.27:0/452783256 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38004230 con 0x7feb48160d20 2023-08-11T15:51:06.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.069+0000 7feb477fe640 1 -- 172.21.15.27:0/452783256 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb38004390 con 0x7feb48160d20 2023-08-11T15:51:06.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.069+0000 7feb477fe640 1 -- 172.21.15.27:0/452783256 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38031830 con 0x7feb48160d20 2023-08-11T15:51:06.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.069+0000 7feb4efb5640 1 -- 172.21.15.27:0/452783256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb48160d20 msgr2=0x7feb480cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:06.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.069+0000 7feb4efb5640 1 --2- 172.21.15.27:0/452783256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb48160d20 0x7feb480cbe10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb38009850 tx=0x7feb3802b8d0 comp rx=0 tx=0).stop 2023-08-11T15:51:06.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.069+0000 7feb4efb5640 1 -- 172.21.15.27:0/452783256 shutdown_connections 2023-08-11T15:51:06.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.069+0000 7feb4efb5640 1 --2- 172.21.15.27:0/452783256 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb4815d3b0 0x7feb480c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:06.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.069+0000 7feb4efb5640 1 --2- 172.21.15.27:0/452783256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb48160d20 0x7feb480cbe10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:06.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.069+0000 7feb4efb5640 1 -- 172.21.15.27:0/452783256 >> 172.21.15.27:0/452783256 conn(0x7feb480afb70 msgr2=0x7feb480aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:06.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.070+0000 7feb4efb5640 1 -- 172.21.15.27:0/452783256 shutdown_connections 2023-08-11T15:51:06.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.070+0000 7feb4efb5640 1 -- 172.21.15.27:0/452783256 wait complete. 2023-08-11T15:51:06.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.071+0000 7feb4efb5640 1 Processor -- start 2023-08-11T15:51:06.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.071+0000 7feb4efb5640 1 -- start start 2023-08-11T15:51:06.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.072+0000 7feb4efb5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb480cc350 0x7feb480c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:06.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.072+0000 7feb4efb5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb4815d3b0 0x7feb480c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:06.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.072+0000 7feb4efb5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb48160d20 0x7feb481463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:06.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.072+0000 7feb4efb5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb480c31b0 con 0x7feb4815d3b0 2023-08-11T15:51:06.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.072+0000 7feb4efb5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb481468e0 con 0x7feb48160d20 2023-08-11T15:51:06.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.072+0000 7feb4efb5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb480c3450 con 0x7feb480cc350 2023-08-11T15:51:06.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.073+0000 7feb47fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb4815d3b0 0x7feb480c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:06.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.073+0000 7feb47fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb4815d3b0 0x7feb480c2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54328/0 (socket says 172.21.15.27:54328) 2023-08-11T15:51:06.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.073+0000 7feb47fff640 1 -- 172.21.15.27:0/1538105821 learned_addr learned my addr 172.21.15.27:0/1538105821 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:06.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.073+0000 7feb4cd51640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb480cc350 0x7feb480c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:06.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.073+0000 7feb4d552640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb48160d20 0x7feb481463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:06.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.073+0000 7feb47fff640 1 -- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb480cc350 msgr2=0x7feb480c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:06.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.073+0000 7feb47fff640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb480cc350 0x7feb480c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.073+0000 7feb47fff640 1 -- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb48160d20 msgr2=0x7feb481463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.073+0000 7feb47fff640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb48160d20 0x7feb481463a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.073+0000 7feb47fff640 1 -- 172.21.15.27:0/1538105821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb38009460 con 0x7feb4815d3b0 2023-08-11T15:51:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.074+0000 7feb47fff640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb4815d3b0 0x7feb480c2c70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00ba50 tx=0x7feb3c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.074+0000 7feb457fa640 1 -- 172.21.15.27:0/1538105821 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00cc10 con 0x7feb4815d3b0 2023-08-11T15:51:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.074+0000 7feb457fa640 1 -- 172.21.15.27:0/1538105821 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c00cd70 con 0x7feb4815d3b0 2023-08-11T15:51:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.074+0000 7feb457fa640 1 -- 172.21.15.27:0/1538105821 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c014680 con 0x7feb4815d3b0 2023-08-11T15:51:06.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.074+0000 7feb4efb5640 1 -- 172.21.15.27:0/1538105821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb480c3730 con 0x7feb4815d3b0 2023-08-11T15:51:06.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.074+0000 7feb4efb5640 1 -- 172.21.15.27:0/1538105821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb480c3cf0 con 0x7feb4815d3b0 2023-08-11T15:51:06.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.075+0000 7feb4efb5640 1 -- 172.21.15.27:0/1538105821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb480cab10 con 0x7feb4815d3b0 2023-08-11T15:51:06.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.076+0000 7feb457fa640 1 -- 172.21.15.27:0/1538105821 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feb3c01d3b0 con 0x7feb4815d3b0 2023-08-11T15:51:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.077+0000 7feb457fa640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7feb3406a050 0x7feb3406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.078+0000 7feb457fa640 1 -- 172.21.15.27:0/1538105821 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6781+0+0 (secure 0 0 0) 0x7feb3c088970 con 0x7feb4815d3b0 2023-08-11T15:51:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.078+0000 7feb4cd51640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7feb3406a050 0x7feb3406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:06.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.079+0000 7feb4cd51640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7feb3406a050 0x7feb3406c510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb38035000 tx=0x7feb38008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.082+0000 7feb457fa640 1 -- 172.21.15.27:0/1538105821 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7feb480cab10 con 0x7feb4815d3b0 2023-08-11T15:51:06.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.321+0000 7feb4efb5640 1 -- 172.21.15.27:0/1538105821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7feb480caae0 con 0x7feb4815d3b0 2023-08-11T15:51:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.854+0000 7feb457fa640 1 -- 172.21.15.27:0/1538105821 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7feb3c019020 con 0x7feb4815d3b0 2023-08-11T15:51:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:added data pool 5 to fsmap 2023-08-11T15:51:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 -- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7feb3406a050 msgr2=0x7feb3406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7feb3406a050 0x7feb3406c510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb38035000 tx=0x7feb38008030 comp rx=0 tx=0).stop 2023-08-11T15:51:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 -- 172.21.15.27:0/1538105821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb4815d3b0 msgr2=0x7feb480c2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb4815d3b0 0x7feb480c2c70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00ba50 tx=0x7feb3c00bf10 comp rx=0 tx=0).stop 2023-08-11T15:51:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 -- 172.21.15.27:0/1538105821 shutdown_connections 2023-08-11T15:51:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7feb3406a050 0x7feb3406c510 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb48160d20 0x7feb481463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb4815d3b0 0x7feb480c2c70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 --2- 172.21.15.27:0/1538105821 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb480cc350 0x7feb480c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 -- 172.21.15.27:0/1538105821 >> 172.21.15.27:0/1538105821 conn(0x7feb480afb70 msgr2=0x7feb4815e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 -- 172.21.15.27:0/1538105821 shutdown_connections 2023-08-11T15:51:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:06.858+0000 7feb4efb5640 1 -- 172.21.15.27:0/1538105821 wait complete. 2023-08-11T15:51:06.881 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:51:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.119+0000 7f70c2922640 1 Processor -- start 2023-08-11T15:51:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.120+0000 7f70c2922640 1 -- start start 2023-08-11T15:51:07.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.121+0000 7f70c2922640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc15a520 0x7f70bc06f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.121+0000 7f70c2922640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70bc15ea70 0x7f70bc06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.121+0000 7f70c2922640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc0cae20 con 0x7f70bc06fcf0 2023-08-11T15:51:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.121+0000 7f70c2922640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70bc0b6100 con 0x7f70bc15ea70 2023-08-11T15:51:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.121+0000 7f70c2922640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc15df90 con 0x7f70bc15a520 2023-08-11T15:51:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.122+0000 7f70c0ebf640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f70bc06fcf0 0x7f70bc0ce000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35388/0 (socket says 172.21.15.27:35388) 2023-08-11T15:51:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.122+0000 7f70c0ebf640 1 -- 172.21.15.27:0/761834726 learned_addr learned my addr 172.21.15.27:0/761834726 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.122+0000 7f70bbfff640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc15a520 0x7f70bc06f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.122+0000 7f70bb7fe640 1 --2- 172.21.15.27:0/761834726 >> v2:172.21.15.119:3300/0 conn(0x7f70bc15ea70 0x7f70bc06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.122+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961497573 0 0) 0x7f70bc0cae20 con 0x7f70bc06fcf0 2023-08-11T15:51:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.122+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70a4003520 con 0x7f70bc06fcf0 2023-08-11T15:51:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 92103449 0 0) 0x7f70a4003520 con 0x7f70bc06fcf0 2023-08-11T15:51:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc15a520 msgr2=0x7f70bc06f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc15a520 0x7f70bc06f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 >> v2:172.21.15.119:3300/0 conn(0x7f70bc15ea70 msgr2=0x7f70bc06f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 --2- 172.21.15.27:0/761834726 >> v2:172.21.15.119:3300/0 conn(0x7f70bc15ea70 0x7f70bc06f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc15e270 con 0x7f70bc06fcf0 2023-08-11T15:51:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f70b40019f0 con 0x7f70bc06fcf0 2023-08-11T15:51:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 >> v1:172.21.15.27:6789/0 conn(0x7f70bc06fcf0 legacy=0x7f70bc0ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc15a520 0x7f70a4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70a4003ef0 0x7f70a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a4004d00 0x7f70a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc15df90 con 0x7f70a4003ef0 2023-08-11T15:51:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a400fd90 con 0x7f70a4004d00 2023-08-11T15:51:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a400fed0 con 0x7f70bc15a520 2023-08-11T15:51:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70bb7fe640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70a4003ef0 0x7f70a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70bbfff640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc15a520 0x7f70a4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.123+0000 7f70c0ebf640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a4004d00 0x7f70a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.124+0000 7f70bb7fe640 1 -- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc15a520 msgr2=0x7f70a4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.124+0000 7f70bb7fe640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc15a520 0x7f70a4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.124+0000 7f70bb7fe640 1 -- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a4004d00 msgr2=0x7f70a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.124+0000 7f70bb7fe640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a4004d00 0x7f70a4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.124+0000 7f70bb7fe640 1 -- 172.21.15.27:0/761834726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4003520 con 0x7f70a4003ef0 2023-08-11T15:51:07.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.125+0000 7f70bb7fe640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70a4003ef0 0x7f70a40047c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009ac0 tx=0x7f70a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c2609dcba39054 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:07.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.125+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b002c6d0 con 0x7f70a4003ef0 2023-08-11T15:51:07.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.125+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b002c830 con 0x7f70a4003ef0 2023-08-11T15:51:07.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.125+0000 7f70baffd640 1 -- 172.21.15.27:0/761834726 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b002cbc0 con 0x7f70a4003ef0 2023-08-11T15:51:07.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.126+0000 7f70c2922640 1 -- 172.21.15.27:0/761834726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70a4003ef0 msgr2=0x7f70a40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.126+0000 7f70c2922640 1 --2- 172.21.15.27:0/761834726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70a4003ef0 0x7f70a40047c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009ac0 tx=0x7f70a40057f0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.126+0000 7f70c0ebf640 1 -- 172.21.15.27:0/761834726 reap_dead start 2023-08-11T15:51:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.126+0000 7f70c2922640 1 -- 172.21.15.27:0/761834726 shutdown_connections 2023-08-11T15:51:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.126+0000 7f70c2922640 1 -- 172.21.15.27:0/761834726 >> 172.21.15.27:0/761834726 conn(0x7f70bc06dad0 msgr2=0x7f70bc06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.126+0000 7f70c2922640 1 -- 172.21.15.27:0/761834726 shutdown_connections 2023-08-11T15:51:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.126+0000 7f70c2922640 1 -- 172.21.15.27:0/761834726 wait complete. 2023-08-11T15:51:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.128+0000 7f70c2922640 1 Processor -- start 2023-08-11T15:51:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.128+0000 7f70c2922640 1 -- start start 2023-08-11T15:51:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70c2922640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a4003ef0 0x7f70bc1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70c2922640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70bc158450 0x7f70bc14f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70c2922640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc0afeb0 0x7f70bc0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70c2922640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc0b08a0 con 0x7f70bc158450 2023-08-11T15:51:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70c2922640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc157d00 con 0x7f70a4003ef0 2023-08-11T15:51:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70c2922640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc0b4270 con 0x7f70bc0afeb0 2023-08-11T15:51:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70bb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70bc158450 0x7f70bc14f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70bb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70bc158450 0x7f70bc14f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54344/0 (socket says 172.21.15.27:54344) 2023-08-11T15:51:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70bb7fe640 1 -- 172.21.15.27:0/1714007544 learned_addr learned my addr 172.21.15.27:0/1714007544 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70bbfff640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a4003ef0 0x7f70bc1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70c0ebf640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc0afeb0 0x7f70bc0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70bb7fe640 1 -- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc0afeb0 msgr2=0x7f70bc0b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70bb7fe640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc0afeb0 0x7f70bc0b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70bb7fe640 1 -- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a4003ef0 msgr2=0x7f70bc1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70bb7fe640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a4003ef0 0x7f70bc1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.129+0000 7f70bb7fe640 1 -- 172.21.15.27:0/1714007544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc0b4550 con 0x7f70bc158450 2023-08-11T15:51:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.130+0000 7f70bb7fe640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70bc158450 0x7f70bc14f930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f70b002c2d0 tx=0x7f70b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.130+0000 7f70b8ff9640 1 -- 172.21.15.27:0/1714007544 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0003810 con 0x7f70bc158450 2023-08-11T15:51:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.130+0000 7f70b8ff9640 1 -- 172.21.15.27:0/1714007544 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b0003970 con 0x7f70bc158450 2023-08-11T15:51:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.130+0000 7f70b8ff9640 1 -- 172.21.15.27:0/1714007544 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0003d20 con 0x7f70bc158450 2023-08-11T15:51:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.130+0000 7f70c2922640 1 -- 172.21.15.27:0/1714007544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70bc0b4830 con 0x7f70bc158450 2023-08-11T15:51:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.130+0000 7f70c2922640 1 -- 172.21.15.27:0/1714007544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70bc0b4df0 con 0x7f70bc158450 2023-08-11T15:51:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.131+0000 7f70c2922640 1 -- 172.21.15.27:0/1714007544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70bc073990 con 0x7f70bc158450 2023-08-11T15:51:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.132+0000 7f70b8ff9640 1 -- 172.21.15.27:0/1714007544 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f70b0040660 con 0x7f70bc158450 2023-08-11T15:51:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.133+0000 7f70b8ff9640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70a806a080 0x7f70a806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.133+0000 7f70b8ff9640 1 -- 172.21.15.27:0/1714007544 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f70b00aacf0 con 0x7f70bc158450 2023-08-11T15:51:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.134+0000 7f70bbfff640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70a806a080 0x7f70a806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.135+0000 7f70bbfff640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70a806a080 0x7f70a806c540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70bc159fd0 tx=0x7f70ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.138+0000 7f70b8ff9640 1 -- 172.21.15.27:0/1714007544 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f70b00760e0 con 0x7f70bc158450 2023-08-11T15:51:07.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.379+0000 7f70c2922640 1 -- 172.21.15.27:0/1714007544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70bc0cdd60 con 0x7f70bc158450 2023-08-11T15:51:07.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.380+0000 7f70b8ff9640 1 -- 172.21.15.27:0/1714007544 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f70b00044e0 con 0x7f70bc158450 2023-08-11T15:51:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:07.382 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:51:01.220970+0000","modified":"2023-08-11T15:51:06.831869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4315":{"gid":4315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-11T15:51:07.383 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-08-11T15:51:07.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.383+0000 7f70c2922640 1 -- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70a806a080 msgr2=0x7f70a806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.383+0000 7f70c2922640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70a806a080 0x7f70a806c540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70bc159fd0 tx=0x7f70ac008040 comp rx=0 tx=0).stop 2023-08-11T15:51:07.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.384+0000 7f70c2922640 1 -- 172.21.15.27:0/1714007544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70bc158450 msgr2=0x7f70bc14f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.384+0000 7f70c2922640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70bc158450 0x7f70bc14f930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f70b002c2d0 tx=0x7f70b0004730 comp rx=0 tx=0).stop 2023-08-11T15:51:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.384+0000 7f70c2922640 1 -- 172.21.15.27:0/1714007544 shutdown_connections 2023-08-11T15:51:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.384+0000 7f70c2922640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70a806a080 0x7f70a806c540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.384+0000 7f70c2922640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70bc0afeb0 0x7f70bc0b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.384+0000 7f70c2922640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70bc158450 0x7f70bc14f930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.384+0000 7f70c2922640 1 --2- 172.21.15.27:0/1714007544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70a4003ef0 0x7f70bc1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.384+0000 7f70c2922640 1 -- 172.21.15.27:0/1714007544 >> 172.21.15.27:0/1714007544 conn(0x7f70bc06dad0 msgr2=0x7f70bc0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.384+0000 7f70c2922640 1 -- 172.21.15.27:0/1714007544 shutdown_connections 2023-08-11T15:51:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.384+0000 7f70c2922640 1 -- 172.21.15.27:0/1714007544 wait complete. 2023-08-11T15:51:07.403 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.625+0000 7f9854df3640 1 Processor -- start 2023-08-11T15:51:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.625+0000 7f9854df3640 1 -- start start 2023-08-11T15:51:07.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f9854df3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9850156e50 0x7f985015a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f9854df3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f98500cbb70 0x7f98500ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f9854df3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f98500caef0 0x7f985015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f9854df3640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98500ca460 con 0x7f9850156e50 2023-08-11T15:51:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f9854df3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98500cb380 con 0x7f98500cbb70 2023-08-11T15:51:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f9854df3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9850070470 con 0x7f98500caef0 2023-08-11T15:51:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f984e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9850156e50 0x7f985015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f984e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9850156e50 0x7f985015a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54348/0 (socket says 172.21.15.27:54348) 2023-08-11T15:51:07.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f984e59c640 1 -- 172.21.15.27:0/2792130387 learned_addr learned my addr 172.21.15.27:0/2792130387 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:07.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f9845d9b640 1 --2- 172.21.15.27:0/2792130387 >> v2:172.21.15.119:3300/0 conn(0x7f98500cbb70 0x7f98500ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.627+0000 7f984ed9d640 1 --2- 172.21.15.27:0/2792130387 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f98500caef0 0x7f985015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.628+0000 7f984e59c640 1 -- 172.21.15.27:0/2792130387 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f98500caef0 msgr2=0x7f985015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.628+0000 7f984e59c640 1 --2- 172.21.15.27:0/2792130387 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f98500caef0 0x7f985015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.628+0000 7f984e59c640 1 -- 172.21.15.27:0/2792130387 >> v2:172.21.15.119:3300/0 conn(0x7f98500cbb70 msgr2=0x7f98500ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.628+0000 7f984e59c640 1 --2- 172.21.15.27:0/2792130387 >> v2:172.21.15.119:3300/0 conn(0x7f98500cbb70 0x7f98500ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.628+0000 7f984e59c640 1 -- 172.21.15.27:0/2792130387 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98500cbfb0 con 0x7f9850156e50 2023-08-11T15:51:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.628+0000 7f984ed9d640 1 --2- 172.21.15.27:0/2792130387 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f98500caef0 0x7f985015ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:07.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.629+0000 7f984e59c640 1 --2- 172.21.15.27:0/2792130387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9850156e50 0x7f985015a180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f985006ff00 tx=0x7f984002bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1b15a791ec59be server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:07.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.629+0000 7f984dd9b640 1 -- 172.21.15.27:0/2792130387 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984002cac0 con 0x7f9850156e50 2023-08-11T15:51:07.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.629+0000 7f984dd9b640 1 -- 172.21.15.27:0/2792130387 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984002cc20 con 0x7f9850156e50 2023-08-11T15:51:07.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.629+0000 7f984dd9b640 1 -- 172.21.15.27:0/2792130387 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98400343f0 con 0x7f9850156e50 2023-08-11T15:51:07.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.630+0000 7f9854df3640 1 -- 172.21.15.27:0/2792130387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9850156e50 msgr2=0x7f985015a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.630+0000 7f9854df3640 1 --2- 172.21.15.27:0/2792130387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9850156e50 0x7f985015a180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f985006ff00 tx=0x7f984002bf20 comp rx=0 tx=0).stop 2023-08-11T15:51:07.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.630+0000 7f9854df3640 1 -- 172.21.15.27:0/2792130387 shutdown_connections 2023-08-11T15:51:07.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.630+0000 7f9854df3640 1 --2- 172.21.15.27:0/2792130387 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f98500caef0 0x7f985015ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.630+0000 7f9854df3640 1 --2- 172.21.15.27:0/2792130387 >> v2:172.21.15.119:3300/0 conn(0x7f98500cbb70 0x7f98500ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.630+0000 7f9854df3640 1 --2- 172.21.15.27:0/2792130387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9850156e50 0x7f985015a180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.630+0000 7f9854df3640 1 -- 172.21.15.27:0/2792130387 >> 172.21.15.27:0/2792130387 conn(0x7f98500afb70 msgr2=0x7f98500aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.630+0000 7f9854df3640 1 -- 172.21.15.27:0/2792130387 shutdown_connections 2023-08-11T15:51:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.630+0000 7f9854df3640 1 -- 172.21.15.27:0/2792130387 wait complete. 2023-08-11T15:51:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.632+0000 7f9854df3640 1 Processor -- start 2023-08-11T15:51:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.632+0000 7f9854df3640 1 -- start start 2023-08-11T15:51:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.633+0000 7f9854df3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98500caef0 0x7f9850151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.633+0000 7f9854df3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98500cbb70 0x7f98500c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.633+0000 7f9854df3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9850156e50 0x7f98500b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.633+0000 7f9854df3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98500c1ad0 con 0x7f98500caef0 2023-08-11T15:51:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.633+0000 7f9854df3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9850151ad0 con 0x7f98500cbb70 2023-08-11T15:51:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.633+0000 7f9854df3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98501518b0 con 0x7f9850156e50 2023-08-11T15:51:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.633+0000 7f984e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98500caef0 0x7f9850151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.634+0000 7f984e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98500caef0 0x7f9850151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54354/0 (socket says 172.21.15.27:54354) 2023-08-11T15:51:07.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.634+0000 7f984e59c640 1 -- 172.21.15.27:0/886614077 learned_addr learned my addr 172.21.15.27:0/886614077 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:07.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.634+0000 7f9845d9b640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98500cbb70 0x7f98500c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.634+0000 7f984ed9d640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9850156e50 0x7f98500b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.634+0000 7f984e59c640 1 -- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9850156e50 msgr2=0x7f98500b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.634+0000 7f984e59c640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9850156e50 0x7f98500b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.634+0000 7f984e59c640 1 -- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98500cbb70 msgr2=0x7f98500c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.634+0000 7f984e59c640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98500cbb70 0x7f98500c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.634+0000 7f984e59c640 1 -- 172.21.15.27:0/886614077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9840009b30 con 0x7f98500caef0 2023-08-11T15:51:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.634+0000 7f984e59c640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98500caef0 0x7f9850151370 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9840000f80 tx=0x7f9840004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.635+0000 7f98477fe640 1 -- 172.21.15.27:0/886614077 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840009510 con 0x7f98500caef0 2023-08-11T15:51:07.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.635+0000 7f98477fe640 1 -- 172.21.15.27:0/886614077 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984003e450 con 0x7f98500caef0 2023-08-11T15:51:07.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.635+0000 7f9854df3640 1 -- 172.21.15.27:0/886614077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98500c1ce0 con 0x7f98500caef0 2023-08-11T15:51:07.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.635+0000 7f98477fe640 1 -- 172.21.15.27:0/886614077 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984003e800 con 0x7f98500caef0 2023-08-11T15:51:07.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.635+0000 7f9854df3640 1 -- 172.21.15.27:0/886614077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9850151ce0 con 0x7f98500caef0 2023-08-11T15:51:07.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.636+0000 7f9854df3640 1 -- 172.21.15.27:0/886614077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9850073990 con 0x7f98500caef0 2023-08-11T15:51:07.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.637+0000 7f98477fe640 1 -- 172.21.15.27:0/886614077 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f984003e960 con 0x7f98500caef0 2023-08-11T15:51:07.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.638+0000 7f98477fe640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f983006a050 0x7f983006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:07.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.638+0000 7f98477fe640 1 -- 172.21.15.27:0/886614077 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f98400ab750 con 0x7f98500caef0 2023-08-11T15:51:07.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.639+0000 7f9845d9b640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f983006a050 0x7f983006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:07.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.640+0000 7f9845d9b640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f983006a050 0x7f983006c510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98501f4f30 tx=0x7f9838008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:07.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.642+0000 7f98477fe640 1 -- 172.21.15.27:0/886614077 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9840076ab0 con 0x7f98500caef0 2023-08-11T15:51:07.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.837+0000 7f9854df3640 1 -- 172.21.15.27:0/886614077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98500cb8d0 con 0x7f98500caef0 2023-08-11T15:51:07.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.843+0000 7f98477fe640 1 -- 172.21.15.27:0/886614077 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7f984007a0d0 con 0x7f98500caef0 2023-08-11T15:51:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:07.845 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:07.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 -- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f983006a050 msgr2=0x7f983006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f983006a050 0x7f983006c510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98501f4f30 tx=0x7f9838008040 comp rx=0 tx=0).stop 2023-08-11T15:51:07.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 -- 172.21.15.27:0/886614077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98500caef0 msgr2=0x7f9850151370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:07.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98500caef0 0x7f9850151370 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9840000f80 tx=0x7f9840004a40 comp rx=0 tx=0).stop 2023-08-11T15:51:07.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 -- 172.21.15.27:0/886614077 shutdown_connections 2023-08-11T15:51:07.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f983006a050 0x7f983006c510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9850156e50 0x7f98500b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98500cbb70 0x7f98500c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 --2- 172.21.15.27:0/886614077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98500caef0 0x7f9850151370 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:07.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 -- 172.21.15.27:0/886614077 >> 172.21.15.27:0/886614077 conn(0x7f98500afb70 msgr2=0x7f98500acd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:07.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 -- 172.21.15.27:0/886614077 shutdown_connections 2023-08-11T15:51:07.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:07.846+0000 7f9854df3640 1 -- 172.21.15.27:0/886614077 wait complete. 2023-08-11T15:51:07.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.086+0000 7f9a1e1cf640 1 Processor -- start 2023-08-11T15:51:08.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.086+0000 7f9a1e1cf640 1 -- start start 2023-08-11T15:51:08.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.087+0000 7f9a1e1cf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a1815c620 0x7f9a180b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.087+0000 7f9a1e1cf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9a180cfb00 0x7f9a180d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.087+0000 7f9a1e1cf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a180cda30 con 0x7f9a180b82f0 2023-08-11T15:51:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.087+0000 7f9a1e1cf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a180ccc10 con 0x7f9a180cfb00 2023-08-11T15:51:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.087+0000 7f9a1e1cf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a18161110 con 0x7f9a1815c620 2023-08-11T15:51:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.088+0000 7f9a16ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9a180b82f0 0x7f9a180cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35394/0 (socket says 172.21.15.27:35394) 2023-08-11T15:51:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.088+0000 7f9a16ffd640 1 -- 172.21.15.27:0/2704767009 learned_addr learned my addr 172.21.15.27:0/2704767009 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.088+0000 7f9a17fff640 1 --2- 172.21.15.27:0/2704767009 >> v2:172.21.15.119:3300/0 conn(0x7f9a180cfb00 0x7f9a180d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.088+0000 7f9a177fe640 1 --2- 172.21.15.27:0/2704767009 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a1815c620 0x7f9a180b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.088+0000 7f9a167fc640 1 -- 172.21.15.27:0/2704767009 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006838749 0 0) 0x7f9a180cda30 con 0x7f9a180b82f0 2023-08-11T15:51:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.088+0000 7f9a167fc640 1 -- 172.21.15.27:0/2704767009 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a04003520 con 0x7f9a180b82f0 2023-08-11T15:51:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.089+0000 7f9a17fff640 1 -- 172.21.15.27:0/2704767009 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a1815c620 msgr2=0x7f9a180b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.089+0000 7f9a17fff640 1 --2- 172.21.15.27:0/2704767009 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a1815c620 0x7f9a180b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.089+0000 7f9a167fc640 1 -- 172.21.15.27:0/2704767009 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522590390 0 0) 0x7f9a04003520 con 0x7f9a180b82f0 2023-08-11T15:51:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.089+0000 7f9a17fff640 1 -- 172.21.15.27:0/2704767009 >> v1:172.21.15.27:6789/0 conn(0x7f9a180b82f0 legacy=0x7f9a180cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.089+0000 7f9a17fff640 1 -- 172.21.15.27:0/2704767009 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180cce20 con 0x7f9a180cfb00 2023-08-11T15:51:08.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.090+0000 7f9a17fff640 1 --2- 172.21.15.27:0/2704767009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a180cfb00 0x7f9a180d0b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a100097a0 tx=0x7f9a1002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3363250e169f980e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.090+0000 7f9a167fc640 1 -- 172.21.15.27:0/2704767009 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a000019f0 con 0x7f9a180cfb00 2023-08-11T15:51:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.090+0000 7f9a167fc640 1 -- 172.21.15.27:0/2704767009 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1002e050 con 0x7f9a180cfb00 2023-08-11T15:51:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.090+0000 7f9a167fc640 1 -- 172.21.15.27:0/2704767009 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10030730 con 0x7f9a180cfb00 2023-08-11T15:51:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.091+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2704767009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a180cfb00 msgr2=0x7f9a180d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.091+0000 7f9a1e1cf640 1 --2- 172.21.15.27:0/2704767009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a180cfb00 0x7f9a180d0b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a100097a0 tx=0x7f9a1002b790 comp rx=0 tx=0).stop 2023-08-11T15:51:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.091+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2704767009 shutdown_connections 2023-08-11T15:51:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.091+0000 7f9a1e1cf640 1 --2- 172.21.15.27:0/2704767009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a180cfb00 0x7f9a180d0b00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.091+0000 7f9a1e1cf640 1 --2- 172.21.15.27:0/2704767009 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a1815c620 0x7f9a180b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.091+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2704767009 >> 172.21.15.27:0/2704767009 conn(0x7f9a18071110 msgr2=0x7f9a180714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.091+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2704767009 shutdown_connections 2023-08-11T15:51:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.091+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2704767009 wait complete. 2023-08-11T15:51:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.093+0000 7f9a1e1cf640 1 Processor -- start 2023-08-11T15:51:08.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.093+0000 7f9a1e1cf640 1 -- start start 2023-08-11T15:51:08.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.094+0000 7f9a1e1cf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a180b82f0 0x7f9a181f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.094+0000 7f9a1e1cf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a180cfb00 0x7f9a180cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.094+0000 7f9a1e1cf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a1815c620 0x7f9a180cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.094+0000 7f9a1e1cf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180c9690 con 0x7f9a1815c620 2023-08-11T15:51:08.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.094+0000 7f9a1e1cf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180cc5a0 con 0x7f9a180b82f0 2023-08-11T15:51:08.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.094+0000 7f9a1e1cf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180ca1d0 con 0x7f9a180cfb00 2023-08-11T15:51:08.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a17fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a1815c620 0x7f9a180cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a17fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a1815c620 0x7f9a180cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54366/0 (socket says 172.21.15.27:54366) 2023-08-11T15:51:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a17fff640 1 -- 172.21.15.27:0/2214324873 learned_addr learned my addr 172.21.15.27:0/2214324873 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a177fe640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a180b82f0 0x7f9a181f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a16ffd640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a180cfb00 0x7f9a180cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a17fff640 1 -- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a180cfb00 msgr2=0x7f9a180cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a17fff640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a180cfb00 0x7f9a180cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a17fff640 1 -- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a180b82f0 msgr2=0x7f9a181f9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a17fff640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a180b82f0 0x7f9a181f9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a17fff640 1 -- 172.21.15.27:0/2214324873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a10009400 con 0x7f9a1815c620 2023-08-11T15:51:08.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.095+0000 7f9a17fff640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a1815c620 0x7f9a180cc060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a100098c0 tx=0x7f9a100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:08.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.096+0000 7f9a0ffff640 1 -- 172.21.15.27:0/2214324873 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1003b810 con 0x7f9a1815c620 2023-08-11T15:51:08.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.096+0000 7f9a0ffff640 1 -- 172.21.15.27:0/2214324873 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1003b970 con 0x7f9a1815c620 2023-08-11T15:51:08.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.096+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2214324873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a180ca660 con 0x7f9a1815c620 2023-08-11T15:51:08.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.096+0000 7f9a0ffff640 1 -- 172.21.15.27:0/2214324873 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1003bce0 con 0x7f9a1815c620 2023-08-11T15:51:08.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.096+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2214324873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a180c9b80 con 0x7f9a1815c620 2023-08-11T15:51:08.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.097+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2214324873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a180ca380 con 0x7f9a1815c620 2023-08-11T15:51:08.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.098+0000 7f9a0ffff640 1 -- 172.21.15.27:0/2214324873 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9a10039040 con 0x7f9a1815c620 2023-08-11T15:51:08.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.099+0000 7f9a0ffff640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f99f806a080 0x7f99f806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.099+0000 7f9a0ffff640 1 -- 172.21.15.27:0/2214324873 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9a100a9ed0 con 0x7f9a1815c620 2023-08-11T15:51:08.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.100+0000 7f9a177fe640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f99f806a080 0x7f99f806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.101+0000 7f9a177fe640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f99f806a080 0x7f99f806c540 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009a30 tx=0x7f9a08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.103+0000 7f9a0ffff640 1 -- 172.21.15.27:0/2214324873 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9a180ca380 con 0x7f9a1815c620 2023-08-11T15:51:08.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.299+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2214324873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a180b4710 con 0x7f9a1815c620 2023-08-11T15:51:08.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.302+0000 7f9a0ffff640 1 -- 172.21.15.27:0/2214324873 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7f9a10078d50 con 0x7f9a1815c620 2023-08-11T15:51:08.303 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:08.304 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f99f806a080 msgr2=0x7f99f806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f99f806a080 0x7f99f806c540 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009a30 tx=0x7f9a08008040 comp rx=0 tx=0).stop 2023-08-11T15:51:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2214324873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a1815c620 msgr2=0x7f9a180cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a1815c620 0x7f9a180cc060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a100098c0 tx=0x7f9a100046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2214324873 shutdown_connections 2023-08-11T15:51:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f99f806a080 0x7f99f806c540 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a180cfb00 0x7f9a180cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a1815c620 0x7f9a180cc060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 --2- 172.21.15.27:0/2214324873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a180b82f0 0x7f9a181f9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2214324873 >> 172.21.15.27:0/2214324873 conn(0x7f9a18071110 msgr2=0x7f9a180d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:08.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2214324873 shutdown_connections 2023-08-11T15:51:08.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.304+0000 7f9a1e1cf640 1 -- 172.21.15.27:0/2214324873 wait complete. 2023-08-11T15:51:08.341 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:51:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.576+0000 7f0e8280d640 1 Processor -- start 2023-08-11T15:51:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.577+0000 7f0e8280d640 1 -- start start 2023-08-11T15:51:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.578+0000 7f0e8280d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0e7c0cf400 0x7f0e7c160360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.578+0000 7f0e8280d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c161290 con 0x7f0e7c0cf020 2023-08-11T15:51:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.578+0000 7f0e8280d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c1609f0 con 0x7f0e7c15c9a0 2023-08-11T15:51:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.578+0000 7f0e8280d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7c0b5680 con 0x7f0e7c0cf400 2023-08-11T15:51:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.579+0000 7f0e7bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0e7c0cf020 0x7f0e7c15c290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35398/0 (socket says 172.21.15.27:35398) 2023-08-11T15:51:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.579+0000 7f0e7bfff640 1 -- 172.21.15.27:0/514415540 learned_addr learned my addr 172.21.15.27:0/514415540 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.579+0000 7f0e80daa640 1 --2- 172.21.15.27:0/514415540 >> v2:172.21.15.119:3300/0 conn(0x7f0e7c0cf400 0x7f0e7c160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.579+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368548273 0 0) 0x7f0e7c161290 con 0x7f0e7c0cf020 2023-08-11T15:51:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.579+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e64003520 con 0x7f0e7c0cf020 2023-08-11T15:51:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.579+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231747671 0 0) 0x7f0e7c1609f0 con 0x7f0e7c15c9a0 2023-08-11T15:51:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e7c161290 con 0x7f0e7c15c9a0 2023-08-11T15:51:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2296860079 0 0) 0x7f0e64003520 con 0x7f0e7c0cf020 2023-08-11T15:51:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 >> v2:172.21.15.119:3300/0 conn(0x7f0e7c0cf400 msgr2=0x7f0e7c160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:08.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 --2- 172.21.15.27:0/514415540 >> v2:172.21.15.119:3300/0 conn(0x7f0e7c0cf400 0x7f0e7c160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 >> v1:172.21.15.119:6789/0 conn(0x7f0e7c15c9a0 legacy=0x7f0e7c0d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c160cd0 con 0x7f0e7c0cf020 2023-08-11T15:51:08.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f0e6c0019f0 con 0x7f0e7c0cf020 2023-08-11T15:51:08.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 >> v1:172.21.15.27:6789/0 conn(0x7f0e7c0cf020 legacy=0x7f0e7c15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e64003b00 0x7f0e64003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e640047c0 0x7f0e64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e64005180 0x7f0e6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0b5680 con 0x7f0e64003b00 2023-08-11T15:51:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6400c000 con 0x7f0e640047c0 2023-08-11T15:51:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6400c1c0 con 0x7f0e64005180 2023-08-11T15:51:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7bfff640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e64003b00 0x7f0e64003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7b7fe640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e640047c0 0x7f0e64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7bfff640 1 -- 172.21.15.27:0/514415540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e64005180 msgr2=0x7f0e6400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7bfff640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e64005180 0x7f0e6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.580+0000 7f0e7bfff640 1 -- 172.21.15.27:0/514415540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e640047c0 msgr2=0x7f0e64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.581+0000 7f0e7bfff640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e640047c0 0x7f0e64004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.581+0000 7f0e7bfff640 1 -- 172.21.15.27:0/514415540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e64003520 con 0x7f0e64003b00 2023-08-11T15:51:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.581+0000 7f0e80daa640 1 -- 172.21.15.27:0/514415540 reap_dead start 2023-08-11T15:51:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.581+0000 7f0e7b7fe640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e640047c0 0x7f0e64004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.582+0000 7f0e7bfff640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e64003b00 0x7f0e64003f00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00a7c0 tx=0x7f0e640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcfe7f815d1f5b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.582+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c034630 con 0x7f0e64003b00 2023-08-11T15:51:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.582+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c02f070 con 0x7f0e64003b00 2023-08-11T15:51:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.582+0000 7f0e7affd640 1 -- 172.21.15.27:0/514415540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c034a30 con 0x7f0e64003b00 2023-08-11T15:51:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.583+0000 7f0e8280d640 1 -- 172.21.15.27:0/514415540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e64003b00 msgr2=0x7f0e64003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.583+0000 7f0e8280d640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e64003b00 0x7f0e64003f00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00a7c0 tx=0x7f0e640057f0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.583+0000 7f0e8280d640 1 -- 172.21.15.27:0/514415540 shutdown_connections 2023-08-11T15:51:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.583+0000 7f0e8280d640 1 --2- 172.21.15.27:0/514415540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e64003b00 0x7f0e64003f00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.583+0000 7f0e8280d640 1 -- 172.21.15.27:0/514415540 >> 172.21.15.27:0/514415540 conn(0x7f0e7c06a4a0 msgr2=0x7f0e7c06a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.583+0000 7f0e8280d640 1 -- 172.21.15.27:0/514415540 shutdown_connections 2023-08-11T15:51:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.583+0000 7f0e8280d640 1 -- 172.21.15.27:0/514415540 wait complete. 2023-08-11T15:51:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.585+0000 7f0e8280d640 1 Processor -- start 2023-08-11T15:51:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.585+0000 7f0e8280d640 1 -- start start 2023-08-11T15:51:08.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e8280d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e64003b00 0x7f0e7c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e8280d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e7c0cbce0 0x7f0e7c156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e8280d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e7c0cc160 0x7f0e7c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e8280d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c154ea0 con 0x7f0e7c0cc160 2023-08-11T15:51:08.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e8280d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c155f70 con 0x7f0e7c0cbce0 2023-08-11T15:51:08.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e8280d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c155360 con 0x7f0e64003b00 2023-08-11T15:51:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e80daa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e7c0cc160 0x7f0e7c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e80daa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e7c0cc160 0x7f0e7c1573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54378/0 (socket says 172.21.15.27:54378) 2023-08-11T15:51:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e80daa640 1 -- 172.21.15.27:0/1094684456 learned_addr learned my addr 172.21.15.27:0/1094684456 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e7bfff640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e64003b00 0x7f0e7c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.586+0000 7f0e7b7fe640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e7c0cbce0 0x7f0e7c156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e80daa640 1 -- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e64003b00 msgr2=0x7f0e7c154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e80daa640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e64003b00 0x7f0e7c154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e80daa640 1 -- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e7c0cbce0 msgr2=0x7f0e7c156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e80daa640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e7c0cbce0 0x7f0e7c156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e80daa640 1 -- 172.21.15.27:0/1094684456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c155640 con 0x7f0e7c0cc160 2023-08-11T15:51:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e80daa640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e7c0cc160 0x7f0e7c1573f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e64004740 tx=0x7f0e7400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e78ff9640 1 -- 172.21.15.27:0/1094684456 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400c6b0 con 0x7f0e7c0cc160 2023-08-11T15:51:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e78ff9640 1 -- 172.21.15.27:0/1094684456 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7400c810 con 0x7f0e7c0cc160 2023-08-11T15:51:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e78ff9640 1 -- 172.21.15.27:0/1094684456 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400cbc0 con 0x7f0e7c0cc160 2023-08-11T15:51:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e8280d640 1 -- 172.21.15.27:0/1094684456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c155920 con 0x7f0e7c0cc160 2023-08-11T15:51:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.587+0000 7f0e8280d640 1 -- 172.21.15.27:0/1094684456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c157ba0 con 0x7f0e7c0cc160 2023-08-11T15:51:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.589+0000 7f0e8280d640 1 -- 172.21.15.27:0/1094684456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c0cfb80 con 0x7f0e7c0cc160 2023-08-11T15:51:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.589+0000 7f0e78ff9640 1 -- 172.21.15.27:0/1094684456 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0e7400cd20 con 0x7f0e7c0cc160 2023-08-11T15:51:08.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.591+0000 7f0e78ff9640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a080 0x7f0e6806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:08.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.591+0000 7f0e78ff9640 1 -- 172.21.15.27:0/1094684456 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0e740883e0 con 0x7f0e7c0cc160 2023-08-11T15:51:08.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.592+0000 7f0e7bfff640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a080 0x7f0e6806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:08.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.593+0000 7f0e7bfff640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a080 0x7f0e6806c540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c034000 tx=0x7f0e6c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:08.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.595+0000 7f0e78ff9640 1 -- 172.21.15.27:0/1094684456 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0e74057260 con 0x7f0e7c0cc160 2023-08-11T15:51:08.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.831+0000 7f0e8280d640 1 -- 172.21.15.27:0/1094684456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e7c073990 con 0x7f0e7c0cc160 2023-08-11T15:51:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.832+0000 7f0e78ff9640 1 -- 172.21.15.27:0/1094684456 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f0e74056df0 con 0x7f0e7c0cc160 2023-08-11T15:51:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:08.834 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:51:01.220970+0000","modified":"2023-08-11T15:51:06.831869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4315":{"gid":4315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-11T15:51:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-08-11T15:51:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.834+0000 7f0e8280d640 1 -- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a080 msgr2=0x7f0e6806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a080 0x7f0e6806c540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c034000 tx=0x7f0e6c009030 comp rx=0 tx=0).stop 2023-08-11T15:51:08.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 -- 172.21.15.27:0/1094684456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e7c0cc160 msgr2=0x7f0e7c1573f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:08.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e7c0cc160 0x7f0e7c1573f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e64004740 tx=0x7f0e7400ec00 comp rx=0 tx=0).stop 2023-08-11T15:51:08.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 -- 172.21.15.27:0/1094684456 shutdown_connections 2023-08-11T15:51:08.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a080 0x7f0e6806c540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e7c0cc160 0x7f0e7c1573f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e7c0cbce0 0x7f0e7c156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 --2- 172.21.15.27:0/1094684456 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e64003b00 0x7f0e7c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 -- 172.21.15.27:0/1094684456 >> 172.21.15.27:0/1094684456 conn(0x7f0e7c06a4a0 msgr2=0x7f0e7c0b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 -- 172.21.15.27:0/1094684456 shutdown_connections 2023-08-11T15:51:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:08.835+0000 7f0e8280d640 1 -- 172.21.15.27:0/1094684456 wait complete. 2023-08-11T15:51:08.858 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:09.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.087+0000 7fe85f463640 1 Processor -- start 2023-08-11T15:51:09.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.087+0000 7fe85f463640 1 -- start start 2023-08-11T15:51:09.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.088+0000 7fe85f463640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8580d1b90 0x7fe8580b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.088+0000 7fe85f463640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85815c970 con 0x7fe85815c590 2023-08-11T15:51:09.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.088+0000 7fe85f463640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8580d0250 con 0x7fe8580cd0e0 2023-08-11T15:51:09.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.088+0000 7fe85f463640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8580d0ed0 con 0x7fe8580d1b90 2023-08-11T15:51:09.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.088+0000 7fe85dc60640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8580d1b90 0x7fe8580b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.088+0000 7fe85dc60640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8580d1b90 0x7fe8580b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54386/0 (socket says 172.21.15.27:54386) 2023-08-11T15:51:09.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.088+0000 7fe85e461640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe85815c590 0x7fe8580d1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35400/0 (socket says 172.21.15.27:35400) 2023-08-11T15:51:09.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.088+0000 7fe85dc60640 1 -- 172.21.15.27:0/1091397689 learned_addr learned my addr 172.21.15.27:0/1091397689 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:09.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.089+0000 7fe85dc60640 1 -- 172.21.15.27:0/1091397689 >> v1:172.21.15.119:6789/0 conn(0x7fe8580cd0e0 legacy=0x7fe858160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:09.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.089+0000 7fe85dc60640 1 -- 172.21.15.27:0/1091397689 >> v1:172.21.15.27:6789/0 conn(0x7fe85815c590 legacy=0x7fe8580d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.089+0000 7fe85dc60640 1 -- 172.21.15.27:0/1091397689 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580d0530 con 0x7fe8580d1b90 2023-08-11T15:51:09.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.089+0000 7fe85d45f640 1 -- 172.21.15.27:0/1091397689 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925362137 0 0) 0x7fe85815c970 con 0x7fe85815c590 2023-08-11T15:51:09.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.090+0000 7fe85dc60640 1 --2- 172.21.15.27:0/1091397689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8580d1b90 0x7fe8580b5440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe848009870 tx=0x7fe84802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56dc9c56ffc715d8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.090+0000 7fe85d45f640 1 -- 172.21.15.27:0/1091397689 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8480317e0 con 0x7fe8580d1b90 2023-08-11T15:51:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.090+0000 7fe85d45f640 1 -- 172.21.15.27:0/1091397689 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe848031940 con 0x7fe8580d1b90 2023-08-11T15:51:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.090+0000 7fe85d45f640 1 -- 172.21.15.27:0/1091397689 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848031cd0 con 0x7fe8580d1b90 2023-08-11T15:51:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.091+0000 7fe85f463640 1 -- 172.21.15.27:0/1091397689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8580d1b90 msgr2=0x7fe8580b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.091+0000 7fe85f463640 1 --2- 172.21.15.27:0/1091397689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8580d1b90 0x7fe8580b5440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe848009870 tx=0x7fe84802b880 comp rx=0 tx=0).stop 2023-08-11T15:51:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.091+0000 7fe85f463640 1 -- 172.21.15.27:0/1091397689 shutdown_connections 2023-08-11T15:51:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.091+0000 7fe85f463640 1 --2- 172.21.15.27:0/1091397689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8580d1b90 0x7fe8580b5440 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.091+0000 7fe85f463640 1 -- 172.21.15.27:0/1091397689 >> 172.21.15.27:0/1091397689 conn(0x7fe85806a250 msgr2=0x7fe85806a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.091+0000 7fe85f463640 1 -- 172.21.15.27:0/1091397689 shutdown_connections 2023-08-11T15:51:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.091+0000 7fe85f463640 1 -- 172.21.15.27:0/1091397689 wait complete. 2023-08-11T15:51:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.093+0000 7fe85f463640 1 Processor -- start 2023-08-11T15:51:09.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.093+0000 7fe85f463640 1 -- start start 2023-08-11T15:51:09.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.094+0000 7fe85f463640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8580cd0e0 0x7fe8580b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.094+0000 7fe85f463640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8580d1b90 0x7fe8580c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.094+0000 7fe85f463640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85815c590 0x7fe8580c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.094+0000 7fe85f463640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580b4b40 con 0x7fe85815c590 2023-08-11T15:51:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.094+0000 7fe85f463640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8581f2cf0 con 0x7fe8580cd0e0 2023-08-11T15:51:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.094+0000 7fe85f463640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe858156fd0 con 0x7fe8580d1b90 2023-08-11T15:51:09.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.095+0000 7fe85ec62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85815c590 0x7fe8580c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.095+0000 7fe85ec62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85815c590 0x7fe8580c7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54396/0 (socket says 172.21.15.27:54396) 2023-08-11T15:51:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.095+0000 7fe85ec62640 1 -- 172.21.15.27:0/2435492115 learned_addr learned my addr 172.21.15.27:0/2435492115 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.095+0000 7fe85e461640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8580cd0e0 0x7fe8580b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.095+0000 7fe85dc60640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8580d1b90 0x7fe8580c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.095+0000 7fe85ec62640 1 -- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8580d1b90 msgr2=0x7fe8580c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.095+0000 7fe85ec62640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8580d1b90 0x7fe8580c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.095+0000 7fe85ec62640 1 -- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8580cd0e0 msgr2=0x7fe8580b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.095+0000 7fe85ec62640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8580cd0e0 0x7fe8580b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.095+0000 7fe85ec62640 1 -- 172.21.15.27:0/2435492115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480094d0 con 0x7fe85815c590 2023-08-11T15:51:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.096+0000 7fe85ec62640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85815c590 0x7fe8580c7470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe85000c780 tx=0x7fe85000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.096+0000 7fe84effd640 1 -- 172.21.15.27:0/2435492115 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850003d80 con 0x7fe85815c590 2023-08-11T15:51:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.096+0000 7fe84effd640 1 -- 172.21.15.27:0/2435492115 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85000ce90 con 0x7fe85815c590 2023-08-11T15:51:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.096+0000 7fe85f463640 1 -- 172.21.15.27:0/2435492115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8581572b0 con 0x7fe85815c590 2023-08-11T15:51:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.096+0000 7fe84effd640 1 -- 172.21.15.27:0/2435492115 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8500052b0 con 0x7fe85815c590 2023-08-11T15:51:09.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.096+0000 7fe85f463640 1 -- 172.21.15.27:0/2435492115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8580c6fe0 con 0x7fe85815c590 2023-08-11T15:51:09.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.097+0000 7fe85f463640 1 -- 172.21.15.27:0/2435492115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8580d07f0 con 0x7fe85815c590 2023-08-11T15:51:09.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.098+0000 7fe84effd640 1 -- 172.21.15.27:0/2435492115 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe850005410 con 0x7fe85815c590 2023-08-11T15:51:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.099+0000 7fe84effd640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe83c06a050 0x7fe83c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.099+0000 7fe84effd640 1 -- 172.21.15.27:0/2435492115 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe850088160 con 0x7fe85815c590 2023-08-11T15:51:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.100+0000 7fe85e461640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe83c06a050 0x7fe83c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.101+0000 7fe85e461640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe83c06a050 0x7fe83c06c510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe854000f60 tx=0x7fe854007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.104+0000 7fe84effd640 1 -- 172.21.15.27:0/2435492115 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe8580d07f0 con 0x7fe85815c590 2023-08-11T15:51:09.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.300+0000 7fe85f463640 1 -- 172.21.15.27:0/2435492115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8581577d0 con 0x7fe85815c590 2023-08-11T15:51:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.304+0000 7fe84effd640 1 -- 172.21.15.27:0/2435492115 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7fe8581577d0 con 0x7fe85815c590 2023-08-11T15:51:09.307 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:09.308 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:09.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.306+0000 7fe85f463640 1 -- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe83c06a050 msgr2=0x7fe83c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.306+0000 7fe85f463640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe83c06a050 0x7fe83c06c510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe854000f60 tx=0x7fe854007c00 comp rx=0 tx=0).stop 2023-08-11T15:51:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.306+0000 7fe85f463640 1 -- 172.21.15.27:0/2435492115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85815c590 msgr2=0x7fe8580c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.306+0000 7fe85f463640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85815c590 0x7fe8580c7470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe85000c780 tx=0x7fe85000cc40 comp rx=0 tx=0).stop 2023-08-11T15:51:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.306+0000 7fe85f463640 1 -- 172.21.15.27:0/2435492115 shutdown_connections 2023-08-11T15:51:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.306+0000 7fe85f463640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe83c06a050 0x7fe83c06c510 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.306+0000 7fe85f463640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8580d1b90 0x7fe8580c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.306+0000 7fe85f463640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85815c590 0x7fe8580c7470 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.306+0000 7fe85f463640 1 --2- 172.21.15.27:0/2435492115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8580cd0e0 0x7fe8580b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.306+0000 7fe85f463640 1 -- 172.21.15.27:0/2435492115 >> 172.21.15.27:0/2435492115 conn(0x7fe85806a250 msgr2=0x7fe85806a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:09.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.307+0000 7fe85f463640 1 -- 172.21.15.27:0/2435492115 shutdown_connections 2023-08-11T15:51:09.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.307+0000 7fe85f463640 1 -- 172.21.15.27:0/2435492115 wait complete. 2023-08-11T15:51:09.731 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-11T15:51:01.220970+0000', 'modified': '2023-08-11T15:51:06.831869+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4315}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4315': {'gid': 4315, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6835/2813789236', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 2813789236}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 2813789236}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-08-11T15:51:09.732 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-11T15:51:09.732 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-11T15:51:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.976+0000 7fb48a799640 1 Processor -- start 2023-08-11T15:51:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.977+0000 7fb48a799640 1 -- start start 2023-08-11T15:51:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.978+0000 7fb48a799640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb4840cd7d0 0x7fb48415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.978+0000 7fb48a799640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48415d2d0 con 0x7fb484159e90 2023-08-11T15:51:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.978+0000 7fb48a799640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4840ca480 con 0x7fb484159280 2023-08-11T15:51:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.978+0000 7fb48a799640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb484159990 con 0x7fb4840cd7d0 2023-08-11T15:51:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.979+0000 7fb488d36640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb484159e90 0x7fb48406b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55426/0 (socket says 172.21.15.27:55426) 2023-08-11T15:51:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.979+0000 7fb488d36640 1 -- 172.21.15.27:0/586112233 learned_addr learned my addr 172.21.15.27:0/586112233 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.979+0000 7fb4837fe640 1 --2- 172.21.15.27:0/586112233 >> v2:172.21.15.27:3300/0 conn(0x7fb4840cd7d0 0x7fb48415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.979+0000 7fb4837fe640 1 -- 172.21.15.27:0/586112233 >> v1:172.21.15.119:6789/0 conn(0x7fb484159280 legacy=0x7fb4840c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.979+0000 7fb4837fe640 1 -- 172.21.15.27:0/586112233 >> v1:172.21.15.27:6789/0 conn(0x7fb484159e90 legacy=0x7fb48406b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.979+0000 7fb482ffd640 1 -- 172.21.15.27:0/586112233 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996704405 0 0) 0x7fb48415d2d0 con 0x7fb484159e90 2023-08-11T15:51:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.979+0000 7fb4837fe640 1 -- 172.21.15.27:0/586112233 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48415d500 con 0x7fb4840cd7d0 2023-08-11T15:51:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.981+0000 7fb4837fe640 1 --2- 172.21.15.27:0/586112233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 0x7fb48415cd90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb478009d90 tx=0x7fb47802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a7eb2743e0906e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.981+0000 7fb482ffd640 1 -- 172.21.15.27:0/586112233 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47802cc60 con 0x7fb4840cd7d0 2023-08-11T15:51:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.981+0000 7fb482ffd640 1 -- 172.21.15.27:0/586112233 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47802cdc0 con 0x7fb4840cd7d0 2023-08-11T15:51:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.981+0000 7fb482ffd640 1 -- 172.21.15.27:0/586112233 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478034660 con 0x7fb4840cd7d0 2023-08-11T15:51:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.981+0000 7fb48a799640 1 -- 172.21.15.27:0/586112233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 msgr2=0x7fb48415cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.981+0000 7fb48a799640 1 --2- 172.21.15.27:0/586112233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 0x7fb48415cd90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb478009d90 tx=0x7fb47802beb0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.982+0000 7fb48a799640 1 -- 172.21.15.27:0/586112233 shutdown_connections 2023-08-11T15:51:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.982+0000 7fb48a799640 1 --2- 172.21.15.27:0/586112233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 0x7fb48415cd90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.982+0000 7fb48a799640 1 -- 172.21.15.27:0/586112233 >> 172.21.15.27:0/586112233 conn(0x7fb484069e90 msgr2=0x7fb4840afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.982+0000 7fb48a799640 1 -- 172.21.15.27:0/586112233 shutdown_connections 2023-08-11T15:51:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.982+0000 7fb48a799640 1 -- 172.21.15.27:0/586112233 wait complete. 2023-08-11T15:51:09.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.983+0000 7fb48a799640 1 Processor -- start 2023-08-11T15:51:09.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.984+0000 7fb48a799640 1 -- start start 2023-08-11T15:51:09.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb48a799640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 0x7fb4840705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb48a799640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb484159280 0x7fb48414f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb48a799640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb484159e90 0x7fb4841f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb48a799640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4841f2880 con 0x7fb4840cd7d0 2023-08-11T15:51:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb48a799640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4841f3610 con 0x7fb484159280 2023-08-11T15:51:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb48a799640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4841f2a90 con 0x7fb484159e90 2023-08-11T15:51:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb483fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 0x7fb4840705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb483fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 0x7fb4840705b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43332/0 (socket says 172.21.15.27:43332) 2023-08-11T15:51:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb483fff640 1 -- 172.21.15.27:0/219821076 learned_addr learned my addr 172.21.15.27:0/219821076 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:09.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb4837fe640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb484159280 0x7fb48414f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb488d36640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb484159e90 0x7fb4841f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb483fff640 1 -- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb484159e90 msgr2=0x7fb4841f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb483fff640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb484159e90 0x7fb4841f2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb483fff640 1 -- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb484159280 msgr2=0x7fb48414f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb483fff640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb484159280 0x7fb48414f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb483fff640 1 -- 172.21.15.27:0/219821076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb478009a70 con 0x7fb4840cd7d0 2023-08-11T15:51:09.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.985+0000 7fb4837fe640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb484159280 0x7fb48414f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:09.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.986+0000 7fb483fff640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 0x7fb4840705b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c450 tx=0x7fb47400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.986+0000 7fb480ff9640 1 -- 172.21.15.27:0/219821076 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740052d0 con 0x7fb4840cd7d0 2023-08-11T15:51:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.986+0000 7fb480ff9640 1 -- 172.21.15.27:0/219821076 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb474005430 con 0x7fb4840cd7d0 2023-08-11T15:51:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.986+0000 7fb480ff9640 1 -- 172.21.15.27:0/219821076 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474012870 con 0x7fb4840cd7d0 2023-08-11T15:51:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.986+0000 7fb48a799640 1 -- 172.21.15.27:0/219821076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4841f2d70 con 0x7fb4840cd7d0 2023-08-11T15:51:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.986+0000 7fb48a799640 1 -- 172.21.15.27:0/219821076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4841f3330 con 0x7fb4840cd7d0 2023-08-11T15:51:09.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.988+0000 7fb480ff9640 1 -- 172.21.15.27:0/219821076 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb4740129d0 con 0x7fb4840cd7d0 2023-08-11T15:51:09.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.989+0000 7fb480ff9640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb47006a080 0x7fb47006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.989+0000 7fb480ff9640 1 -- 172.21.15.27:0/219821076 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb474088300 con 0x7fb4840cd7d0 2023-08-11T15:51:09.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.990+0000 7fb4837fe640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb47006a080 0x7fb47006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.990+0000 7fb48d3ec740 1 Processor -- start 2023-08-11T15:51:09.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.991+0000 7fb48d3ec740 1 -- start start 2023-08-11T15:51:09.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.991+0000 7fb4837fe640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb47006a080 0x7fb47006c540 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb478036000 tx=0x7fb478030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:09.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x560689076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x560689046840 0x560689073b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068904a440 0x560689074070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:09.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x560688fb8ef0 con 0x5606890768c0 2023-08-11T15:51:09.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560688fb65e0 con 0x56068904a440 2023-08-11T15:51:09.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x560688fb1ed0 con 0x560689046840 2023-08-11T15:51:09.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb483fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x560689076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:09.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb483fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x560689076ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43338/0 (socket says 172.21.15.27:43338) 2023-08-11T15:51:09.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb483fff640 1 -- 172.21.15.27:0/2014830784 learned_addr learned my addr 172.21.15.27:0/2014830784 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:09.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb483fff640 1 -- 172.21.15.27:0/2014830784 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x560689046840 msgr2=0x560689073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:09.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb483fff640 1 --2- 172.21.15.27:0/2014830784 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x560689046840 0x560689073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb483fff640 1 -- 172.21.15.27:0/2014830784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068904a440 msgr2=0x560689074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:09.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb483fff640 1 --2- 172.21.15.27:0/2014830784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068904a440 0x560689074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.992+0000 7fb483fff640 1 -- 172.21.15.27:0/2014830784 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560688fb6b40 con 0x5606890768c0 2023-08-11T15:51:09.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.993+0000 7fb483fff640 1 --2- 172.21.15.27:0/2014830784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x560689076ca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb474086410 tx=0x7fb474018370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb3e0e5b61516a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:09.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.993+0000 7fb469ffb640 1 -- 172.21.15.27:0/2014830784 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47405d0c0 con 0x5606890768c0 2023-08-11T15:51:09.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.993+0000 7fb469ffb640 1 -- 172.21.15.27:0/2014830784 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47405f160 con 0x5606890768c0 2023-08-11T15:51:09.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.993+0000 7fb469ffb640 1 -- 172.21.15.27:0/2014830784 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740557f0 con 0x5606890768c0 2023-08-11T15:51:09.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.993+0000 7fb48d3ec740 1 -- 172.21.15.27:0/2014830784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 msgr2=0x560689076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:09.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.993+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/2014830784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x560689076ca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb474086410 tx=0x7fb474018370 comp rx=0 tx=0).stop 2023-08-11T15:51:09.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.994+0000 7fb48d3ec740 1 -- 172.21.15.27:0/2014830784 shutdown_connections 2023-08-11T15:51:09.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.994+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/2014830784 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x560689046840 0x560689073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.994+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/2014830784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068904a440 0x560689074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.994+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/2014830784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x560689076ca0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:09.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.994+0000 7fb48d3ec740 1 -- 172.21.15.27:0/2014830784 >> 172.21.15.27:0/2014830784 conn(0x560689048b50 msgr2=0x5606890916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:09.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.994+0000 7fb48d3ec740 1 -- 172.21.15.27:0/2014830784 shutdown_connections 2023-08-11T15:51:09.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.994+0000 7fb48d3ec740 1 -- 172.21.15.27:0/2014830784 wait complete. 2023-08-11T15:51:09.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.994+0000 7fb48d3ec740 1 -- start start 2023-08-11T15:51:10.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.995+0000 7fb48d3ec740 1 Processor -- start 2023-08-11T15:51:10.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x56068904a440 0x56068906f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x56068907bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068907b000 0x56068907b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x560688f51dc0 con 0x5606890768c0 2023-08-11T15:51:10.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560688f51a60 con 0x56068907b000 2023-08-11T15:51:10.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x560688d17f20 con 0x56068904a440 2023-08-11T15:51:10.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb48d3ec740 10 client.0 _ll_register_callbacks cb 0x560688fa9d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:10.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x56068907bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x56068907bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43348/0 (socket says 172.21.15.27:43348) 2023-08-11T15:51:10.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb4837fe640 1 -- 172.21.15.27:0/384742027 learned_addr learned my addr 172.21.15.27:0/384742027 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:10.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb488d36640 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068907b000 0x56068907b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb483fff640 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x56068904a440 0x56068906f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb4837fe640 1 -- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x56068904a440 msgr2=0x56068906f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:10.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb4837fe640 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x56068904a440 0x56068906f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb4837fe640 1 -- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068907b000 msgr2=0x56068907b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:10.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb4837fe640 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068907b000 0x56068907b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.996+0000 7fb4837fe640 1 -- 172.21.15.27:0/384742027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4841f3330 con 0x5606890768c0 2023-08-11T15:51:10.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.997+0000 7fb4837fe640 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x56068907bd20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb47802c4c0 tx=0x7fb478004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199b7c8e77af489c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:10.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.997+0000 7fb462ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-08-11T15:51:10.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.997+0000 7fb462ffd640 1 -- 172.21.15.27:0/384742027 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478035650 con 0x5606890768c0 2023-08-11T15:51:10.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.997+0000 7fb48d3ec740 1 -- 172.21.15.27:0/384742027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560688f89cf0 con 0x5606890768c0 2023-08-11T15:51:10.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.997+0000 7fb462ffd640 1 -- 172.21.15.27:0/384742027 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb4780357b0 con 0x5606890768c0 2023-08-11T15:51:10.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.997+0000 7fb462ffd640 1 -- 172.21.15.27:0/384742027 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb478035970 con 0x5606890768c0 2023-08-11T15:51:10.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.997+0000 7fb48d3ec740 10 client.4398 fetch_fsmap learned FSMap version 7 2023-08-11T15:51:10.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.997+0000 7fb48d3ec740 1 -- 172.21.15.27:0/384742027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x560688fb0550 con 0x5606890768c0 2023-08-11T15:51:10.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.998+0000 7fb462ffd640 1 -- 172.21.15.27:0/384742027 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560688f89cf0 con 0x5606890768c0 2023-08-11T15:51:10.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.998+0000 7fb462ffd640 1 -- 172.21.15.27:0/384742027 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb478035ad0 con 0x5606890768c0 2023-08-11T15:51:10.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.998+0000 7fb48d3ec740 10 client.4398 fetch_fsmap finished waiting for FSMap version 7 2023-08-11T15:51:10.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.998+0000 7fb48d3ec740 10 client.4398 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T15:51:10.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.998+0000 7fb48d3ec740 20 client.4398 populate_metadata read hostname 'smithi027' 2023-08-11T15:51:10.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.998+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x5606890776d0 0x560689077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.998+0000 7fb48d3ec740 4 client.4398 mds_command: new command op to 4315 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:51:10.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.998+0000 7fb48d3ec740 1 -- 172.21.15.27:0/384742027 --> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560688f86c60 con 0x5606890776d0 2023-08-11T15:51:10.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.999+0000 7fb483fff640 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x5606890776d0 0x560689077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.999+0000 7fb483fff640 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x5606890776d0 0x560689077ab0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e680a3be77a09 server_cookie=df5937bdb753cc83 in_seq=0 out_seq=0 2023-08-11T15:51:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:09.999+0000 7fb462ffd640 10 client.4398 ms_handle_connect on v2:172.21.15.119:6834/2813789236 2023-08-11T15:51:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.001+0000 7fb462ffd640 1 -- 172.21.15.27:0/384742027 <== mds.0 v2:172.21.15.119:6834/2813789236 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x560688f86c60 con 0x5606890776d0 2023-08-11T15:51:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.001+0000 7fb462ffd640 10 client.4398 handle_command_reply: tid=0 2023-08-11T15:51:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.001+0000 7fb48d3ec740 1 client.4398 shutdown 2023-08-11T15:51:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.001+0000 7fb48d3ec740 1 -- 172.21.15.27:0/384742027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 msgr2=0x56068907bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.001+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x56068907bd20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb47802c4c0 tx=0x7fb478004a40 comp rx=0 tx=0).stop 2023-08-11T15:51:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.001+0000 7fb48d3ec740 1 -- 172.21.15.27:0/384742027 shutdown_connections 2023-08-11T15:51:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.001+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x5606890776d0 0x560689077ab0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.001+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068907b000 0x56068907b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.002+0000 7fb462ffd640 0 client.4398 ms_handle_reset on v2:172.21.15.119:6834/2813789236 2023-08-11T15:51:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.001+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606890768c0 0x56068907bd20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.002+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/384742027 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x56068904a440 0x56068906f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.002+0000 7fb48d3ec740 1 -- 172.21.15.27:0/384742027 >> 172.21.15.27:0/384742027 conn(0x560689046840 msgr2=0x560689094be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.002+0000 7fb48d3ec740 1 -- 172.21.15.27:0/384742027 shutdown_connections 2023-08-11T15:51:10.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.002+0000 7fb48d3ec740 1 -- 172.21.15.27:0/384742027 wait complete. 2023-08-11T15:51:10.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.002+0000 7fb48d3ec740 20 client.4398 trim_cache size 0 max 16384 2023-08-11T15:51:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.019+0000 7fb48d3ec740 1 Processor -- start 2023-08-11T15:51:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.019+0000 7fb48d3ec740 1 -- start start 2023-08-11T15:51:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.020+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689047d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.020+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606890768c0 0x56068907a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.020+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x56068907b000 0x560689091410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.020+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068904d630 con 0x56068904a440 2023-08-11T15:51:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.020+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068907b460 con 0x5606890768c0 2023-08-11T15:51:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.020+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606890b48a0 con 0x56068907b000 2023-08-11T15:51:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.020+0000 7fb483fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689047d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.020+0000 7fb483fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689047d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43360/0 (socket says 172.21.15.27:43360) 2023-08-11T15:51:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.020+0000 7fb483fff640 1 -- 172.21.15.27:0/3531633063 learned_addr learned my addr 172.21.15.27:0/3531633063 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.021+0000 7fb483fff640 1 -- 172.21.15.27:0/3531633063 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x56068907b000 msgr2=0x560689091410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.021+0000 7fb483fff640 1 --2- 172.21.15.27:0/3531633063 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x56068907b000 0x560689091410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.021+0000 7fb483fff640 1 -- 172.21.15.27:0/3531633063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606890768c0 msgr2=0x56068907a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.021+0000 7fb483fff640 1 --2- 172.21.15.27:0/3531633063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606890768c0 0x56068907a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.021+0000 7fb483fff640 1 -- 172.21.15.27:0/3531633063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560688fe39b0 con 0x56068904a440 2023-08-11T15:51:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.021+0000 7fb483fff640 1 --2- 172.21.15.27:0/3531633063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689047d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb474053950 tx=0x7fb47401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decdd9a33de308cb server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.021+0000 7fb46a7fc640 1 -- 172.21.15.27:0/3531633063 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47405f1e0 con 0x56068904a440 2023-08-11T15:51:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.021+0000 7fb46a7fc640 1 -- 172.21.15.27:0/3531633063 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4740880b0 con 0x56068904a440 2023-08-11T15:51:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.021+0000 7fb46a7fc640 1 -- 172.21.15.27:0/3531633063 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474053b60 con 0x56068904a440 2023-08-11T15:51:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.022+0000 7fb48d3ec740 1 -- 172.21.15.27:0/3531633063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 msgr2=0x560689047d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.022+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/3531633063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689047d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb474053950 tx=0x7fb47401d000 comp rx=0 tx=0).stop 2023-08-11T15:51:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.022+0000 7fb48d3ec740 1 -- 172.21.15.27:0/3531633063 shutdown_connections 2023-08-11T15:51:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.022+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/3531633063 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x56068907b000 0x560689091410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.022+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/3531633063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606890768c0 0x56068907a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.022+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/3531633063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689047d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.022+0000 7fb48d3ec740 1 -- 172.21.15.27:0/3531633063 >> 172.21.15.27:0/3531633063 conn(0x560689046840 msgr2=0x56068907c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:10.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.022+0000 7fb48d3ec740 1 -- 172.21.15.27:0/3531633063 shutdown_connections 2023-08-11T15:51:10.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.022+0000 7fb48d3ec740 1 -- 172.21.15.27:0/3531633063 wait complete. 2023-08-11T15:51:10.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.023+0000 7fb48d3ec740 1 -- start start 2023-08-11T15:51:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.023+0000 7fb48d3ec740 1 Processor -- start 2023-08-11T15:51:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689102120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x5606890768c0 0x560689047d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb48d3ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068907b000 0x5606890911e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068908e190 con 0x56068904a440 2023-08-11T15:51:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068908e6c0 con 0x56068907b000 2023-08-11T15:51:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb48d3ec740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x56068908e3e0 con 0x5606890768c0 2023-08-11T15:51:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb48d3ec740 10 client.0 _ll_register_callbacks cb 0x7fb474053790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb483fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb483fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689102120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43364/0 (socket says 172.21.15.27:43364) 2023-08-11T15:51:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb483fff640 1 -- 172.21.15.27:0/1567475792 learned_addr learned my addr 172.21.15.27:0/1567475792 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb483fff640 1 -- 172.21.15.27:0/1567475792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x5606890768c0 msgr2=0x560689047d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb483fff640 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x5606890768c0 0x560689047d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb483fff640 1 -- 172.21.15.27:0/1567475792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068907b000 msgr2=0x5606890911e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb483fff640 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068907b000 0x5606890911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.024+0000 7fb483fff640 1 -- 172.21.15.27:0/1567475792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560688fb6b40 con 0x56068904a440 2023-08-11T15:51:10.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.025+0000 7fb483fff640 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689102120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb474053950 tx=0x7fb474001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e05d9204c08d78 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:10.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.025+0000 7fb462ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-08-11T15:51:10.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.025+0000 7fb462ffd640 1 -- 172.21.15.27:0/1567475792 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47405f1e0 con 0x56068904a440 2023-08-11T15:51:10.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.025+0000 7fb462ffd640 1 -- 172.21.15.27:0/1567475792 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb474053b60 con 0x56068904a440 2023-08-11T15:51:10.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.025+0000 7fb48d3ec740 1 -- 172.21.15.27:0/1567475792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560688fd9eb0 con 0x56068904a440 2023-08-11T15:51:10.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.025+0000 7fb462ffd640 1 -- 172.21.15.27:0/1567475792 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb474053d40 con 0x56068904a440 2023-08-11T15:51:10.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.025+0000 7fb48d3ec740 10 client.4404 fetch_fsmap learned FSMap version 7 2023-08-11T15:51:10.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.025+0000 7fb48d3ec740 1 -- 172.21.15.27:0/1567475792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x560688f50a90 con 0x56068904a440 2023-08-11T15:51:10.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.026+0000 7fb462ffd640 1 -- 172.21.15.27:0/1567475792 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560688fd9eb0 con 0x56068904a440 2023-08-11T15:51:10.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.026+0000 7fb462ffd640 1 -- 172.21.15.27:0/1567475792 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb474019020 con 0x56068904a440 2023-08-11T15:51:10.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.026+0000 7fb48d3ec740 10 client.4404 fetch_fsmap finished waiting for FSMap version 7 2023-08-11T15:51:10.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.026+0000 7fb48d3ec740 10 client.4404 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T15:51:10.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.026+0000 7fb48d3ec740 20 client.4404 populate_metadata read hostname 'smithi027' 2023-08-11T15:51:10.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.026+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x560689071c70 0x560689072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.026+0000 7fb48d3ec740 4 client.4404 mds_command: new command op to 4315 tid=0[{"prefix": "status"}] 2023-08-11T15:51:10.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.026+0000 7fb48d3ec740 1 -- 172.21.15.27:0/1567475792 --> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560688faaeb0 con 0x560689071c70 2023-08-11T15:51:10.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.026+0000 7fb4837fe640 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x560689071c70 0x560689072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.027+0000 7fb4837fe640 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x560689071c70 0x560689072050 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0f057b58298a6c server_cookie=a93fb14184207c3b in_seq=0 out_seq=0 2023-08-11T15:51:10.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.027+0000 7fb462ffd640 10 client.4404 ms_handle_connect on v2:172.21.15.119:6834/2813789236 2023-08-11T15:51:10.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb462ffd640 1 -- 172.21.15.27:0/1567475792 <== mds.0 v2:172.21.15.119:6834/2813789236 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x560688faaeb0 con 0x560689071c70 2023-08-11T15:51:10.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb462ffd640 10 client.4404 handle_command_reply: tid=0 2023-08-11T15:51:10.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 client.4404 shutdown 2023-08-11T15:51:10.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 -- 172.21.15.27:0/1567475792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 msgr2=0x560689102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689102120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb474053950 tx=0x7fb474001120 comp rx=0 tx=0).stop 2023-08-11T15:51:10.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 -- 172.21.15.27:0/1567475792 shutdown_connections 2023-08-11T15:51:10.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x560689071c70 0x560689072050 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x5606890768c0 0x560689047d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068907b000 0x5606890911e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 --2- 172.21.15.27:0/1567475792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56068904a440 0x560689102120 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb462ffd640 0 client.4404 ms_handle_reset on v2:172.21.15.119:6834/2813789236 2023-08-11T15:51:10.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 -- 172.21.15.27:0/1567475792 >> 172.21.15.27:0/1567475792 conn(0x560689046840 msgr2=0x56068906f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:10.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 -- 172.21.15.27:0/1567475792 shutdown_connections 2023-08-11T15:51:10.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.028+0000 7fb48d3ec740 1 -- 172.21.15.27:0/1567475792 wait complete. 2023-08-11T15:51:10.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48d3ec740 20 client.4404 trim_cache size 0 max 16384 2023-08-11T15:51:10.036 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:51:10.036 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "bee60691-4c46-4397-a560-d178aa4db950", 2023-08-11T15:51:10.037 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-08-11T15:51:10.037 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4315, 2023-08-11T15:51:10.037 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-08-11T15:51:10.037 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-08-11T15:51:10.037 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-08-11T15:51:10.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 8.2755556800000001, 2023-08-11T15:51:10.037 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 7, 2023-08-11T15:51:10.038 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 28, 2023-08-11T15:51:10.038 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 28, 2023-08-11T15:51:10.038 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 15.153670288000001 2023-08-11T15:51:10.038 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:51:10.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48a799640 1 -- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb47006a080 msgr2=0x7fb47006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48a799640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb47006a080 0x7fb47006c540 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb478036000 tx=0x7fb478030000 comp rx=0 tx=0).stop 2023-08-11T15:51:10.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48a799640 1 -- 172.21.15.27:0/219821076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 msgr2=0x7fb4840705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48a799640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 0x7fb4840705b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c450 tx=0x7fb47400c910 comp rx=0 tx=0).stop 2023-08-11T15:51:10.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48a799640 1 -- 172.21.15.27:0/219821076 shutdown_connections 2023-08-11T15:51:10.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48a799640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb47006a080 0x7fb47006c540 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48a799640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb484159e90 0x7fb4841f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48a799640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb484159280 0x7fb48414f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48a799640 1 --2- 172.21.15.27:0/219821076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4840cd7d0 0x7fb4840705b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.029+0000 7fb48a799640 1 -- 172.21.15.27:0/219821076 >> 172.21.15.27:0/219821076 conn(0x7fb484069e90 msgr2=0x7fb4840aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:10.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.030+0000 7fb48a799640 1 -- 172.21.15.27:0/219821076 shutdown_connections 2023-08-11T15:51:10.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.030+0000 7fb48a799640 1 -- 172.21.15.27:0/219821076 wait complete. 2023-08-11T15:51:10.050 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-11T15:51:10.050 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:51:10.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-11T15:51:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.281+0000 7f67e5524640 1 Processor -- start 2023-08-11T15:51:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.282+0000 7f67e5524640 1 -- start start 2023-08-11T15:51:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.283+0000 7f67e5524640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00cd7f0 0x7f67e00b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.283+0000 7f67e5524640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e015c970 con 0x7f67e015c590 2023-08-11T15:51:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.283+0000 7f67e5524640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e00d0250 con 0x7f67e00d1b90 2023-08-11T15:51:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.283+0000 7f67e5524640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00d0ed0 con 0x7f67e00cd7f0 2023-08-11T15:51:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.283+0000 7f67dffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f67e015c590 0x7f67e00d1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55430/0 (socket says 172.21.15.27:55430) 2023-08-11T15:51:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.284+0000 7f67dffff640 1 -- 172.21.15.27:0/1161390031 learned_addr learned my addr 172.21.15.27:0/1161390031 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.284+0000 7f67e4d23640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00cd7f0 0x7f67e00b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.284+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369931371 0 0) 0x7f67e015c970 con 0x7f67e015c590 2023-08-11T15:51:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.284+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67cc003520 con 0x7f67e015c590 2023-08-11T15:51:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.284+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2321382435 0 0) 0x7f67cc003520 con 0x7f67e015c590 2023-08-11T15:51:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.284+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00cd7f0 msgr2=0x7f67e00b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.284+0000 7f67df7fe640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00cd7f0 0x7f67e00b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.284+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 >> v1:172.21.15.119:6789/0 conn(0x7f67e00d1b90 legacy=0x7f67e00cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.284+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00d0530 con 0x7f67e015c590 2023-08-11T15:51:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f67d0001670 con 0x7f67e015c590 2023-08-11T15:51:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 >> v1:172.21.15.27:6789/0 conn(0x7f67e015c590 legacy=0x7f67e00d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67df7fe640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67cc0039a0 0x7f67cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67df7fe640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67cc0047c0 0x7f67cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67df7fe640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00cd7f0 0x7f67cc005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00d0ed0 con 0x7f67cc0047c0 2023-08-11T15:51:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc00bb40 con 0x7f67cc0039a0 2023-08-11T15:51:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc00bd00 con 0x7f67e00cd7f0 2023-08-11T15:51:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67d7fff640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67cc0047c0 0x7f67cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67dffff640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67cc0039a0 0x7f67cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67e4d23640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00cd7f0 0x7f67cc005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.285+0000 7f67d7fff640 1 -- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00cd7f0 msgr2=0x7f67cc005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.286+0000 7f67d7fff640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00cd7f0 0x7f67cc005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.286+0000 7f67d7fff640 1 -- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67cc0039a0 msgr2=0x7f67cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.286+0000 7f67d7fff640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67cc0039a0 0x7f67cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.286+0000 7f67d7fff640 1 -- 172.21.15.27:0/1161390031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc003520 con 0x7f67cc0047c0 2023-08-11T15:51:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.286+0000 7f67dffff640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67cc0039a0 0x7f67cc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.287+0000 7f67d7fff640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67cc0047c0 0x7f67cc004c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f67c8001720 tx=0x7f67cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325569f5c1ea3a0a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.287+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80052d0 con 0x7f67cc0047c0 2023-08-11T15:51:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.287+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c802e050 con 0x7f67cc0047c0 2023-08-11T15:51:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.287+0000 7f67df7fe640 1 -- 172.21.15.27:0/1161390031 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80035f0 con 0x7f67cc0047c0 2023-08-11T15:51:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.288+0000 7f67e5524640 1 -- 172.21.15.27:0/1161390031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67cc0047c0 msgr2=0x7f67cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.288+0000 7f67e5524640 1 --2- 172.21.15.27:0/1161390031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67cc0047c0 0x7f67cc004c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f67c8001720 tx=0x7f67cc0057f0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.288+0000 7f67e4d23640 1 -- 172.21.15.27:0/1161390031 reap_dead start 2023-08-11T15:51:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.288+0000 7f67e5524640 1 -- 172.21.15.27:0/1161390031 shutdown_connections 2023-08-11T15:51:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.288+0000 7f67e5524640 1 -- 172.21.15.27:0/1161390031 >> 172.21.15.27:0/1161390031 conn(0x7f67e006a250 msgr2=0x7f67e006a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.288+0000 7f67e5524640 1 -- 172.21.15.27:0/1161390031 shutdown_connections 2023-08-11T15:51:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.288+0000 7f67e5524640 1 -- 172.21.15.27:0/1161390031 wait complete. 2023-08-11T15:51:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.290+0000 7f67e5524640 1 Processor -- start 2023-08-11T15:51:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.290+0000 7f67e5524640 1 -- start start 2023-08-11T15:51:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.291+0000 7f67e5524640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67cc0047c0 0x7f67e01454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.291+0000 7f67e5524640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e00c86a0 0x7f67e00ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.291+0000 7f67e5524640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00c8b20 0x7f67e00ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.291+0000 7f67e5524640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0148d80 con 0x7f67e00c86a0 2023-08-11T15:51:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.291+0000 7f67e5524640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00c7920 con 0x7f67cc0047c0 2023-08-11T15:51:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.291+0000 7f67e5524640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0145a00 con 0x7f67e00c8b20 2023-08-11T15:51:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.291+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e00c86a0 0x7f67e00ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.291+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e00c86a0 0x7f67e00ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43384/0 (socket says 172.21.15.27:43384) 2023-08-11T15:51:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.291+0000 7f67d7fff640 1 -- 172.21.15.27:0/2150613502 learned_addr learned my addr 172.21.15.27:0/2150613502 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.292+0000 7f67dffff640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67cc0047c0 0x7f67e01454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.292+0000 7f67e4d23640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00c8b20 0x7f67e00ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.292+0000 7f67d7fff640 1 -- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00c8b20 msgr2=0x7f67e00ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.292+0000 7f67d7fff640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00c8b20 0x7f67e00ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.292+0000 7f67d7fff640 1 -- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67cc0047c0 msgr2=0x7f67e01454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.292+0000 7f67d7fff640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67cc0047c0 0x7f67e01454c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.292+0000 7f67d7fff640 1 -- 172.21.15.27:0/2150613502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e0148aa0 con 0x7f67e00c86a0 2023-08-11T15:51:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.292+0000 7f67d7fff640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e00c86a0 0x7f67e00ca310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f67c8008fd0 tx=0x7f67c8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.292+0000 7f67dd7fa640 1 -- 172.21.15.27:0/2150613502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80076c0 con 0x7f67e00c86a0 2023-08-11T15:51:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.293+0000 7f67dd7fa640 1 -- 172.21.15.27:0/2150613502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c8007a30 con 0x7f67e00c86a0 2023-08-11T15:51:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.293+0000 7f67dd7fa640 1 -- 172.21.15.27:0/2150613502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8006be0 con 0x7f67e00c86a0 2023-08-11T15:51:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.293+0000 7f67e5524640 1 -- 172.21.15.27:0/2150613502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e00c7bc0 con 0x7f67e00c86a0 2023-08-11T15:51:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.293+0000 7f67e5524640 1 -- 172.21.15.27:0/2150613502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e00c8180 con 0x7f67e00c86a0 2023-08-11T15:51:10.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.294+0000 7f67e5524640 1 -- 172.21.15.27:0/2150613502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e00c8310 con 0x7f67e00c86a0 2023-08-11T15:51:10.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.295+0000 7f67dd7fa640 1 -- 172.21.15.27:0/2150613502 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f67c8006d40 con 0x7f67e00c86a0 2023-08-11T15:51:10.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.296+0000 7f67dd7fa640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67c006a080 0x7f67c006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:10.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.296+0000 7f67dd7fa640 1 -- 172.21.15.27:0/2150613502 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f67c80aa9a0 con 0x7f67e00c86a0 2023-08-11T15:51:10.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.296+0000 7f67dffff640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67c006a080 0x7f67c006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:10.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.297+0000 7f67dffff640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67c006a080 0x7f67c006c540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67d0001200 tx=0x7f67d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:10.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.300+0000 7f67dd7fa640 1 -- 172.21.15.27:0/2150613502 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f67c8079820 con 0x7f67e00c86a0 2023-08-11T15:51:10.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.531+0000 7f67e5524640 1 -- 172.21.15.27:0/2150613502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f67e004e420 con 0x7f67e00c86a0 2023-08-11T15:51:10.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.532+0000 7f67dd7fa640 1 -- 172.21.15.27:0/2150613502 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f67c80793b0 con 0x7f67e00c86a0 2023-08-11T15:51:10.535 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-11T15:51:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.534+0000 7f67e5524640 1 -- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67c006a080 msgr2=0x7f67c006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.534+0000 7f67e5524640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67c006a080 0x7f67c006c540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67d0001200 tx=0x7f67d0009040 comp rx=0 tx=0).stop 2023-08-11T15:51:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.534+0000 7f67e5524640 1 -- 172.21.15.27:0/2150613502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e00c86a0 msgr2=0x7f67e00ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:10.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.534+0000 7f67e5524640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e00c86a0 0x7f67e00ca310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f67c8008fd0 tx=0x7f67c8001120 comp rx=0 tx=0).stop 2023-08-11T15:51:10.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.535+0000 7f67e5524640 1 -- 172.21.15.27:0/2150613502 shutdown_connections 2023-08-11T15:51:10.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.535+0000 7f67e5524640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67c006a080 0x7f67c006c540 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.535+0000 7f67e5524640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f67e00c8b20 0x7f67e00ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.535+0000 7f67e5524640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e00c86a0 0x7f67e00ca310 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.535+0000 7f67e5524640 1 --2- 172.21.15.27:0/2150613502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67cc0047c0 0x7f67e01454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:10.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.535+0000 7f67e5524640 1 -- 172.21.15.27:0/2150613502 >> 172.21.15.27:0/2150613502 conn(0x7f67e006a250 msgr2=0x7f67e015b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:10.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.535+0000 7f67e5524640 1 -- 172.21.15.27:0/2150613502 shutdown_connections 2023-08-11T15:51:10.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:10.535+0000 7f67e5524640 1 -- 172.21.15.27:0/2150613502 wait complete. 2023-08-11T15:51:10.555 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-11T15:51:10.555 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T15:51:10.556 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-08-11T15:51:10.556 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-08-11T15:51:10.556 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:51:10.556 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-11T15:51:10.557 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T15:51:10.557 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T15:51:10.557 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-08-11T15:51:10.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:10.558 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-08-11T15:51:10.614 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:51:10.615 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:51:10.615 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:51:10.615 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:10.615 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-08-11T15:51:10.615 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:10.615 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:10.616 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:10.616 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-08-11T15:51:10.616 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:10.616 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:10.616 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-08-11T15:51:10.616 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T15:51:10.617 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:10.617 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-08-11T15:51:10.617 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-11T15:51:10.617 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42030sec preferred_lft 42030sec 2023-08-11T15:51:10.617 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2023-08-11T15:51:10.617 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:10.617 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:10.618 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:10.618 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-08-11T15:51:10.619 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-11T15:51:10.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:10.619 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:10.620 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:10.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2023-08-11T15:51:10.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2023-08-11T15:51:10.621 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2023-08-11T15:51:10.621 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-11T15:51:10.621 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:11.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:11.206 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T15:51:11.300 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-08-11T15:51:11.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:11.303 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-08-11T15:51:11.364 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-08-11T15:51:11.365 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T15:51:11.365 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-11T15:51:11.365 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-11T15:51:11.365 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T15:51:11.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:11.366 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:11.367 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:11.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-11T15:51:11.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-11T15:51:11.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-11T15:51:11.368 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:12.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:12.291 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-08-11T15:51:12.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:12.354 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2023-08-11T15:51:12.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:12.414 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:12.414 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:12.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-11T15:51:12.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-08-11T15:51:12.415 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:12.588 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-08-11T15:51:12.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:12.589 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:12.589 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:12.589 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-08-11T15:51:12.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-11T15:51:12.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-08-11T15:51:12.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-08-11T15:51:12.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-08-11T15:51:12.591 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:12.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:12.925 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:12.925 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:12.925 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-08-11T15:51:12.926 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2023-08-11T15:51:12.926 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:13.115 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-08-11T15:51:13.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:51:13.116 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:51:13.167 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-08-11T15:51:13.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:51:13.167 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:51:13.222 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-08-11T15:51:13.328 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:13.328 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:13.427 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:13.431 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:13.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:13.432 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-08-11T15:51:13.487 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-11T15:51:13.487 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-08-11T15:51:13.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-08-11T15:51:13.577 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:13.578 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:13.659 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:13.662 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:13.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:13.663 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-08-11T15:51:13.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:2023-08-11T15:51:13.701+0000 7f3af5648480 -1 init, newargv = 0x55ab57058c80 newargc=15 2023-08-11T15:51:13.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[95419]: starting ceph client 2023-08-11T15:51:13.718 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[95419]: starting fuse 2023-08-11T15:51:14.809 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-08-11T15:51:14.932 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:14.933 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:15.043 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:15.048 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:15.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:15.049 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-08-11T15:51:15.109 INFO:teuthology.orchestra.run.smithi027.stdout:39 2023-08-11T15:51:15.109 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-11T15:51:15.110 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2023-08-11T15:51:15.110 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-08-11T15:51:15.110 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-08-11T15:51:15.110 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-08-11T15:51:15.111 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-08-11T15:51:15.111 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:15.111 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-08-11T15:51:15.112 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-08-11T15:51:15.112 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-08-11T15:51:15.112 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-08-11T15:51:15.113 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:15.113 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-08-11T15:51:15.113 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-08-11T15:51:15.113 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-08-11T15:51:15.114 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-08-11T15:51:15.114 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:15.115 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-08-11T15:51:15.115 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-11T15:51:15.115 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-11T15:51:15.115 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-11T15:51:15.116 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-08-11T15:51:15.116 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-08-11T15:51:15.116 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-08-11T15:51:15.116 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-11T15:51:15.117 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:15.117 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.admin")) 2023-08-11T15:51:15.117 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-08-11T15:51:15.317 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.admin.95419.asok 2023-08-11T15:51:15.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-11T15:51:15.335 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.95419.asok 2023-08-11T15:51:15.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:15.336 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.95419.asok status 2023-08-11T15:51:15.600 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:51:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-08-11T15:51:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "92684807f016ce045ac66e37fdde6f8c47857bcc", 2023-08-11T15:51:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:51:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "admin", 2023-08-11T15:51:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-08-11T15:51:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-08-11T15:51:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "95419", 2023-08-11T15:51:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-08-11T15:51:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-08-11T15:51:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-08-11T15:51:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4416, 2023-08-11T15:51:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-08-11T15:51:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-08-11T15:51:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-08-11T15:51:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 4416 2023-08-11T15:51:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-08-11T15:51:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-08-11T15:51:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:51:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1645362194 2023-08-11T15:51:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:51:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-08-11T15:51:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-08-11T15:51:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:51:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1645362194 2023-08-11T15:51:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.4416 192.168.0.1:0/1645362194", 2023-08-11T15:51:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/1645362194", 2023-08-11T15:51:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-08-11T15:51:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 7, 2023-08-11T15:51:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 28, 2023-08-11T15:51:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-08-11T15:51:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-08-11T15:51:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-08-11T15:51:15.605 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:51:15.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:15.617 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-08-11T15:51:15.617 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:51:15.676 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-08-11T15:51:15.676 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:51:15.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:15.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:51:15.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:15.785 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-08-11T15:51:15.878 INFO:teuthology.orchestra.run.smithi027.stdout:# file: . 2023-08-11T15:51:15.878 INFO:teuthology.orchestra.run.smithi027.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-08-11T15:51:15.878 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:15.879 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:51:15.880 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:51:15.939 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:16.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.178+0000 7f2218287640 1 Processor -- start 2023-08-11T15:51:16.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.178+0000 7f2218287640 1 -- start start 2023-08-11T15:51:16.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.179+0000 7f2218287640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 0x7f22100b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.179+0000 7f2218287640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22100cfb00 0x7f22100d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.179+0000 7f2218287640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22100cda30 con 0x7f22100b82f0 2023-08-11T15:51:16.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.179+0000 7f2218287640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22100ccc10 con 0x7f22100cfb00 2023-08-11T15:51:16.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.179+0000 7f2218287640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2210161110 con 0x7f221015c620 2023-08-11T15:51:16.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.180+0000 7f2216824640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22100cfb00 0x7f22100d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.180+0000 7f2216023640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 0x7f22100b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.180+0000 7f2216824640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22100cfb00 0x7f22100d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55964/0 (socket says 172.21.15.27:55964) 2023-08-11T15:51:16.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.180+0000 7f2216023640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 0x7f22100b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:51504/0 (socket says 172.21.15.27:51504) 2023-08-11T15:51:16.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.180+0000 7f2216824640 1 -- 172.21.15.27:0/957610769 learned_addr learned my addr 172.21.15.27:0/957610769 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:16.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.181+0000 7f2216824640 1 -- 172.21.15.27:0/957610769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 msgr2=0x7f22100b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:16.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.181+0000 7f2216824640 1 --2- 172.21.15.27:0/957610769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 0x7f22100b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.181+0000 7f2216824640 1 -- 172.21.15.27:0/957610769 >> v1:172.21.15.119:6789/0 conn(0x7f22100b82f0 legacy=0x7f22100cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.181+0000 7f2216824640 1 -- 172.21.15.27:0/957610769 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100cce20 con 0x7f22100cfb00 2023-08-11T15:51:16.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.181+0000 7f2216023640 1 --2- 172.21.15.27:0/957610769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 0x7f22100b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:16.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.182+0000 7f2216824640 1 --2- 172.21.15.27:0/957610769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22100cfb00 0x7f22100d0b00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2208009870 tx=0x7f220802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5227e4eaa9ad8627 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:16.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.183+0000 7f2215021640 1 -- 172.21.15.27:0/957610769 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208030840 con 0x7f22100cfb00 2023-08-11T15:51:16.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.183+0000 7f2215021640 1 -- 172.21.15.27:0/957610769 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22080309a0 con 0x7f22100cfb00 2023-08-11T15:51:16.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.183+0000 7f2215021640 1 -- 172.21.15.27:0/957610769 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208030d30 con 0x7f22100cfb00 2023-08-11T15:51:16.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.183+0000 7f2218287640 1 -- 172.21.15.27:0/957610769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22100cfb00 msgr2=0x7f22100d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.183+0000 7f2218287640 1 --2- 172.21.15.27:0/957610769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22100cfb00 0x7f22100d0b00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2208009870 tx=0x7f220802b980 comp rx=0 tx=0).stop 2023-08-11T15:51:16.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.184+0000 7f2218287640 1 -- 172.21.15.27:0/957610769 shutdown_connections 2023-08-11T15:51:16.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.184+0000 7f2218287640 1 --2- 172.21.15.27:0/957610769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22100cfb00 0x7f22100d0b00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.184+0000 7f2218287640 1 --2- 172.21.15.27:0/957610769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 0x7f22100b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.184+0000 7f2218287640 1 -- 172.21.15.27:0/957610769 >> 172.21.15.27:0/957610769 conn(0x7f2210071110 msgr2=0x7f22100714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:16.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.184+0000 7f2218287640 1 -- 172.21.15.27:0/957610769 shutdown_connections 2023-08-11T15:51:16.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.184+0000 7f2218287640 1 -- 172.21.15.27:0/957610769 wait complete. 2023-08-11T15:51:16.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.185+0000 7f2218287640 1 Processor -- start 2023-08-11T15:51:16.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.186+0000 7f2218287640 1 -- start start 2023-08-11T15:51:16.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.186+0000 7f2218287640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22100b82f0 0x7f221014a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.186+0000 7f2218287640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22100cfb00 0x7f22100d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.186+0000 7f2218287640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 0x7f22100c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.186+0000 7f2218287640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221015e940 con 0x7f22100cfb00 2023-08-11T15:51:16.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2218287640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22100d23c0 con 0x7f22100b82f0 2023-08-11T15:51:16.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2218287640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22100c8da0 con 0x7f221015c620 2023-08-11T15:51:16.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2215822640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22100cfb00 0x7f22100d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2215822640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22100cfb00 0x7f22100d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43390/0 (socket says 172.21.15.27:43390) 2023-08-11T15:51:16.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2215822640 1 -- 172.21.15.27:0/1819413948 learned_addr learned my addr 172.21.15.27:0/1819413948 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:16.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2216023640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22100b82f0 0x7f221014a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2216824640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 0x7f22100c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2215822640 1 -- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 msgr2=0x7f22100c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2215822640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 0x7f22100c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2215822640 1 -- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22100b82f0 msgr2=0x7f221014a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2215822640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22100b82f0 0x7f221014a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.187+0000 7f2215822640 1 -- 172.21.15.27:0/1819413948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2200009a70 con 0x7f22100cfb00 2023-08-11T15:51:16.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.188+0000 7f2215822640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22100cfb00 0x7f22100d1e80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f220400c3c0 tx=0x7f220400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:16.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.188+0000 7f220effd640 1 -- 172.21.15.27:0/1819413948 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22040052d0 con 0x7f22100cfb00 2023-08-11T15:51:16.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.188+0000 7f220effd640 1 -- 172.21.15.27:0/1819413948 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2204005450 con 0x7f22100cfb00 2023-08-11T15:51:16.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.188+0000 7f220effd640 1 -- 172.21.15.27:0/1819413948 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204003830 con 0x7f22100cfb00 2023-08-11T15:51:16.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.188+0000 7f2218287640 1 -- 172.21.15.27:0/1819413948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22080094d0 con 0x7f22100cfb00 2023-08-11T15:51:16.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.188+0000 7f2218287640 1 -- 172.21.15.27:0/1819413948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221015dd30 con 0x7f22100cfb00 2023-08-11T15:51:16.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.190+0000 7f2218287640 1 -- 172.21.15.27:0/1819413948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2210073990 con 0x7f22100cfb00 2023-08-11T15:51:16.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.191+0000 7f220effd640 1 -- 172.21.15.27:0/1819413948 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2204003990 con 0x7f22100cfb00 2023-08-11T15:51:16.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.192+0000 7f220effd640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f21fc06a050 0x7f21fc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.192+0000 7f220effd640 1 -- 172.21.15.27:0/1819413948 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2204087cd0 con 0x7f22100cfb00 2023-08-11T15:51:16.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.192+0000 7f2216023640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f21fc06a050 0x7f21fc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.193+0000 7f2216023640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f21fc06a050 0x7f21fc06c510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2200009fd0 tx=0x7f220000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:16.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.196+0000 7f220effd640 1 -- 172.21.15.27:0/1819413948 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2204053170 con 0x7f22100cfb00 2023-08-11T15:51:16.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.389+0000 7f2218287640 1 -- 172.21.15.27:0/1819413948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f221015ea80 con 0x7f22100cfb00 2023-08-11T15:51:16.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.393+0000 7f220effd640 1 -- 172.21.15.27:0/1819413948 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7f2204056790 con 0x7f22100cfb00 2023-08-11T15:51:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:16.396 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:16.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.395+0000 7f2218287640 1 -- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f21fc06a050 msgr2=0x7f21fc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.395+0000 7f2218287640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f21fc06a050 0x7f21fc06c510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2200009fd0 tx=0x7f220000b040 comp rx=0 tx=0).stop 2023-08-11T15:51:16.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.395+0000 7f2218287640 1 -- 172.21.15.27:0/1819413948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22100cfb00 msgr2=0x7f22100d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.395+0000 7f2218287640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22100cfb00 0x7f22100d1e80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f220400c3c0 tx=0x7f220400c880 comp rx=0 tx=0).stop 2023-08-11T15:51:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.395+0000 7f2218287640 1 -- 172.21.15.27:0/1819413948 shutdown_connections 2023-08-11T15:51:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.396+0000 7f2218287640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f21fc06a050 0x7f21fc06c510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.396+0000 7f2218287640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f221015c620 0x7f22100c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.396+0000 7f2218287640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22100cfb00 0x7f22100d1e80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.396+0000 7f2218287640 1 --2- 172.21.15.27:0/1819413948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22100b82f0 0x7f221014a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.396+0000 7f2218287640 1 -- 172.21.15.27:0/1819413948 >> 172.21.15.27:0/1819413948 conn(0x7f2210071110 msgr2=0x7f22100b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.396+0000 7f2218287640 1 -- 172.21.15.27:0/1819413948 shutdown_connections 2023-08-11T15:51:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.396+0000 7f2218287640 1 -- 172.21.15.27:0/1819413948 wait complete. 2023-08-11T15:51:16.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:51:16.417 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:51:16.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.650+0000 7f4dad939640 1 Processor -- start 2023-08-11T15:51:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.650+0000 7f4dad939640 1 -- start start 2023-08-11T15:51:16.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.651+0000 7f4dad939640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4da815c760 0x7f4da80d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.651+0000 7f4dad939640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4da80d2a20 0x7f4da80af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.651+0000 7f4dad939640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da806bc10 con 0x7f4da806b670 2023-08-11T15:51:16.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.651+0000 7f4dad939640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da806c790 con 0x7f4da80d2a20 2023-08-11T15:51:16.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.651+0000 7f4dad939640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da806bef0 con 0x7f4da815c760 2023-08-11T15:51:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.652+0000 7f4da77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4da80d2a20 0x7f4da80af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.652+0000 7f4da77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4da80d2a20 0x7f4da80af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43396/0 (socket says 172.21.15.27:43396) 2023-08-11T15:51:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.652+0000 7f4da77fe640 1 -- 172.21.15.27:0/1244721322 learned_addr learned my addr 172.21.15.27:0/1244721322 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.652+0000 7f4da6ffd640 1 --2- 172.21.15.27:0/1244721322 >> v2:172.21.15.119:3300/0 conn(0x7f4da815c760 0x7f4da80d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.652+0000 7f4da77fe640 1 -- 172.21.15.27:0/1244721322 >> v2:172.21.15.119:3300/0 conn(0x7f4da815c760 msgr2=0x7f4da80d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.652+0000 7f4da77fe640 1 --2- 172.21.15.27:0/1244721322 >> v2:172.21.15.119:3300/0 conn(0x7f4da815c760 0x7f4da80d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.652+0000 7f4da77fe640 1 -- 172.21.15.27:0/1244721322 >> v1:172.21.15.119:6789/0 conn(0x7f4da806b670 legacy=0x7f4da80d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.652+0000 7f4da77fe640 1 -- 172.21.15.27:0/1244721322 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da806c1d0 con 0x7f4da80d2a20 2023-08-11T15:51:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.652+0000 7f4da6ffd640 1 --2- 172.21.15.27:0/1244721322 >> v2:172.21.15.119:3300/0 conn(0x7f4da815c760 0x7f4da80d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.653+0000 7f4da77fe640 1 --2- 172.21.15.27:0/1244721322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da80d2a20 0x7f4da80af610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4da80a0580 tx=0x7f4da002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387cbf1b29db0db1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.654+0000 7f4da5ffb640 1 -- 172.21.15.27:0/1244721322 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0030860 con 0x7f4da80d2a20 2023-08-11T15:51:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.654+0000 7f4da5ffb640 1 -- 172.21.15.27:0/1244721322 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da00309c0 con 0x7f4da80d2a20 2023-08-11T15:51:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.654+0000 7f4da5ffb640 1 -- 172.21.15.27:0/1244721322 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0030d50 con 0x7f4da80d2a20 2023-08-11T15:51:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.654+0000 7f4dad939640 1 -- 172.21.15.27:0/1244721322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da80d2a20 msgr2=0x7f4da80af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.654+0000 7f4dad939640 1 --2- 172.21.15.27:0/1244721322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da80d2a20 0x7f4da80af610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4da80a0580 tx=0x7f4da002b9a0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.654+0000 7f4dad939640 1 -- 172.21.15.27:0/1244721322 shutdown_connections 2023-08-11T15:51:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.654+0000 7f4dad939640 1 --2- 172.21.15.27:0/1244721322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da80d2a20 0x7f4da80af610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.654+0000 7f4dad939640 1 --2- 172.21.15.27:0/1244721322 >> v2:172.21.15.119:3300/0 conn(0x7f4da815c760 0x7f4da80d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.654+0000 7f4dad939640 1 -- 172.21.15.27:0/1244721322 >> 172.21.15.27:0/1244721322 conn(0x7f4da8089ff0 msgr2=0x7f4da809b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.655+0000 7f4dad939640 1 -- 172.21.15.27:0/1244721322 shutdown_connections 2023-08-11T15:51:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.655+0000 7f4dad939640 1 -- 172.21.15.27:0/1244721322 wait complete. 2023-08-11T15:51:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.656+0000 7f4dad939640 1 Processor -- start 2023-08-11T15:51:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.656+0000 7f4dad939640 1 -- start start 2023-08-11T15:51:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.657+0000 7f4dad939640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da806b670 0x7f4da80cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.657+0000 7f4dad939640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da80d2a20 0x7f4da80b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.657+0000 7f4dad939640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4da815c760 0x7f4da80b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.657+0000 7f4dad939640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80cfe00 con 0x7f4da806b670 2023-08-11T15:51:16.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.657+0000 7f4dad939640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80b5f20 con 0x7f4da80d2a20 2023-08-11T15:51:16.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.657+0000 7f4dad939640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80b6140 con 0x7f4da815c760 2023-08-11T15:51:16.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.658+0000 7f4da6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da806b670 0x7f4da80cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.658+0000 7f4da6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da806b670 0x7f4da80cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43398/0 (socket says 172.21.15.27:43398) 2023-08-11T15:51:16.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.658+0000 7f4da6ffd640 1 -- 172.21.15.27:0/2092429018 learned_addr learned my addr 172.21.15.27:0/2092429018 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:16.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.658+0000 7f4da67fc640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da80d2a20 0x7f4da80b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.658+0000 7f4da77fe640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4da815c760 0x7f4da80b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.658+0000 7f4da6ffd640 1 -- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4da815c760 msgr2=0x7f4da80b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.658+0000 7f4da6ffd640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4da815c760 0x7f4da80b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.658+0000 7f4da6ffd640 1 -- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da80d2a20 msgr2=0x7f4da80b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.658+0000 7f4da6ffd640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da80d2a20 0x7f4da80b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.658+0000 7f4da6ffd640 1 -- 172.21.15.27:0/2092429018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00094d0 con 0x7f4da806b670 2023-08-11T15:51:16.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.659+0000 7f4da6ffd640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da806b670 0x7f4da80cd510 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800d800 tx=0x7f4d9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.659+0000 7f4d9ffff640 1 -- 172.21.15.27:0/2092429018 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98004030 con 0x7f4da806b670 2023-08-11T15:51:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.659+0000 7f4d9ffff640 1 -- 172.21.15.27:0/2092429018 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98004190 con 0x7f4da806b670 2023-08-11T15:51:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.659+0000 7f4d9ffff640 1 -- 172.21.15.27:0/2092429018 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98004520 con 0x7f4da806b670 2023-08-11T15:51:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.659+0000 7f4dad939640 1 -- 172.21.15.27:0/2092429018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da80ceb10 con 0x7f4da806b670 2023-08-11T15:51:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.659+0000 7f4dad939640 1 -- 172.21.15.27:0/2092429018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da80cf0d0 con 0x7f4da806b670 2023-08-11T15:51:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.660+0000 7f4dad939640 1 -- 172.21.15.27:0/2092429018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da80ce970 con 0x7f4da806b670 2023-08-11T15:51:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.661+0000 7f4d9ffff640 1 -- 172.21.15.27:0/2092429018 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4d98009640 con 0x7f4da806b670 2023-08-11T15:51:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.662+0000 7f4d9ffff640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4d8806a050 0x7f4d8806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.662+0000 7f4d9ffff640 1 -- 172.21.15.27:0/2092429018 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4d98088a50 con 0x7f4da806b670 2023-08-11T15:51:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.663+0000 7f4da67fc640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4d8806a050 0x7f4d8806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.664+0000 7f4da67fc640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4d8806a050 0x7f4d8806c510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4da80cc350 tx=0x7f4d90009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.667+0000 7f4d9ffff640 1 -- 172.21.15.27:0/2092429018 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f4da80ce970 con 0x7f4da806b670 2023-08-11T15:51:16.857 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:51:16.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:51:16.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.864+0000 7f4dad939640 1 -- 172.21.15.27:0/2092429018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4da80ce970 con 0x7f4da806b670 2023-08-11T15:51:16.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.865+0000 7f4d9ffff640 1 -- 172.21.15.27:0/2092429018 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4da80ce970 con 0x7f4da806b670 2023-08-11T15:51:16.868 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:51:16.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.868+0000 7f4dad939640 1 -- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4d8806a050 msgr2=0x7f4d8806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.868+0000 7f4dad939640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4d8806a050 0x7f4d8806c510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4da80cc350 tx=0x7f4d90009040 comp rx=0 tx=0).stop 2023-08-11T15:51:16.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.868+0000 7f4dad939640 1 -- 172.21.15.27:0/2092429018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da806b670 msgr2=0x7f4da80cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:16.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.868+0000 7f4dad939640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da806b670 0x7f4da80cd510 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800d800 tx=0x7f4d9800dcc0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.868+0000 7f4dad939640 1 -- 172.21.15.27:0/2092429018 shutdown_connections 2023-08-11T15:51:16.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.868+0000 7f4dad939640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4d8806a050 0x7f4d8806c510 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.868+0000 7f4dad939640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4da815c760 0x7f4da80b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.868+0000 7f4dad939640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da80d2a20 0x7f4da80b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.868+0000 7f4dad939640 1 --2- 172.21.15.27:0/2092429018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da806b670 0x7f4da80cd510 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:16.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.868+0000 7f4dad939640 1 -- 172.21.15.27:0/2092429018 >> 172.21.15.27:0/2092429018 conn(0x7f4da8089ff0 msgr2=0x7f4da80b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:16.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.869+0000 7f4dad939640 1 -- 172.21.15.27:0/2092429018 shutdown_connections 2023-08-11T15:51:16.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:16.869+0000 7f4dad939640 1 -- 172.21.15.27:0/2092429018 wait complete. 2023-08-11T15:51:16.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:51:17.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.150+0000 7fdbbca71640 1 Processor -- start 2023-08-11T15:51:17.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.150+0000 7fdbbca71640 1 -- start start 2023-08-11T15:51:17.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.151+0000 7fdbbca71640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbb815c760 0x7fdbb80d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbbca71640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdbb806b670 0x7fdbb80d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbbca71640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdbb80d2850 0x7fdbb81602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbbca71640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb806c790 con 0x7fdbb806b670 2023-08-11T15:51:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbbca71640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb806bc10 con 0x7fdbb80d2850 2023-08-11T15:51:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbbca71640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb806bef0 con 0x7fdbb815c760 2023-08-11T15:51:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbadd9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdbb806b670 0x7fdbb80d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbadd9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdbb806b670 0x7fdbb80d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43410/0 (socket says 172.21.15.27:43410) 2023-08-11T15:51:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbadd9b640 1 -- 172.21.15.27:0/1838405262 learned_addr learned my addr 172.21.15.27:0/1838405262 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:17.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbb6d9d640 1 --2- 172.21.15.27:0/1838405262 >> v2:172.21.15.119:3300/0 conn(0x7fdbb80d2850 0x7fdbb81602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbadd9b640 1 -- 172.21.15.27:0/1838405262 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbb815c760 msgr2=0x7fdbb80d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:17.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbadd9b640 1 --2- 172.21.15.27:0/1838405262 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbb815c760 0x7fdbb80d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbadd9b640 1 -- 172.21.15.27:0/1838405262 >> v2:172.21.15.119:3300/0 conn(0x7fdbb80d2850 msgr2=0x7fdbb81602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:17.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbadd9b640 1 --2- 172.21.15.27:0/1838405262 >> v2:172.21.15.119:3300/0 conn(0x7fdbb80d2850 0x7fdbb81602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbadd9b640 1 -- 172.21.15.27:0/1838405262 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb806c1d0 con 0x7fdbb806b670 2023-08-11T15:51:17.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.152+0000 7fdbb6d9d640 1 --2- 172.21.15.27:0/1838405262 >> v2:172.21.15.119:3300/0 conn(0x7fdbb80d2850 0x7fdbb81602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:17.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.154+0000 7fdbadd9b640 1 --2- 172.21.15.27:0/1838405262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 0x7fdbb80d2310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdbb808cdc0 tx=0x7fdba002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0692271eb075f29 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:17.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.154+0000 7fdbb5d9b640 1 -- 172.21.15.27:0/1838405262 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0031870 con 0x7fdbb806b670 2023-08-11T15:51:17.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.154+0000 7fdbb5d9b640 1 -- 172.21.15.27:0/1838405262 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba00319d0 con 0x7fdbb806b670 2023-08-11T15:51:17.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.154+0000 7fdbb5d9b640 1 -- 172.21.15.27:0/1838405262 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0031d60 con 0x7fdbb806b670 2023-08-11T15:51:17.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.155+0000 7fdbbca71640 1 -- 172.21.15.27:0/1838405262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 msgr2=0x7fdbb80d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.155+0000 7fdbbca71640 1 --2- 172.21.15.27:0/1838405262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 0x7fdbb80d2310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdbb808cdc0 tx=0x7fdba002b910 comp rx=0 tx=0).stop 2023-08-11T15:51:17.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.155+0000 7fdbbca71640 1 -- 172.21.15.27:0/1838405262 shutdown_connections 2023-08-11T15:51:17.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.155+0000 7fdbbca71640 1 --2- 172.21.15.27:0/1838405262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 0x7fdbb80d2310 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.155+0000 7fdbbca71640 1 --2- 172.21.15.27:0/1838405262 >> v2:172.21.15.119:3300/0 conn(0x7fdbb80d2850 0x7fdbb81602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.155+0000 7fdbbca71640 1 --2- 172.21.15.27:0/1838405262 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbb815c760 0x7fdbb80d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.155+0000 7fdbbca71640 1 -- 172.21.15.27:0/1838405262 >> 172.21.15.27:0/1838405262 conn(0x7fdbb8089ff0 msgr2=0x7fdbb809b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:17.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.155+0000 7fdbbca71640 1 -- 172.21.15.27:0/1838405262 shutdown_connections 2023-08-11T15:51:17.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.155+0000 7fdbbca71640 1 -- 172.21.15.27:0/1838405262 wait complete. 2023-08-11T15:51:17.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.157+0000 7fdbbca71640 1 Processor -- start 2023-08-11T15:51:17.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.157+0000 7fdbbca71640 1 -- start start 2023-08-11T15:51:17.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbbca71640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 0x7fdbb815cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbbca71640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbb80d2850 0x7fdbb81f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbbca71640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbb815c760 0x7fdbb81f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbbca71640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb815d2c0 con 0x7fdbb806b670 2023-08-11T15:51:17.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbbca71640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb815d560 con 0x7fdbb80d2850 2023-08-11T15:51:17.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbbca71640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb815d840 con 0x7fdbb815c760 2023-08-11T15:51:17.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbb659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 0x7fdbb815cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbb659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 0x7fdbb815cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43416/0 (socket says 172.21.15.27:43416) 2023-08-11T15:51:17.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbb659c640 1 -- 172.21.15.27:0/1733757677 learned_addr learned my addr 172.21.15.27:0/1733757677 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:17.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbadd9b640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbb80d2850 0x7fdbb81f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.158+0000 7fdbb6d9d640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbb815c760 0x7fdbb81f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbb659c640 1 -- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbb815c760 msgr2=0x7fdbb81f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbb659c640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbb815c760 0x7fdbb81f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbb659c640 1 -- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbb80d2850 msgr2=0x7fdbb81f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbb659c640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbb80d2850 0x7fdbb81f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbb659c640 1 -- 172.21.15.27:0/1733757677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba00094d0 con 0x7fdbb806b670 2023-08-11T15:51:17.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbadd9b640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbb80d2850 0x7fdbb81f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:17.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbb659c640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 0x7fdbb815cd80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdba800b410 tx=0x7fdba800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:17.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbaf7fe640 1 -- 172.21.15.27:0/1733757677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8011830 con 0x7fdbb806b670 2023-08-11T15:51:17.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbaf7fe640 1 -- 172.21.15.27:0/1733757677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba80119b0 con 0x7fdbb806b670 2023-08-11T15:51:17.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbaf7fe640 1 -- 172.21.15.27:0/1733757677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8011d60 con 0x7fdbb806b670 2023-08-11T15:51:17.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbbca71640 1 -- 172.21.15.27:0/1733757677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb81f1df0 con 0x7fdbb806b670 2023-08-11T15:51:17.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.159+0000 7fdbbca71640 1 -- 172.21.15.27:0/1733757677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb815db20 con 0x7fdbb806b670 2023-08-11T15:51:17.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.160+0000 7fdbbca71640 1 -- 172.21.15.27:0/1733757677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb81f2310 con 0x7fdbb806b670 2023-08-11T15:51:17.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.161+0000 7fdbaf7fe640 1 -- 172.21.15.27:0/1733757677 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdba801a400 con 0x7fdbb806b670 2023-08-11T15:51:17.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.162+0000 7fdbaf7fe640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdb9806a050 0x7fdb9806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.162+0000 7fdbaf7fe640 1 -- 172.21.15.27:0/1733757677 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdba8088af0 con 0x7fdbb806b670 2023-08-11T15:51:17.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.163+0000 7fdbadd9b640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdb9806a050 0x7fdb9806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.164+0000 7fdbadd9b640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdb9806a050 0x7fdb9806c510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdba0036000 tx=0x7fdba003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:17.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.167+0000 7fdbaf7fe640 1 -- 172.21.15.27:0/1733757677 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fdbb81f2310 con 0x7fdbb806b670 2023-08-11T15:51:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.369+0000 7fdbbca71640 1 -- 172.21.15.27:0/1733757677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdbb81f2310 con 0x7fdbb806b670 2023-08-11T15:51:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.370+0000 7fdbaf7fe640 1 -- 172.21.15.27:0/1733757677 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdbb81f2310 con 0x7fdbb806b670 2023-08-11T15:51:17.373 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:51:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 -- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdb9806a050 msgr2=0x7fdb9806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdb9806a050 0x7fdb9806c510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdba0036000 tx=0x7fdba003c040 comp rx=0 tx=0).stop 2023-08-11T15:51:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 -- 172.21.15.27:0/1733757677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 msgr2=0x7fdbb815cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 0x7fdbb815cd80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdba800b410 tx=0x7fdba800b8d0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 -- 172.21.15.27:0/1733757677 shutdown_connections 2023-08-11T15:51:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdb9806a050 0x7fdb9806c510 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbb815c760 0x7fdbb81f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbb80d2850 0x7fdbb81f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 --2- 172.21.15.27:0/1733757677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb806b670 0x7fdbb815cd80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 -- 172.21.15.27:0/1733757677 >> 172.21.15.27:0/1733757677 conn(0x7fdbb8089ff0 msgr2=0x7fdbb815b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:17.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 -- 172.21.15.27:0/1733757677 shutdown_connections 2023-08-11T15:51:17.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.373+0000 7fdbbca71640 1 -- 172.21.15.27:0/1733757677 wait complete. 2023-08-11T15:51:17.395 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:51:17.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.632+0000 7eff9359e640 1 Processor -- start 2023-08-11T15:51:17.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.633+0000 7eff9359e640 1 -- start start 2023-08-11T15:51:17.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.634+0000 7eff9359e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7eff940c86c0 0x7eff940afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.634+0000 7eff9359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7eff940afff0 0x7eff9406ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.634+0000 7eff9359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff940b0410 con 0x7eff9415c3f0 2023-08-11T15:51:17.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.634+0000 7eff9359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff940ca810 con 0x7eff940afff0 2023-08-11T15:51:17.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.634+0000 7eff9359e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff940c8b30 con 0x7eff940c86c0 2023-08-11T15:51:17.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.635+0000 7eff9259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7eff9415c3f0 0x7eff940cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55432/0 (socket says 172.21.15.27:55432) 2023-08-11T15:51:17.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.635+0000 7eff9259c640 1 -- 172.21.15.27:0/1173258543 learned_addr learned my addr 172.21.15.27:0/1173258543 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:17.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.635+0000 7eff92d9d640 1 --2- 172.21.15.27:0/1173258543 >> v2:172.21.15.27:3300/0 conn(0x7eff940afff0 0x7eff9406ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.635+0000 7eff89d9b640 1 --2- 172.21.15.27:0/1173258543 >> v2:172.21.15.119:3300/0 conn(0x7eff940c86c0 0x7eff940afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.635+0000 7eff91d9b640 1 -- 172.21.15.27:0/1173258543 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662876927 0 0) 0x7eff940b0410 con 0x7eff9415c3f0 2023-08-11T15:51:17.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.635+0000 7eff92d9d640 1 -- 172.21.15.27:0/1173258543 >> v2:172.21.15.119:3300/0 conn(0x7eff940c86c0 msgr2=0x7eff940afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:17.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.635+0000 7eff92d9d640 1 --2- 172.21.15.27:0/1173258543 >> v2:172.21.15.119:3300/0 conn(0x7eff940c86c0 0x7eff940afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.635+0000 7eff92d9d640 1 -- 172.21.15.27:0/1173258543 >> v1:172.21.15.27:6789/0 conn(0x7eff9415c3f0 legacy=0x7eff940cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.635+0000 7eff92d9d640 1 -- 172.21.15.27:0/1173258543 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff941600f0 con 0x7eff940afff0 2023-08-11T15:51:17.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.636+0000 7eff92d9d640 1 --2- 172.21.15.27:0/1173258543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff940afff0 0x7eff9406ba90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff940c50b0 tx=0x7eff8c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e7013192660c66 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:17.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.637+0000 7eff91d9b640 1 -- 172.21.15.27:0/1173258543 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c02cd00 con 0x7eff940afff0 2023-08-11T15:51:17.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.637+0000 7eff91d9b640 1 -- 172.21.15.27:0/1173258543 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c02ce60 con 0x7eff940afff0 2023-08-11T15:51:17.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.637+0000 7eff91d9b640 1 -- 172.21.15.27:0/1173258543 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c034660 con 0x7eff940afff0 2023-08-11T15:51:17.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.637+0000 7eff9359e640 1 -- 172.21.15.27:0/1173258543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff940afff0 msgr2=0x7eff9406ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.637+0000 7eff9359e640 1 --2- 172.21.15.27:0/1173258543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff940afff0 0x7eff9406ba90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff940c50b0 tx=0x7eff8c02bf50 comp rx=0 tx=0).stop 2023-08-11T15:51:17.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.637+0000 7eff9359e640 1 -- 172.21.15.27:0/1173258543 shutdown_connections 2023-08-11T15:51:17.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.637+0000 7eff9359e640 1 --2- 172.21.15.27:0/1173258543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff940afff0 0x7eff9406ba90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.637+0000 7eff9359e640 1 --2- 172.21.15.27:0/1173258543 >> v2:172.21.15.119:3300/0 conn(0x7eff940c86c0 0x7eff940afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.638+0000 7eff9359e640 1 -- 172.21.15.27:0/1173258543 >> 172.21.15.27:0/1173258543 conn(0x7eff940caab0 msgr2=0x7eff940cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.638+0000 7eff9359e640 1 -- 172.21.15.27:0/1173258543 shutdown_connections 2023-08-11T15:51:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.638+0000 7eff9359e640 1 -- 172.21.15.27:0/1173258543 wait complete. 2023-08-11T15:51:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.639+0000 7eff9359e640 1 Processor -- start 2023-08-11T15:51:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.640+0000 7eff9359e640 1 -- start start 2023-08-11T15:51:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff9359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff940afff0 0x7eff9414b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff9359e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff940c86c0 0x7eff940af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff9359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9415c3f0 0x7eff940c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff9359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff941584a0 con 0x7eff9415c3f0 2023-08-11T15:51:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff9359e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff941578d0 con 0x7eff940afff0 2023-08-11T15:51:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff9359e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff94157b70 con 0x7eff940c86c0 2023-08-11T15:51:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff92d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9415c3f0 0x7eff940c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff92d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9415c3f0 0x7eff940c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43440/0 (socket says 172.21.15.27:43440) 2023-08-11T15:51:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff92d9d640 1 -- 172.21.15.27:0/4011409945 learned_addr learned my addr 172.21.15.27:0/4011409945 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff9259c640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff940afff0 0x7eff9414b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.641+0000 7eff89d9b640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff940c86c0 0x7eff940af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.642+0000 7eff92d9d640 1 -- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff940c86c0 msgr2=0x7eff940af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.642+0000 7eff92d9d640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff940c86c0 0x7eff940af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.642+0000 7eff92d9d640 1 -- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff940afff0 msgr2=0x7eff9414b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.642+0000 7eff92d9d640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff940afff0 0x7eff9414b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.642+0000 7eff92d9d640 1 -- 172.21.15.27:0/4011409945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c009a70 con 0x7eff9415c3f0 2023-08-11T15:51:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.642+0000 7eff92d9d640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9415c3f0 0x7eff940c6300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009e80 tx=0x7eff8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.642+0000 7eff8b7fe640 1 -- 172.21.15.27:0/4011409945 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c003b90 con 0x7eff9415c3f0 2023-08-11T15:51:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.642+0000 7eff8b7fe640 1 -- 172.21.15.27:0/4011409945 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c003cf0 con 0x7eff9415c3f0 2023-08-11T15:51:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.642+0000 7eff8b7fe640 1 -- 172.21.15.27:0/4011409945 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c004230 con 0x7eff9415c3f0 2023-08-11T15:51:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.642+0000 7eff9359e640 1 -- 172.21.15.27:0/4011409945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff94157e50 con 0x7eff9415c3f0 2023-08-11T15:51:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.643+0000 7eff9359e640 1 -- 172.21.15.27:0/4011409945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff94158930 con 0x7eff9415c3f0 2023-08-11T15:51:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.644+0000 7eff9359e640 1 -- 172.21.15.27:0/4011409945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff94158650 con 0x7eff9415c3f0 2023-08-11T15:51:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.645+0000 7eff8b7fe640 1 -- 172.21.15.27:0/4011409945 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7eff8c03b050 con 0x7eff9415c3f0 2023-08-11T15:51:17.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.646+0000 7eff8b7fe640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7eff7406a050 0x7eff7406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:17.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.646+0000 7eff8b7fe640 1 -- 172.21.15.27:0/4011409945 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7eff8c0ab360 con 0x7eff9415c3f0 2023-08-11T15:51:17.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.646+0000 7eff9259c640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7eff7406a050 0x7eff7406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:17.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.647+0000 7eff9259c640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7eff7406a050 0x7eff7406c510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7eff84007fd0 tx=0x7eff84007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:17.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.650+0000 7eff8b7fe640 1 -- 172.21.15.27:0/4011409945 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7eff94158650 con 0x7eff9415c3f0 2023-08-11T15:51:17.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.849+0000 7eff9359e640 1 -- 172.21.15.27:0/4011409945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff9404e420 con 0x7eff9415c3f0 2023-08-11T15:51:17.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.850+0000 7eff8b7fe640 1 -- 172.21.15.27:0/4011409945 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff8c076870 con 0x7eff9415c3f0 2023-08-11T15:51:17.853 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:51:17.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.853+0000 7eff9359e640 1 -- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7eff7406a050 msgr2=0x7eff7406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.853+0000 7eff9359e640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7eff7406a050 0x7eff7406c510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7eff84007fd0 tx=0x7eff84007de0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.853+0000 7eff9359e640 1 -- 172.21.15.27:0/4011409945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9415c3f0 msgr2=0x7eff940c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:17.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.853+0000 7eff9359e640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9415c3f0 0x7eff940c6300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009e80 tx=0x7eff8c004a40 comp rx=0 tx=0).stop 2023-08-11T15:51:17.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.853+0000 7eff9359e640 1 -- 172.21.15.27:0/4011409945 shutdown_connections 2023-08-11T15:51:17.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.853+0000 7eff9359e640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7eff7406a050 0x7eff7406c510 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.853+0000 7eff9359e640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff940c86c0 0x7eff940af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.853+0000 7eff9359e640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9415c3f0 0x7eff940c6300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.853+0000 7eff9359e640 1 --2- 172.21.15.27:0/4011409945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff940afff0 0x7eff9414b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:17.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.853+0000 7eff9359e640 1 -- 172.21.15.27:0/4011409945 >> 172.21.15.27:0/4011409945 conn(0x7eff940caab0 msgr2=0x7eff9406c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:17.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.854+0000 7eff9359e640 1 -- 172.21.15.27:0/4011409945 shutdown_connections 2023-08-11T15:51:17.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:17.854+0000 7eff9359e640 1 -- 172.21.15.27:0/4011409945 wait complete. 2023-08-11T15:51:17.875 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:51:18.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.111+0000 7ff4d197f640 1 Processor -- start 2023-08-11T15:51:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.111+0000 7ff4d197f640 1 -- start start 2023-08-11T15:51:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.113+0000 7ff4d197f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4cc06df00 0x7ff4cc1670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.113+0000 7ff4d197f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc0cf6a0 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.113+0000 7ff4d197f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc0ceb20 con 0x7ff4cc15c430 2023-08-11T15:51:18.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.113+0000 7ff4d197f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4cc06d660 con 0x7ff4cc06df00 2023-08-11T15:51:18.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.113+0000 7ff4cbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff4cc06d1d0 0x7ff4cc06d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55440/0 (socket says 172.21.15.27:55440) 2023-08-11T15:51:18.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.114+0000 7ff4cbfff640 1 -- 172.21.15.27:0/2598354798 learned_addr learned my addr 172.21.15.27:0/2598354798 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.114+0000 7ff4d117e640 1 --2- 172.21.15.27:0/2598354798 >> v2:172.21.15.119:3300/0 conn(0x7ff4cc06df00 0x7ff4cc1670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.114+0000 7ff4cb7fe640 1 -- 172.21.15.27:0/2598354798 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282492673 0 0) 0x7ff4cc0cf6a0 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.114+0000 7ff4cb7fe640 1 -- 172.21.15.27:0/2598354798 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b4003520 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.115+0000 7ff4cb7fe640 1 -- 172.21.15.27:0/2598354798 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 317620404 0 0) 0x7ff4b4003520 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.115+0000 7ff4d117e640 1 -- 172.21.15.27:0/2598354798 >> v1:172.21.15.119:6789/0 conn(0x7ff4cc15c430 legacy=0x7ff4cc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.115+0000 7ff4d117e640 1 -- 172.21.15.27:0/2598354798 >> v1:172.21.15.27:6789/0 conn(0x7ff4cc06d1d0 legacy=0x7ff4cc06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.115+0000 7ff4d117e640 1 -- 172.21.15.27:0/2598354798 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc0cee00 con 0x7ff4cc06df00 2023-08-11T15:51:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.116+0000 7ff4d117e640 1 --2- 172.21.15.27:0/2598354798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06df00 0x7ff4cc1670b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc06d7a0 tx=0x7ff4c402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33be5aa0dd674dea server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.116+0000 7ff4cb7fe640 1 -- 172.21.15.27:0/2598354798 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c402f050 con 0x7ff4cc06df00 2023-08-11T15:51:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.116+0000 7ff4cb7fe640 1 -- 172.21.15.27:0/2598354798 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c4031500 con 0x7ff4cc06df00 2023-08-11T15:51:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.116+0000 7ff4cb7fe640 1 -- 172.21.15.27:0/2598354798 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0001670 con 0x7ff4cc06df00 2023-08-11T15:51:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.117+0000 7ff4d197f640 1 -- 172.21.15.27:0/2598354798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06df00 msgr2=0x7ff4cc1670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.117+0000 7ff4d197f640 1 --2- 172.21.15.27:0/2598354798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06df00 0x7ff4cc1670b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc06d7a0 tx=0x7ff4c402b820 comp rx=0 tx=0).stop 2023-08-11T15:51:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.117+0000 7ff4d197f640 1 -- 172.21.15.27:0/2598354798 shutdown_connections 2023-08-11T15:51:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.117+0000 7ff4d197f640 1 --2- 172.21.15.27:0/2598354798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06df00 0x7ff4cc1670b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.117+0000 7ff4d197f640 1 -- 172.21.15.27:0/2598354798 >> 172.21.15.27:0/2598354798 conn(0x7ff4cc099390 msgr2=0x7ff4cc0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.117+0000 7ff4d197f640 1 -- 172.21.15.27:0/2598354798 shutdown_connections 2023-08-11T15:51:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.117+0000 7ff4d197f640 1 -- 172.21.15.27:0/2598354798 wait complete. 2023-08-11T15:51:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.120+0000 7ff4d197f640 1 Processor -- start 2023-08-11T15:51:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.120+0000 7ff4d197f640 1 -- start start 2023-08-11T15:51:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.121+0000 7ff4d197f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4cc06d1d0 0x7ff4cc1e2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.121+0000 7ff4d197f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4cc06df00 0x7ff4cc0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.121+0000 7ff4d197f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc15c430 0x7ff4cc158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.121+0000 7ff4d197f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1e2320 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.121+0000 7ff4d197f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc0b26c0 con 0x7ff4cc15c430 2023-08-11T15:51:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.121+0000 7ff4d197f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1e2c80 con 0x7ff4cc06df00 2023-08-11T15:51:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4d097d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4cc06d1d0 0x7ff4cc1e2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4d097d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4cc06d1d0 0x7ff4cc1e2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43446/0 (socket says 172.21.15.27:43446) 2023-08-11T15:51:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4d097d640 1 -- 172.21.15.27:0/4264302627 learned_addr learned my addr 172.21.15.27:0/4264302627 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4cbfff640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4cc06df00 0x7ff4cc0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4d117e640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc15c430 0x7ff4cc158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4d097d640 1 -- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4cc06df00 msgr2=0x7ff4cc0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4d097d640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4cc06df00 0x7ff4cc0cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4d097d640 1 -- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc15c430 msgr2=0x7ff4cc158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4d097d640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc15c430 0x7ff4cc158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4d097d640 1 -- 172.21.15.27:0/4264302627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4009400 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.122+0000 7ff4d117e640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc15c430 0x7ff4cc158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.123+0000 7ff4d097d640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4cc06d1d0 0x7ff4cc1e2740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c730 tx=0x7ff4bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.123+0000 7ff4c97fa640 1 -- 172.21.15.27:0/4264302627 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc006da0 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.123+0000 7ff4c97fa640 1 -- 172.21.15.27:0/4264302627 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc0039c0 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.123+0000 7ff4c97fa640 1 -- 172.21.15.27:0/4264302627 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc003db0 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.123+0000 7ff4d197f640 1 -- 172.21.15.27:0/4264302627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc1e1980 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.123+0000 7ff4d197f640 1 -- 172.21.15.27:0/4264302627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc1e1f40 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.125+0000 7ff4d197f640 1 -- 172.21.15.27:0/4264302627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc1e2460 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.126+0000 7ff4c97fa640 1 -- 172.21.15.27:0/4264302627 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff4bc0052e0 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.127+0000 7ff4c97fa640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b806a050 0x7ff4b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.127+0000 7ff4c97fa640 1 -- 172.21.15.27:0/4264302627 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff4bc088690 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.128+0000 7ff4cbfff640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b806a050 0x7ff4b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.129+0000 7ff4cbfff640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b806a050 0x7ff4b806c510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc1e3500 tx=0x7ff4c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:18.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.132+0000 7ff4c97fa640 1 -- 172.21.15.27:0/4264302627 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ff4cc1e2460 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.333+0000 7ff4d197f640 1 -- 172.21.15.27:0/4264302627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4cc1e2460 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.334+0000 7ff4c97fa640 1 -- 172.21.15.27:0/4264302627 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4cc1e2460 con 0x7ff4cc06d1d0 2023-08-11T15:51:18.336 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:51:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 -- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b806a050 msgr2=0x7ff4b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b806a050 0x7ff4b806c510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc1e3500 tx=0x7ff4c0009040 comp rx=0 tx=0).stop 2023-08-11T15:51:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 -- 172.21.15.27:0/4264302627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4cc06d1d0 msgr2=0x7ff4cc1e2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4cc06d1d0 0x7ff4cc1e2740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c730 tx=0x7ff4bc00cbf0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 -- 172.21.15.27:0/4264302627 shutdown_connections 2023-08-11T15:51:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b806a050 0x7ff4b806c510 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4cc06df00 0x7ff4cc0cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc15c430 0x7ff4cc158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 --2- 172.21.15.27:0/4264302627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4cc06d1d0 0x7ff4cc1e2740 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 -- 172.21.15.27:0/4264302627 >> 172.21.15.27:0/4264302627 conn(0x7ff4cc099390 msgr2=0x7ff4cc15b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 -- 172.21.15.27:0/4264302627 shutdown_connections 2023-08-11T15:51:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.337+0000 7ff4d197f640 1 -- 172.21.15.27:0/4264302627 wait complete. 2023-08-11T15:51:18.360 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:51:18.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.592+0000 7fe23e28a640 1 Processor -- start 2023-08-11T15:51:18.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.592+0000 7fe23e28a640 1 -- start start 2023-08-11T15:51:18.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.593+0000 7fe23e28a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe23815a110 0x7fe2380cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.593+0000 7fe23e28a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe23815a4f0 0x7fe2380cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.594+0000 7fe23e28a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe2380cd770 0x7fe2380ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.594+0000 7fe23e28a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2380ce540 con 0x7fe2380cd770 2023-08-11T15:51:18.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.594+0000 7fe23e28a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2380ce820 con 0x7fe23815a110 2023-08-11T15:51:18.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.594+0000 7fe23e28a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380ceb00 con 0x7fe23815a4f0 2023-08-11T15:51:18.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.594+0000 7fe23da89640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe2380cd770 0x7fe2380ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.594+0000 7fe23da89640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe2380cd770 0x7fe2380ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43448/0 (socket says 172.21.15.27:43448) 2023-08-11T15:51:18.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.594+0000 7fe23da89640 1 -- 172.21.15.27:0/3435512899 learned_addr learned my addr 172.21.15.27:0/3435512899 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:18.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.594+0000 7fe23d288640 1 --2- 172.21.15.27:0/3435512899 >> v2:172.21.15.119:3300/0 conn(0x7fe23815a110 0x7fe2380cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.594+0000 7fe23ca87640 1 --2- 172.21.15.27:0/3435512899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe23815a4f0 0x7fe2380cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.595+0000 7fe23da89640 1 -- 172.21.15.27:0/3435512899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe23815a4f0 msgr2=0x7fe2380cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:18.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.595+0000 7fe23da89640 1 --2- 172.21.15.27:0/3435512899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe23815a4f0 0x7fe2380cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.595+0000 7fe23da89640 1 -- 172.21.15.27:0/3435512899 >> v2:172.21.15.119:3300/0 conn(0x7fe23815a110 msgr2=0x7fe2380cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:18.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.595+0000 7fe23da89640 1 --2- 172.21.15.27:0/3435512899 >> v2:172.21.15.119:3300/0 conn(0x7fe23815a110 0x7fe2380cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.595+0000 7fe23da89640 1 -- 172.21.15.27:0/3435512899 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2380cede0 con 0x7fe2380cd770 2023-08-11T15:51:18.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.596+0000 7fe23da89640 1 --2- 172.21.15.27:0/3435512899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2380cd770 0x7fe2380ccec0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe230009870 tx=0x7fe23002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545026e963161914 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:18.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.596+0000 7fe237fff640 1 -- 172.21.15.27:0/3435512899 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230031640 con 0x7fe2380cd770 2023-08-11T15:51:18.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.596+0000 7fe237fff640 1 -- 172.21.15.27:0/3435512899 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2300317a0 con 0x7fe2380cd770 2023-08-11T15:51:18.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.596+0000 7fe237fff640 1 -- 172.21.15.27:0/3435512899 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230031b30 con 0x7fe2380cd770 2023-08-11T15:51:18.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.597+0000 7fe23e28a640 1 -- 172.21.15.27:0/3435512899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2380cd770 msgr2=0x7fe2380ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.597+0000 7fe23e28a640 1 --2- 172.21.15.27:0/3435512899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2380cd770 0x7fe2380ccec0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe230009870 tx=0x7fe23002b8f0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.597+0000 7fe23e28a640 1 -- 172.21.15.27:0/3435512899 shutdown_connections 2023-08-11T15:51:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.597+0000 7fe23e28a640 1 --2- 172.21.15.27:0/3435512899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2380cd770 0x7fe2380ccec0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.597+0000 7fe23e28a640 1 --2- 172.21.15.27:0/3435512899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe23815a4f0 0x7fe2380cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.597+0000 7fe23e28a640 1 --2- 172.21.15.27:0/3435512899 >> v2:172.21.15.119:3300/0 conn(0x7fe23815a110 0x7fe2380cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.597+0000 7fe23e28a640 1 -- 172.21.15.27:0/3435512899 >> 172.21.15.27:0/3435512899 conn(0x7fe238099390 msgr2=0x7fe2380d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.597+0000 7fe23e28a640 1 -- 172.21.15.27:0/3435512899 shutdown_connections 2023-08-11T15:51:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.597+0000 7fe23e28a640 1 -- 172.21.15.27:0/3435512899 wait complete. 2023-08-11T15:51:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.599+0000 7fe23e28a640 1 Processor -- start 2023-08-11T15:51:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.599+0000 7fe23e28a640 1 -- start start 2023-08-11T15:51:18.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.600+0000 7fe23e28a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2380cd770 0x7fe2380c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.600+0000 7fe23e28a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe23815a110 0x7fe2380b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.600+0000 7fe23e28a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23815a4f0 0x7fe238159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.600+0000 7fe23e28a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380c8730 con 0x7fe23815a110 2023-08-11T15:51:18.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.600+0000 7fe23e28a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380b1960 con 0x7fe23815a4f0 2023-08-11T15:51:18.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.600+0000 7fe23e28a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380b05e0 con 0x7fe2380cd770 2023-08-11T15:51:18.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.600+0000 7fe23ca87640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe23815a110 0x7fe2380b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.601+0000 7fe23ca87640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe23815a110 0x7fe2380b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43450/0 (socket says 172.21.15.27:43450) 2023-08-11T15:51:18.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.601+0000 7fe23ca87640 1 -- 172.21.15.27:0/4003773864 learned_addr learned my addr 172.21.15.27:0/4003773864 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:18.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.601+0000 7fe23da89640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23815a4f0 0x7fe238159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.601+0000 7fe23d288640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2380cd770 0x7fe2380c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.601+0000 7fe23ca87640 1 -- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2380cd770 msgr2=0x7fe2380c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.601+0000 7fe23ca87640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2380cd770 0x7fe2380c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.601+0000 7fe23ca87640 1 -- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23815a4f0 msgr2=0x7fe238159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.601+0000 7fe23ca87640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23815a4f0 0x7fe238159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.601+0000 7fe23ca87640 1 -- 172.21.15.27:0/4003773864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2300094d0 con 0x7fe23815a110 2023-08-11T15:51:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.601+0000 7fe23ca87640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe23815a110 0x7fe2380b1d30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00d5c0 tx=0x7fe22c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.602+0000 7fe235ffb640 1 -- 172.21.15.27:0/4003773864 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0125f0 con 0x7fe23815a110 2023-08-11T15:51:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.602+0000 7fe235ffb640 1 -- 172.21.15.27:0/4003773864 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c012750 con 0x7fe23815a110 2023-08-11T15:51:18.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.602+0000 7fe235ffb640 1 -- 172.21.15.27:0/4003773864 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c012b00 con 0x7fe23815a110 2023-08-11T15:51:18.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.602+0000 7fe23e28a640 1 -- 172.21.15.27:0/4003773864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2380b08c0 con 0x7fe23815a110 2023-08-11T15:51:18.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.602+0000 7fe23e28a640 1 -- 172.21.15.27:0/4003773864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2380b0e80 con 0x7fe23815a110 2023-08-11T15:51:18.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.603+0000 7fe23e28a640 1 -- 172.21.15.27:0/4003773864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23804e420 con 0x7fe23815a110 2023-08-11T15:51:18.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.604+0000 7fe235ffb640 1 -- 172.21.15.27:0/4003773864 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe22c012c60 con 0x7fe23815a110 2023-08-11T15:51:18.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.605+0000 7fe235ffb640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe22406a080 0x7fe22406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:18.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.606+0000 7fe235ffb640 1 -- 172.21.15.27:0/4003773864 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe22c088c30 con 0x7fe23815a110 2023-08-11T15:51:18.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.606+0000 7fe23d288640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe22406a080 0x7fe22406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.607+0000 7fe23d288640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe22406a080 0x7fe22406c540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe2381583f0 tx=0x7fe2280039b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.610+0000 7fe235ffb640 1 -- 172.21.15.27:0/4003773864 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe22c054020 con 0x7fe23815a110 2023-08-11T15:51:18.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.803+0000 7fe23e28a640 1 -- 172.21.15.27:0/4003773864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe2380cfc60 con 0x7fe23815a110 2023-08-11T15:51:18.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.804+0000 7fe235ffb640 1 -- 172.21.15.27:0/4003773864 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe22c057640 con 0x7fe23815a110 2023-08-11T15:51:18.806 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:51:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 -- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe22406a080 msgr2=0x7fe22406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe22406a080 0x7fe22406c540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe2381583f0 tx=0x7fe2280039b0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 -- 172.21.15.27:0/4003773864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe23815a110 msgr2=0x7fe2380b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe23815a110 0x7fe2380b1d30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00d5c0 tx=0x7fe22c00da80 comp rx=0 tx=0).stop 2023-08-11T15:51:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 -- 172.21.15.27:0/4003773864 shutdown_connections 2023-08-11T15:51:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe22406a080 0x7fe22406c540 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23815a4f0 0x7fe238159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe23815a110 0x7fe2380b1d30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 --2- 172.21.15.27:0/4003773864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2380cd770 0x7fe2380c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 -- 172.21.15.27:0/4003773864 >> 172.21.15.27:0/4003773864 conn(0x7fe238099390 msgr2=0x7fe2380ae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 -- 172.21.15.27:0/4003773864 shutdown_connections 2023-08-11T15:51:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:18.806+0000 7fe23e28a640 1 -- 172.21.15.27:0/4003773864 wait complete. 2023-08-11T15:51:18.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:51:19.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.057+0000 7fb125c39640 1 Processor -- start 2023-08-11T15:51:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.057+0000 7fb125c39640 1 -- start start 2023-08-11T15:51:19.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.059+0000 7fb125c39640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12013d310 0x7fb12015a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.059+0000 7fb125c39640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb12015ac50 0x7fb1200c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.059+0000 7fb125c39640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12015b760 con 0x7fb1200c8840 2023-08-11T15:51:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.059+0000 7fb125c39640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1200c8cb0 con 0x7fb12015ac50 2023-08-11T15:51:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.059+0000 7fb125c39640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12006cd40 con 0x7fb12013d310 2023-08-11T15:51:19.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.060+0000 7fb117fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb12015ac50 0x7fb1200c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.060+0000 7fb124c37640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12013d310 0x7fb12015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.060+0000 7fb117fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb12015ac50 0x7fb1200c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56068/0 (socket says 172.21.15.27:56068) 2023-08-11T15:51:19.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.060+0000 7fb117fff640 1 -- 172.21.15.27:0/3553249858 learned_addr learned my addr 172.21.15.27:0/3553249858 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:19.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.061+0000 7fb124c37640 1 -- 172.21.15.27:0/3553249858 >> v2:172.21.15.119:3300/0 conn(0x7fb12015ac50 msgr2=0x7fb1200c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:19.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.061+0000 7fb124c37640 1 --2- 172.21.15.27:0/3553249858 >> v2:172.21.15.119:3300/0 conn(0x7fb12015ac50 0x7fb1200c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.061+0000 7fb124c37640 1 -- 172.21.15.27:0/3553249858 >> v1:172.21.15.119:6789/0 conn(0x7fb1200c8840 legacy=0x7fb12013f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.061+0000 7fb124c37640 1 -- 172.21.15.27:0/3553249858 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12015b1a0 con 0x7fb12013d310 2023-08-11T15:51:19.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.061+0000 7fb117fff640 1 --2- 172.21.15.27:0/3553249858 >> v2:172.21.15.119:3300/0 conn(0x7fb12015ac50 0x7fb1200c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:19.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.062+0000 7fb124c37640 1 --2- 172.21.15.27:0/3553249858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12013d310 0x7fb12015a710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0097a0 tx=0x7fb11c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f50eabc3d826c029 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:19.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.063+0000 7fb1177fe640 1 -- 172.21.15.27:0/3553249858 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c0316f0 con 0x7fb12013d310 2023-08-11T15:51:19.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.063+0000 7fb1177fe640 1 -- 172.21.15.27:0/3553249858 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c031850 con 0x7fb12013d310 2023-08-11T15:51:19.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.063+0000 7fb1177fe640 1 -- 172.21.15.27:0/3553249858 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c031be0 con 0x7fb12013d310 2023-08-11T15:51:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.063+0000 7fb125c39640 1 -- 172.21.15.27:0/3553249858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12013d310 msgr2=0x7fb12015a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.063+0000 7fb125c39640 1 --2- 172.21.15.27:0/3553249858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12013d310 0x7fb12015a710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0097a0 tx=0x7fb11c02b790 comp rx=0 tx=0).stop 2023-08-11T15:51:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.063+0000 7fb125c39640 1 -- 172.21.15.27:0/3553249858 shutdown_connections 2023-08-11T15:51:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.063+0000 7fb125c39640 1 --2- 172.21.15.27:0/3553249858 >> v2:172.21.15.119:3300/0 conn(0x7fb12015ac50 0x7fb1200c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.063+0000 7fb125c39640 1 --2- 172.21.15.27:0/3553249858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12013d310 0x7fb12015a710 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.063+0000 7fb125c39640 1 -- 172.21.15.27:0/3553249858 >> 172.21.15.27:0/3553249858 conn(0x7fb120069800 msgr2=0x7fb120069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.063+0000 7fb125c39640 1 -- 172.21.15.27:0/3553249858 shutdown_connections 2023-08-11T15:51:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.064+0000 7fb125c39640 1 -- 172.21.15.27:0/3553249858 wait complete. 2023-08-11T15:51:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.065+0000 7fb125c39640 1 Processor -- start 2023-08-11T15:51:19.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.065+0000 7fb125c39640 1 -- start start 2023-08-11T15:51:19.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.066+0000 7fb125c39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1200c8840 0x7fb120138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.066+0000 7fb125c39640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb12013d310 0x7fb120142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.066+0000 7fb125c39640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12015ac50 0x7fb1200c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.066+0000 7fb125c39640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1200c4d80 con 0x7fb12013d310 2023-08-11T15:51:19.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.066+0000 7fb125c39640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1200c5340 con 0x7fb1200c8840 2023-08-11T15:51:19.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.066+0000 7fb125c39640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb120138e70 con 0x7fb12015ac50 2023-08-11T15:51:19.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb117fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb12013d310 0x7fb120142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb117fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb12013d310 0x7fb120142550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43456/0 (socket says 172.21.15.27:43456) 2023-08-11T15:51:19.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb117fff640 1 -- 172.21.15.27:0/3230478567 learned_addr learned my addr 172.21.15.27:0/3230478567 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:19.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb124c37640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1200c8840 0x7fb120138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb125438640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12015ac50 0x7fb1200c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb117fff640 1 -- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12015ac50 msgr2=0x7fb1200c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb117fff640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12015ac50 0x7fb1200c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb117fff640 1 -- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1200c8840 msgr2=0x7fb120138930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb117fff640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1200c8840 0x7fb120138930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb117fff640 1 -- 172.21.15.27:0/3230478567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1100094d0 con 0x7fb12013d310 2023-08-11T15:51:19.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.067+0000 7fb124c37640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1200c8840 0x7fb120138930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:19.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.068+0000 7fb117fff640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb12013d310 0x7fb120142550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb11000ea00 tx=0x7fb11000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:19.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.068+0000 7fb1157fa640 1 -- 172.21.15.27:0/3230478567 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11000ca60 con 0x7fb12013d310 2023-08-11T15:51:19.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.068+0000 7fb125c39640 1 -- 172.21.15.27:0/3230478567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c009400 con 0x7fb12013d310 2023-08-11T15:51:19.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.068+0000 7fb1157fa640 1 -- 172.21.15.27:0/3230478567 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11000cbc0 con 0x7fb12013d310 2023-08-11T15:51:19.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.068+0000 7fb1157fa640 1 -- 172.21.15.27:0/3230478567 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110013470 con 0x7fb12013d310 2023-08-11T15:51:19.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.068+0000 7fb125c39640 1 -- 172.21.15.27:0/3230478567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1200c4980 con 0x7fb12013d310 2023-08-11T15:51:19.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.069+0000 7fb125c39640 1 -- 172.21.15.27:0/3230478567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1200c9230 con 0x7fb12013d310 2023-08-11T15:51:19.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.070+0000 7fb1157fa640 1 -- 172.21.15.27:0/3230478567 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb1100135d0 con 0x7fb12013d310 2023-08-11T15:51:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.071+0000 7fb1157fa640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb10c06a050 0x7fb10c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.071+0000 7fb1157fa640 1 -- 172.21.15.27:0/3230478567 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb110088cb0 con 0x7fb12013d310 2023-08-11T15:51:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.072+0000 7fb124c37640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb10c06a050 0x7fb10c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.073+0000 7fb124c37640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb10c06a050 0x7fb10c06c510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb11c036000 tx=0x7fb11c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.075+0000 7fb1157fa640 1 -- 172.21.15.27:0/3230478567 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fb110054020 con 0x7fb12013d310 2023-08-11T15:51:19.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.289+0000 7fb125c39640 1 -- 172.21.15.27:0/3230478567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb120066e80 con 0x7fb12013d310 2023-08-11T15:51:19.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.290+0000 7fb1157fa640 1 -- 172.21.15.27:0/3230478567 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb110057640 con 0x7fb12013d310 2023-08-11T15:51:19.293 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:51:19.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.293+0000 7fb125c39640 1 -- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb10c06a050 msgr2=0x7fb10c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.293+0000 7fb125c39640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb10c06a050 0x7fb10c06c510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb11c036000 tx=0x7fb11c008030 comp rx=0 tx=0).stop 2023-08-11T15:51:19.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.293+0000 7fb125c39640 1 -- 172.21.15.27:0/3230478567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb12013d310 msgr2=0x7fb120142550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.293+0000 7fb125c39640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb12013d310 0x7fb120142550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb11000ea00 tx=0x7fb11000eec0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.293+0000 7fb125c39640 1 -- 172.21.15.27:0/3230478567 shutdown_connections 2023-08-11T15:51:19.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.293+0000 7fb125c39640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb10c06a050 0x7fb10c06c510 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.293+0000 7fb125c39640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb12015ac50 0x7fb1200c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.293+0000 7fb125c39640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb12013d310 0x7fb120142550 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.293+0000 7fb125c39640 1 --2- 172.21.15.27:0/3230478567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1200c8840 0x7fb120138930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.293+0000 7fb125c39640 1 -- 172.21.15.27:0/3230478567 >> 172.21.15.27:0/3230478567 conn(0x7fb120069800 msgr2=0x7fb120069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:19.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.294+0000 7fb125c39640 1 -- 172.21.15.27:0/3230478567 shutdown_connections 2023-08-11T15:51:19.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.294+0000 7fb125c39640 1 -- 172.21.15.27:0/3230478567 wait complete. 2023-08-11T15:51:19.316 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:51:19.317 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:51:19.372 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.603+0000 7ffa90bab640 1 Processor -- start 2023-08-11T15:51:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.604+0000 7ffa90bab640 1 -- start start 2023-08-11T15:51:19.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.605+0000 7ffa90bab640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffa8c156e50 0x7ffa8c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.605+0000 7ffa90bab640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c0cb0c0 0x7ffa8c0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.606+0000 7ffa90bab640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8c0cbfb0 con 0x7ffa8c0cbb70 2023-08-11T15:51:19.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.606+0000 7ffa90bab640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8c0ca460 con 0x7ffa8c156e50 2023-08-11T15:51:19.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.606+0000 7ffa90bab640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c070470 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.606+0000 7ffa83fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ffa8c0cbb70 0x7ffa8c0ca9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48940/0 (socket says 172.21.15.27:48940) 2023-08-11T15:51:19.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.606+0000 7ffa83fff640 1 -- 172.21.15.27:0/3760391394 learned_addr learned my addr 172.21.15.27:0/3760391394 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:19.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.606+0000 7ffa8a59c640 1 --2- 172.21.15.27:0/3760391394 >> v2:172.21.15.119:3300/0 conn(0x7ffa8c156e50 0x7ffa8c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.606+0000 7ffa8ad9d640 1 --2- 172.21.15.27:0/3760391394 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c0cb0c0 0x7ffa8c0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.607+0000 7ffa89d9b640 1 -- 172.21.15.27:0/3760391394 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521106200 0 0) 0x7ffa8c0cbfb0 con 0x7ffa8c0cbb70 2023-08-11T15:51:19.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.607+0000 7ffa89d9b640 1 -- 172.21.15.27:0/3760391394 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa78005730 con 0x7ffa8c0cbb70 2023-08-11T15:51:19.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.607+0000 7ffa8a59c640 1 -- 172.21.15.27:0/3760391394 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c0cb0c0 msgr2=0x7ffa8c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:19.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.607+0000 7ffa8a59c640 1 --2- 172.21.15.27:0/3760391394 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c0cb0c0 0x7ffa8c0708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.607+0000 7ffa8a59c640 1 -- 172.21.15.27:0/3760391394 >> v1:172.21.15.27:6789/0 conn(0x7ffa8c0cbb70 legacy=0x7ffa8c0ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.607+0000 7ffa8a59c640 1 -- 172.21.15.27:0/3760391394 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c0cc270 con 0x7ffa8c156e50 2023-08-11T15:51:19.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.608+0000 7ffa8a59c640 1 --2- 172.21.15.27:0/3760391394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa8c156e50 0x7ffa8c15a180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009790 tx=0x7ffa7c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ff83b1c841e6af server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:19.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.609+0000 7ffa89d9b640 1 -- 172.21.15.27:0/3760391394 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c02d040 con 0x7ffa8c156e50 2023-08-11T15:51:19.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.609+0000 7ffa89d9b640 1 -- 172.21.15.27:0/3760391394 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c031810 con 0x7ffa8c156e50 2023-08-11T15:51:19.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.609+0000 7ffa89d9b640 1 -- 172.21.15.27:0/3760391394 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c031ba0 con 0x7ffa8c156e50 2023-08-11T15:51:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.609+0000 7ffa90bab640 1 -- 172.21.15.27:0/3760391394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa8c156e50 msgr2=0x7ffa8c15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.609+0000 7ffa90bab640 1 --2- 172.21.15.27:0/3760391394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa8c156e50 0x7ffa8c15a180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009790 tx=0x7ffa7c02b8b0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.609+0000 7ffa90bab640 1 -- 172.21.15.27:0/3760391394 shutdown_connections 2023-08-11T15:51:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.609+0000 7ffa90bab640 1 --2- 172.21.15.27:0/3760391394 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c0cb0c0 0x7ffa8c0708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.609+0000 7ffa90bab640 1 --2- 172.21.15.27:0/3760391394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa8c156e50 0x7ffa8c15a180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.609+0000 7ffa90bab640 1 -- 172.21.15.27:0/3760391394 >> 172.21.15.27:0/3760391394 conn(0x7ffa8c0afb70 msgr2=0x7ffa8c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.609+0000 7ffa90bab640 1 -- 172.21.15.27:0/3760391394 shutdown_connections 2023-08-11T15:51:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.610+0000 7ffa90bab640 1 -- 172.21.15.27:0/3760391394 wait complete. 2023-08-11T15:51:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.611+0000 7ffa90bab640 1 Processor -- start 2023-08-11T15:51:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.612+0000 7ffa90bab640 1 -- start start 2023-08-11T15:51:19.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa90bab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa8c0cb0c0 0x7ffa8c154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa90bab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa8c0cbb70 0x7ffa8c0aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa90bab640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c156e50 0x7ffa8c1532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa90bab640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c152770 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa90bab640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0b2870 con 0x7ffa8c0cbb70 2023-08-11T15:51:19.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa90bab640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0b2b50 con 0x7ffa8c156e50 2023-08-11T15:51:19.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa8a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa8c0cb0c0 0x7ffa8c154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa8a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa8c0cb0c0 0x7ffa8c154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46126/0 (socket says 172.21.15.27:46126) 2023-08-11T15:51:19.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa8a59c640 1 -- 172.21.15.27:0/1665451473 learned_addr learned my addr 172.21.15.27:0/1665451473 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:19.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa83fff640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa8c0cbb70 0x7ffa8c0aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa8ad9d640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c156e50 0x7ffa8c1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa8a59c640 1 -- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c156e50 msgr2=0x7ffa8c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa8a59c640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c156e50 0x7ffa8c1532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa8a59c640 1 -- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa8c0cbb70 msgr2=0x7ffa8c0aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa8a59c640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa8c0cbb70 0x7ffa8c0aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa8a59c640 1 -- 172.21.15.27:0/1665451473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c0093a0 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa83fff640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa8c0cbb70 0x7ffa8c0aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:19.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.613+0000 7ffa8ad9d640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c156e50 0x7ffa8c1532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:19.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.614+0000 7ffa8a59c640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa8c0cb0c0 0x7ffa8c154c20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0098b0 tx=0x7ffa7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:19.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.614+0000 7ffa82ffd640 1 -- 172.21.15.27:0/1665451473 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c03ad60 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.614+0000 7ffa82ffd640 1 -- 172.21.15.27:0/1665451473 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c005990 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.614+0000 7ffa82ffd640 1 -- 172.21.15.27:0/1665451473 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c005b40 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.614+0000 7ffa90bab640 1 -- 172.21.15.27:0/1665451473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8c0b2e30 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.614+0000 7ffa90bab640 1 -- 172.21.15.27:0/1665451473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c0af2c0 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.616+0000 7ffa90bab640 1 -- 172.21.15.27:0/1665451473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c0af4b0 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.616+0000 7ffa82ffd640 1 -- 172.21.15.27:0/1665451473 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffa7c04d440 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.617+0000 7ffa82ffd640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa6c06a050 0x7ffa6c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:19.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.618+0000 7ffa82ffd640 1 -- 172.21.15.27:0/1665451473 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffa7c0aab70 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.618+0000 7ffa83fff640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa6c06a050 0x7ffa6c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:19.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.619+0000 7ffa83fff640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa6c06a050 0x7ffa6c06c510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0b04d0 tx=0x7ffa74009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:19.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.622+0000 7ffa82ffd640 1 -- 172.21.15.27:0/1665451473 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ffa7c075f60 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.820+0000 7ffa90bab640 1 -- 172.21.15.27:0/1665451473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa8c073990 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.823+0000 7ffa82ffd640 1 -- 172.21.15.27:0/1665451473 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7ffa7c079580 con 0x7ffa8c0cb0c0 2023-08-11T15:51:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:19.827 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.826+0000 7ffa90bab640 1 -- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa6c06a050 msgr2=0x7ffa6c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.826+0000 7ffa90bab640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa6c06a050 0x7ffa6c06c510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0b04d0 tx=0x7ffa74009040 comp rx=0 tx=0).stop 2023-08-11T15:51:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.826+0000 7ffa90bab640 1 -- 172.21.15.27:0/1665451473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa8c0cb0c0 msgr2=0x7ffa8c154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.826+0000 7ffa90bab640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa8c0cb0c0 0x7ffa8c154c20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0098b0 tx=0x7ffa7c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.826+0000 7ffa90bab640 1 -- 172.21.15.27:0/1665451473 shutdown_connections 2023-08-11T15:51:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.826+0000 7ffa90bab640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffa6c06a050 0x7ffa6c06c510 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.826+0000 7ffa90bab640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa8c156e50 0x7ffa8c1532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.827+0000 7ffa90bab640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa8c0cbb70 0x7ffa8c0aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.827+0000 7ffa90bab640 1 --2- 172.21.15.27:0/1665451473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa8c0cb0c0 0x7ffa8c154c20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.827+0000 7ffa90bab640 1 -- 172.21.15.27:0/1665451473 >> 172.21.15.27:0/1665451473 conn(0x7ffa8c0afb70 msgr2=0x7ffa8c15a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.827+0000 7ffa90bab640 1 -- 172.21.15.27:0/1665451473 shutdown_connections 2023-08-11T15:51:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:19.827+0000 7ffa90bab640 1 -- 172.21.15.27:0/1665451473 wait complete. 2023-08-11T15:51:19.848 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:51:19.849 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:51:20.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.087+0000 7ff789f37640 1 Processor -- start 2023-08-11T15:51:20.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.087+0000 7ff789f37640 1 -- start start 2023-08-11T15:51:20.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.089+0000 7ff789f37640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff78415c2a0 0x7ff7840b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.089+0000 7ff789f37640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff7840ceb70 0x7ff7840cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.089+0000 7ff789f37640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7840cd890 con 0x7ff7840b45d0 2023-08-11T15:51:20.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.089+0000 7ff789f37640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7840b4140 con 0x7ff78415c2a0 2023-08-11T15:51:20.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.089+0000 7ff789f37640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7840cf4e0 con 0x7ff7840ceb70 2023-08-11T15:51:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.089+0000 7ff788f35640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff78415c2a0 0x7ff7840b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.089+0000 7ff788f35640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff78415c2a0 0x7ff7840b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46134/0 (socket says 172.21.15.27:46134) 2023-08-11T15:51:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.089+0000 7ff788f35640 1 -- 172.21.15.27:0/401237420 learned_addr learned my addr 172.21.15.27:0/401237420 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.090+0000 7ff789736640 1 --2- 172.21.15.27:0/401237420 >> v2:172.21.15.119:3300/0 conn(0x7ff7840ceb70 0x7ff7840cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.090+0000 7ff788f35640 1 -- 172.21.15.27:0/401237420 >> v2:172.21.15.119:3300/0 conn(0x7ff7840ceb70 msgr2=0x7ff7840cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:20.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.090+0000 7ff788f35640 1 --2- 172.21.15.27:0/401237420 >> v2:172.21.15.119:3300/0 conn(0x7ff7840ceb70 0x7ff7840cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.090+0000 7ff788f35640 1 -- 172.21.15.27:0/401237420 >> v1:172.21.15.119:6789/0 conn(0x7ff7840b45d0 legacy=0x7ff7840ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:20.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.090+0000 7ff788f35640 1 -- 172.21.15.27:0/401237420 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7840cd2d0 con 0x7ff78415c2a0 2023-08-11T15:51:20.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.092+0000 7ff788f35640 1 --2- 172.21.15.27:0/401237420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 0x7ff7840b1580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff7840b12e0 tx=0x7ff78002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3589cb0cfcef562 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:20.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.092+0000 7ff77b7fe640 1 -- 172.21.15.27:0/401237420 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78002ca80 con 0x7ff78415c2a0 2023-08-11T15:51:20.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.092+0000 7ff77b7fe640 1 -- 172.21.15.27:0/401237420 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78002cbe0 con 0x7ff78415c2a0 2023-08-11T15:51:20.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.092+0000 7ff77b7fe640 1 -- 172.21.15.27:0/401237420 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7800343f0 con 0x7ff78415c2a0 2023-08-11T15:51:20.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.093+0000 7ff789f37640 1 -- 172.21.15.27:0/401237420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 msgr2=0x7ff7840b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.093+0000 7ff789f37640 1 --2- 172.21.15.27:0/401237420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 0x7ff7840b1580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff7840b12e0 tx=0x7ff78002bee0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.093+0000 7ff789f37640 1 -- 172.21.15.27:0/401237420 shutdown_connections 2023-08-11T15:51:20.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.093+0000 7ff789f37640 1 --2- 172.21.15.27:0/401237420 >> v2:172.21.15.119:3300/0 conn(0x7ff7840ceb70 0x7ff7840cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.093+0000 7ff789f37640 1 --2- 172.21.15.27:0/401237420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 0x7ff7840b1580 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.093+0000 7ff789f37640 1 -- 172.21.15.27:0/401237420 >> 172.21.15.27:0/401237420 conn(0x7ff78406a250 msgr2=0x7ff78406a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:20.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.093+0000 7ff789f37640 1 -- 172.21.15.27:0/401237420 shutdown_connections 2023-08-11T15:51:20.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.093+0000 7ff789f37640 1 -- 172.21.15.27:0/401237420 wait complete. 2023-08-11T15:51:20.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.095+0000 7ff789f37640 1 Processor -- start 2023-08-11T15:51:20.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.095+0000 7ff789f37640 1 -- start start 2023-08-11T15:51:20.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.096+0000 7ff789f37640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7840b45d0 0x7ff7841f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.096+0000 7ff789f37640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7840ceb70 0x7ff7840cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.096+0000 7ff789f37640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 0x7ff7840cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.096+0000 7ff789f37640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840c9320 con 0x7ff78415c2a0 2023-08-11T15:51:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.096+0000 7ff789f37640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840c9590 con 0x7ff7840ceb70 2023-08-11T15:51:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.096+0000 7ff789f37640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840c9870 con 0x7ff7840b45d0 2023-08-11T15:51:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.096+0000 7ff789736640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 0x7ff7840cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.096+0000 7ff789736640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 0x7ff7840cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46136/0 (socket says 172.21.15.27:46136) 2023-08-11T15:51:20.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.096+0000 7ff789736640 1 -- 172.21.15.27:0/558772137 learned_addr learned my addr 172.21.15.27:0/558772137 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.097+0000 7ff77bfff640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7840ceb70 0x7ff7840cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.097+0000 7ff788f35640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7840b45d0 0x7ff7841f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.097+0000 7ff789736640 1 -- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7840b45d0 msgr2=0x7ff7841f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.097+0000 7ff789736640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7840b45d0 0x7ff7841f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.097+0000 7ff789736640 1 -- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7840ceb70 msgr2=0x7ff7840cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.097+0000 7ff789736640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7840ceb70 0x7ff7840cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.097+0000 7ff789736640 1 -- 172.21.15.27:0/558772137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff780009b30 con 0x7ff78415c2a0 2023-08-11T15:51:20.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.097+0000 7ff77bfff640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7840ceb70 0x7ff7840cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:20.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.097+0000 7ff789736640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 0x7ff7840cbc50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00d800 tx=0x7ff77c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:20.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.098+0000 7ff7797fa640 1 -- 172.21.15.27:0/558772137 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00b6c0 con 0x7ff78415c2a0 2023-08-11T15:51:20.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.098+0000 7ff7797fa640 1 -- 172.21.15.27:0/558772137 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c00b840 con 0x7ff78415c2a0 2023-08-11T15:51:20.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.098+0000 7ff789f37640 1 -- 172.21.15.27:0/558772137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7840c9b50 con 0x7ff78415c2a0 2023-08-11T15:51:20.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.098+0000 7ff7797fa640 1 -- 172.21.15.27:0/558772137 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00bbf0 con 0x7ff78415c2a0 2023-08-11T15:51:20.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.098+0000 7ff789f37640 1 -- 172.21.15.27:0/558772137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7841f1d70 con 0x7ff78415c2a0 2023-08-11T15:51:20.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.099+0000 7ff789f37640 1 -- 172.21.15.27:0/558772137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7840cdae0 con 0x7ff78415c2a0 2023-08-11T15:51:20.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.100+0000 7ff7797fa640 1 -- 172.21.15.27:0/558772137 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff77c00bd50 con 0x7ff78415c2a0 2023-08-11T15:51:20.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.101+0000 7ff7797fa640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff77006a050 0x7ff77006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.101+0000 7ff7797fa640 1 -- 172.21.15.27:0/558772137 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff77c088e60 con 0x7ff78415c2a0 2023-08-11T15:51:20.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.102+0000 7ff788f35640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff77006a050 0x7ff77006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.103+0000 7ff788f35640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff77006a050 0x7ff77006c510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff780035000 tx=0x7ff780030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:20.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.106+0000 7ff7797fa640 1 -- 172.21.15.27:0/558772137 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ff77c0541c0 con 0x7ff78415c2a0 2023-08-11T15:51:20.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.300+0000 7ff789f37640 1 -- 172.21.15.27:0/558772137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7840ca010 con 0x7ff78415c2a0 2023-08-11T15:51:20.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.301+0000 7ff7797fa640 1 -- 172.21.15.27:0/558772137 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff77c0577e0 con 0x7ff78415c2a0 2023-08-11T15:51:20.304 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:51:20.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.304+0000 7ff789f37640 1 -- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff77006a050 msgr2=0x7ff77006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.304+0000 7ff789f37640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff77006a050 0x7ff77006c510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff780035000 tx=0x7ff780030000 comp rx=0 tx=0).stop 2023-08-11T15:51:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.304+0000 7ff789f37640 1 -- 172.21.15.27:0/558772137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 msgr2=0x7ff7840cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.304+0000 7ff789f37640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 0x7ff7840cbc50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00d800 tx=0x7ff77c00dcc0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.304+0000 7ff789f37640 1 -- 172.21.15.27:0/558772137 shutdown_connections 2023-08-11T15:51:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.304+0000 7ff789f37640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff77006a050 0x7ff77006c510 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.304+0000 7ff789f37640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff78415c2a0 0x7ff7840cbc50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.304+0000 7ff789f37640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7840ceb70 0x7ff7840cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.304+0000 7ff789f37640 1 --2- 172.21.15.27:0/558772137 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7840b45d0 0x7ff7841f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.304+0000 7ff789f37640 1 -- 172.21.15.27:0/558772137 >> 172.21.15.27:0/558772137 conn(0x7ff78406a250 msgr2=0x7ff7840b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:20.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.305+0000 7ff789f37640 1 -- 172.21.15.27:0/558772137 shutdown_connections 2023-08-11T15:51:20.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.305+0000 7ff789f37640 1 -- 172.21.15.27:0/558772137 wait complete. 2023-08-11T15:51:20.327 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:51:20.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.563+0000 7f471001f640 1 Processor -- start 2023-08-11T15:51:20.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.563+0000 7f471001f640 1 -- start start 2023-08-11T15:51:20.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.565+0000 7f471001f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f470806b670 0x7f47080d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.565+0000 7f471001f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47080d2850 0x7f47080af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.565+0000 7f471001f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470806bc10 con 0x7f470815c760 2023-08-11T15:51:20.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.565+0000 7f471001f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470806c790 con 0x7f470806b670 2023-08-11T15:51:20.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.565+0000 7f471001f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47080d13f0 con 0x7f47080d2850 2023-08-11T15:51:20.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.565+0000 7f470ddbb640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f470815c760 0x7f47080d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:55738/0 (socket says 172.21.15.27:55738) 2023-08-11T15:51:20.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.565+0000 7f470ddbb640 1 -- 172.21.15.27:0/3776571246 learned_addr learned my addr 172.21.15.27:0/3776571246 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:20.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.565+0000 7f470d5ba640 1 --2- 172.21.15.27:0/3776571246 >> v2:172.21.15.119:3300/0 conn(0x7f470806b670 0x7f47080d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.566+0000 7f470e5bc640 1 --2- 172.21.15.27:0/3776571246 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47080d2850 0x7f47080af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.566+0000 7f470d5ba640 1 -- 172.21.15.27:0/3776571246 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47080d2850 msgr2=0x7f47080af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:20.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.566+0000 7f470d5ba640 1 --2- 172.21.15.27:0/3776571246 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47080d2850 0x7f47080af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.566+0000 7f470d5ba640 1 -- 172.21.15.27:0/3776571246 >> v1:172.21.15.119:6789/0 conn(0x7f470815c760 legacy=0x7f47080d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.566+0000 7f470d5ba640 1 -- 172.21.15.27:0/3776571246 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470806bef0 con 0x7f470806b670 2023-08-11T15:51:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.567+0000 7f470d5ba640 1 --2- 172.21.15.27:0/3776571246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470806b670 0x7f47080d2310 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f46f80097a0 tx=0x7f46f802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ef2d7983687ca2b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:20.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.568+0000 7f470cdb9640 1 -- 172.21.15.27:0/3776571246 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f802d040 con 0x7f470806b670 2023-08-11T15:51:20.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.568+0000 7f470cdb9640 1 -- 172.21.15.27:0/3776571246 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8031810 con 0x7f470806b670 2023-08-11T15:51:20.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.568+0000 7f470cdb9640 1 -- 172.21.15.27:0/3776571246 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8031ba0 con 0x7f470806b670 2023-08-11T15:51:20.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.569+0000 7f471001f640 1 -- 172.21.15.27:0/3776571246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470806b670 msgr2=0x7f47080d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.569+0000 7f471001f640 1 --2- 172.21.15.27:0/3776571246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470806b670 0x7f47080d2310 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f46f80097a0 tx=0x7f46f802b8b0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.569+0000 7f471001f640 1 -- 172.21.15.27:0/3776571246 shutdown_connections 2023-08-11T15:51:20.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.569+0000 7f471001f640 1 --2- 172.21.15.27:0/3776571246 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47080d2850 0x7f47080af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.569+0000 7f471001f640 1 --2- 172.21.15.27:0/3776571246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470806b670 0x7f47080d2310 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.569+0000 7f471001f640 1 -- 172.21.15.27:0/3776571246 >> 172.21.15.27:0/3776571246 conn(0x7f4708089ff0 msgr2=0x7f470809b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:20.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.569+0000 7f471001f640 1 -- 172.21.15.27:0/3776571246 shutdown_connections 2023-08-11T15:51:20.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.569+0000 7f471001f640 1 -- 172.21.15.27:0/3776571246 wait complete. 2023-08-11T15:51:20.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.571+0000 7f471001f640 1 Processor -- start 2023-08-11T15:51:20.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.571+0000 7f471001f640 1 -- start start 2023-08-11T15:51:20.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.572+0000 7f471001f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f470806b670 0x7f47081581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.572+0000 7f471001f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47080d2850 0x7f47080cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.572+0000 7f471001f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470815c760 0x7f47080cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.572+0000 7f471001f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47080c62d0 con 0x7f470806b670 2023-08-11T15:51:20.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.572+0000 7f471001f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47080c65b0 con 0x7f470815c760 2023-08-11T15:51:20.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.572+0000 7f471001f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47080c6890 con 0x7f47080d2850 2023-08-11T15:51:20.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.572+0000 7f470ddbb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f470806b670 0x7f47081581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.572+0000 7f470ddbb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f470806b670 0x7f47081581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46150/0 (socket says 172.21.15.27:46150) 2023-08-11T15:51:20.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.572+0000 7f470ddbb640 1 -- 172.21.15.27:0/3323990263 learned_addr learned my addr 172.21.15.27:0/3323990263 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:20.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.572+0000 7f470e5bc640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470815c760 0x7f47080cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.573+0000 7f470d5ba640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47080d2850 0x7f47080cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.573+0000 7f470ddbb640 1 -- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47080d2850 msgr2=0x7f47080cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.573+0000 7f470ddbb640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47080d2850 0x7f47080cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.573+0000 7f470ddbb640 1 -- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470815c760 msgr2=0x7f47080cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.573+0000 7f470ddbb640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470815c760 0x7f47080cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.573+0000 7f470ddbb640 1 -- 172.21.15.27:0/3323990263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f8009400 con 0x7f470806b670 2023-08-11T15:51:20.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.573+0000 7f470ddbb640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f470806b670 0x7f47081581b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f470400cbc0 tx=0x7f470400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:20.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.573+0000 7f46fe7fc640 1 -- 172.21.15.27:0/3323990263 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470400aca0 con 0x7f470806b670 2023-08-11T15:51:20.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.573+0000 7f46fe7fc640 1 -- 172.21.15.27:0/3323990263 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470400ae00 con 0x7f470806b670 2023-08-11T15:51:20.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.573+0000 7f46fe7fc640 1 -- 172.21.15.27:0/3323990263 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704012600 con 0x7f470806b670 2023-08-11T15:51:20.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.574+0000 7f471001f640 1 -- 172.21.15.27:0/3323990263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47080c6b70 con 0x7f470806b670 2023-08-11T15:51:20.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.574+0000 7f471001f640 1 -- 172.21.15.27:0/3323990263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47081586f0 con 0x7f470806b670 2023-08-11T15:51:20.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.575+0000 7f471001f640 1 -- 172.21.15.27:0/3323990263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470804e420 con 0x7f470806b670 2023-08-11T15:51:20.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.576+0000 7f46fe7fc640 1 -- 172.21.15.27:0/3323990263 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4704007070 con 0x7f470806b670 2023-08-11T15:51:20.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.577+0000 7f46fe7fc640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46ec06a050 0x7f46ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.577+0000 7f46fe7fc640 1 -- 172.21.15.27:0/3323990263 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4704088700 con 0x7f470806b670 2023-08-11T15:51:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.577+0000 7f470d5ba640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46ec06a050 0x7f46ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:20.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.579+0000 7f470d5ba640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46ec06a050 0x7f46ec06c510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f46f8036000 tx=0x7f46f803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.581+0000 7f46fe7fc640 1 -- 172.21.15.27:0/3323990263 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f4704053ba0 con 0x7f470806b670 2023-08-11T15:51:20.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.775+0000 7f471001f640 1 -- 172.21.15.27:0/3323990263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f47080d34b0 con 0x7f470806b670 2023-08-11T15:51:20.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.775+0000 7f46fe7fc640 1 -- 172.21.15.27:0/3323990263 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f47040571c0 con 0x7f470806b670 2023-08-11T15:51:20.778 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:51:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.778+0000 7f471001f640 1 -- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46ec06a050 msgr2=0x7f46ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.778+0000 7f471001f640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46ec06a050 0x7f46ec06c510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f46f8036000 tx=0x7f46f803c040 comp rx=0 tx=0).stop 2023-08-11T15:51:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.778+0000 7f471001f640 1 -- 172.21.15.27:0/3323990263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f470806b670 msgr2=0x7f47081581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.778+0000 7f471001f640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f470806b670 0x7f47081581b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f470400cbc0 tx=0x7f470400a060 comp rx=0 tx=0).stop 2023-08-11T15:51:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.778+0000 7f471001f640 1 -- 172.21.15.27:0/3323990263 shutdown_connections 2023-08-11T15:51:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.778+0000 7f471001f640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46ec06a050 0x7f46ec06c510 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.778+0000 7f471001f640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47080d2850 0x7f47080cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.778+0000 7f471001f640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470815c760 0x7f47080cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.778+0000 7f471001f640 1 --2- 172.21.15.27:0/3323990263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f470806b670 0x7f47081581b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.778+0000 7f471001f640 1 -- 172.21.15.27:0/3323990263 >> 172.21.15.27:0/3323990263 conn(0x7f4708089ff0 msgr2=0x7f47080d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.779+0000 7f471001f640 1 -- 172.21.15.27:0/3323990263 shutdown_connections 2023-08-11T15:51:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:20.779+0000 7f471001f640 1 -- 172.21.15.27:0/3323990263 wait complete. 2023-08-11T15:51:20.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:51:21.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.042+0000 7f2ce7e83640 1 Processor -- start 2023-08-11T15:51:21.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.042+0000 7f2ce7e83640 1 -- start start 2023-08-11T15:51:21.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.044+0000 7f2ce7e83640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2ce00cb110 0x7f2ce00cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.044+0000 7f2ce7e83640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce01600d0 con 0x7f2ce0159f80 2023-08-11T15:51:21.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.044+0000 7f2ce7e83640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce0160320 con 0x7f2ce00cc170 2023-08-11T15:51:21.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.044+0000 7f2ce7e83640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce0160600 con 0x7f2ce00cb110 2023-08-11T15:51:21.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.045+0000 7f2ce5c1f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2ce0159f80 0x7f2ce00caa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48948/0 (socket says 172.21.15.27:48948) 2023-08-11T15:51:21.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.045+0000 7f2ce5c1f640 1 -- 172.21.15.27:0/3345726609 learned_addr learned my addr 172.21.15.27:0/3345726609 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:21.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.045+0000 7f2ce541e640 1 --2- 172.21.15.27:0/3345726609 >> v2:172.21.15.119:3300/0 conn(0x7f2ce00cb110 0x7f2ce00cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.045+0000 7f2ce4c1d640 1 -- 172.21.15.27:0/3345726609 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532958269 0 0) 0x7f2ce01600d0 con 0x7f2ce0159f80 2023-08-11T15:51:21.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.045+0000 7f2ce4c1d640 1 -- 172.21.15.27:0/3345726609 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ccc003520 con 0x7f2ce0159f80 2023-08-11T15:51:21.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.046+0000 7f2ce4c1d640 1 -- 172.21.15.27:0/3345726609 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2786854855 0 0) 0x7f2ccc003520 con 0x7f2ce0159f80 2023-08-11T15:51:21.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.046+0000 7f2ce541e640 1 -- 172.21.15.27:0/3345726609 >> v1:172.21.15.119:6789/0 conn(0x7f2ce00cc170 legacy=0x7f2ce015cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.046+0000 7f2ce541e640 1 -- 172.21.15.27:0/3345726609 >> v1:172.21.15.27:6789/0 conn(0x7f2ce0159f80 legacy=0x7f2ce00caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.046+0000 7f2ce541e640 1 -- 172.21.15.27:0/3345726609 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce01608e0 con 0x7f2ce00cb110 2023-08-11T15:51:21.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.047+0000 7f2ce541e640 1 --2- 172.21.15.27:0/3345726609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce00cb110 0x7f2ce00cbc30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00097a0 tx=0x7f2cd002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccda95dd91e30d25 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:21.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.048+0000 7f2ce4c1d640 1 -- 172.21.15.27:0/3345726609 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd002f050 con 0x7f2ce00cb110 2023-08-11T15:51:21.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.048+0000 7f2ce4c1d640 1 -- 172.21.15.27:0/3345726609 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd0031500 con 0x7f2ce00cb110 2023-08-11T15:51:21.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.048+0000 7f2ce4c1d640 1 -- 172.21.15.27:0/3345726609 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc001670 con 0x7f2ce00cb110 2023-08-11T15:51:21.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.049+0000 7f2ce7e83640 1 -- 172.21.15.27:0/3345726609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce00cb110 msgr2=0x7f2ce00cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.049+0000 7f2ce7e83640 1 --2- 172.21.15.27:0/3345726609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce00cb110 0x7f2ce00cbc30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00097a0 tx=0x7f2cd002b820 comp rx=0 tx=0).stop 2023-08-11T15:51:21.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.049+0000 7f2ce7e83640 1 -- 172.21.15.27:0/3345726609 shutdown_connections 2023-08-11T15:51:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.049+0000 7f2ce7e83640 1 --2- 172.21.15.27:0/3345726609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce00cb110 0x7f2ce00cbc30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.049+0000 7f2ce7e83640 1 -- 172.21.15.27:0/3345726609 >> 172.21.15.27:0/3345726609 conn(0x7f2ce00afbe0 msgr2=0x7f2ce00affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.049+0000 7f2ce7e83640 1 -- 172.21.15.27:0/3345726609 shutdown_connections 2023-08-11T15:51:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.049+0000 7f2ce7e83640 1 -- 172.21.15.27:0/3345726609 wait complete. 2023-08-11T15:51:21.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.051+0000 7f2ce7e83640 1 Processor -- start 2023-08-11T15:51:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.051+0000 7f2ce7e83640 1 -- start start 2023-08-11T15:51:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.052+0000 7f2ce7e83640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce00cb110 0x7f2ce0155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.052+0000 7f2ce7e83640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce00cc170 0x7f2ce00b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.052+0000 7f2ce7e83640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ce0159f80 0x7f2ce00b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.052+0000 7f2ce7e83640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce00c9b20 con 0x7f2ce00cc170 2023-08-11T15:51:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.052+0000 7f2ce7e83640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce00c9f40 con 0x7f2ce00cb110 2023-08-11T15:51:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.052+0000 7f2ce7e83640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce00c7a20 con 0x7f2ce0159f80 2023-08-11T15:51:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce541e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce00cc170 0x7f2ce00b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce541e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce00cc170 0x7f2ce00b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46158/0 (socket says 172.21.15.27:46158) 2023-08-11T15:51:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce541e640 1 -- 172.21.15.27:0/2320793531 learned_addr learned my addr 172.21.15.27:0/2320793531 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce5c1f640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce00cb110 0x7f2ce0155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce6420640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ce0159f80 0x7f2ce00b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce541e640 1 -- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ce0159f80 msgr2=0x7f2ce00b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce541e640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ce0159f80 0x7f2ce00b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce541e640 1 -- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce00cb110 msgr2=0x7f2ce0155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce541e640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce00cb110 0x7f2ce0155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce541e640 1 -- 172.21.15.27:0/2320793531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0009400 con 0x7f2ce00cc170 2023-08-11T15:51:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.053+0000 7f2ce5c1f640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce00cb110 0x7f2ce0155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.054+0000 7f2ce541e640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce00cc170 0x7f2ce00b32e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00098c0 tx=0x7f2cd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.054+0000 7f2cd67fc640 1 -- 172.21.15.27:0/2320793531 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd003d940 con 0x7f2ce00cc170 2023-08-11T15:51:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.054+0000 7f2cd67fc640 1 -- 172.21.15.27:0/2320793531 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd003daa0 con 0x7f2ce00cc170 2023-08-11T15:51:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.054+0000 7f2cd67fc640 1 -- 172.21.15.27:0/2320793531 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd003de10 con 0x7f2ce00cc170 2023-08-11T15:51:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.054+0000 7f2ce7e83640 1 -- 172.21.15.27:0/2320793531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce01f2af0 con 0x7f2ce00cc170 2023-08-11T15:51:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.055+0000 7f2ce7e83640 1 -- 172.21.15.27:0/2320793531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce00c9180 con 0x7f2ce00cc170 2023-08-11T15:51:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.056+0000 7f2ce7e83640 1 -- 172.21.15.27:0/2320793531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce00c9c60 con 0x7f2ce00cc170 2023-08-11T15:51:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.057+0000 7f2cd67fc640 1 -- 172.21.15.27:0/2320793531 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2cd003a040 con 0x7f2ce00cc170 2023-08-11T15:51:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.059+0000 7f2cd67fc640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cc406a050 0x7f2cc406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.059+0000 7f2cd67fc640 1 -- 172.21.15.27:0/2320793531 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2cd00aa600 con 0x7f2ce00cc170 2023-08-11T15:51:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.059+0000 7f2ce5c1f640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cc406a050 0x7f2cc406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.061+0000 7f2ce5c1f640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cc406a050 0x7f2cc406c510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc000f60 tx=0x7f2cdc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.063+0000 7f2cd67fc640 1 -- 172.21.15.27:0/2320793531 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2ce00c9c60 con 0x7f2ce00cc170 2023-08-11T15:51:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.257+0000 7f2ce7e83640 1 -- 172.21.15.27:0/2320793531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ce00c9c60 con 0x7f2ce00cc170 2023-08-11T15:51:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.258+0000 7f2cd67fc640 1 -- 172.21.15.27:0/2320793531 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ce00c9c60 con 0x7f2ce00cc170 2023-08-11T15:51:21.261 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:51:21.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 -- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cc406a050 msgr2=0x7f2cc406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cc406a050 0x7f2cc406c510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc000f60 tx=0x7f2cdc009040 comp rx=0 tx=0).stop 2023-08-11T15:51:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 -- 172.21.15.27:0/2320793531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce00cc170 msgr2=0x7f2ce00b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce00cc170 0x7f2ce00b32e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00098c0 tx=0x7f2cd00046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 -- 172.21.15.27:0/2320793531 shutdown_connections 2023-08-11T15:51:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cc406a050 0x7f2cc406c510 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ce0159f80 0x7f2ce00b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce00cc170 0x7f2ce00b32e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 --2- 172.21.15.27:0/2320793531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce00cb110 0x7f2ce0155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 -- 172.21.15.27:0/2320793531 >> 172.21.15.27:0/2320793531 conn(0x7f2ce00afbe0 msgr2=0x7f2ce006f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 -- 172.21.15.27:0/2320793531 shutdown_connections 2023-08-11T15:51:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.261+0000 7f2ce7e83640 1 -- 172.21.15.27:0/2320793531 wait complete. 2023-08-11T15:51:21.284 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:51:21.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.517+0000 7fcf75762640 1 Processor -- start 2023-08-11T15:51:21.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.517+0000 7fcf75762640 1 -- start start 2023-08-11T15:51:21.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.519+0000 7fcf75762640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcf701590d0 0x7fcf700ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.519+0000 7fcf75762640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 0x7fcf7006cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.519+0000 7fcf75762640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7006d330 con 0x7fcf7015d760 2023-08-11T15:51:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.519+0000 7fcf75762640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf700cce80 con 0x7fcf701590d0 2023-08-11T15:51:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.519+0000 7fcf75762640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf700cb020 con 0x7fcf7006c9a0 2023-08-11T15:51:21.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.519+0000 7fcf6ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcf701590d0 0x7fcf700ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.519+0000 7fcf677fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcf7015d760 0x7fcf70070030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48952/0 (socket says 172.21.15.27:48952) 2023-08-11T15:51:21.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.519+0000 7fcf677fe640 1 -- 172.21.15.27:0/2739572588 learned_addr learned my addr 172.21.15.27:0/2739572588 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:21.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.520+0000 7fcf74f61640 1 --2- 172.21.15.27:0/2739572588 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 0x7fcf7006cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.520+0000 7fcf6ffff640 1 -- 172.21.15.27:0/2739572588 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 msgr2=0x7fcf7006cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:21.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.520+0000 7fcf6f7fe640 1 -- 172.21.15.27:0/2739572588 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990740321 0 0) 0x7fcf58001110 con 0x7fcf7015d760 2023-08-11T15:51:21.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.520+0000 7fcf6ffff640 1 --2- 172.21.15.27:0/2739572588 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 0x7fcf7006cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.520+0000 7fcf6ffff640 1 -- 172.21.15.27:0/2739572588 >> v1:172.21.15.27:6789/0 conn(0x7fcf7015d760 legacy=0x7fcf70070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.520+0000 7fcf6ffff640 1 -- 172.21.15.27:0/2739572588 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf700cb300 con 0x7fcf701590d0 2023-08-11T15:51:21.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.521+0000 7fcf6ffff640 1 --2- 172.21.15.27:0/2739572588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 0x7fcf700ca6f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009d90 tx=0x7fcf6002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9d953632ea7279 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:21.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf6f7fe640 1 -- 172.21.15.27:0/2739572588 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6002cbc0 con 0x7fcf701590d0 2023-08-11T15:51:21.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf6f7fe640 1 -- 172.21.15.27:0/2739572588 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6002cd20 con 0x7fcf701590d0 2023-08-11T15:51:21.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf6f7fe640 1 -- 172.21.15.27:0/2739572588 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60034660 con 0x7fcf701590d0 2023-08-11T15:51:21.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf75762640 1 -- 172.21.15.27:0/2739572588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 msgr2=0x7fcf700ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf75762640 1 --2- 172.21.15.27:0/2739572588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 0x7fcf700ca6f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009d90 tx=0x7fcf6002be10 comp rx=0 tx=0).stop 2023-08-11T15:51:21.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf75762640 1 -- 172.21.15.27:0/2739572588 shutdown_connections 2023-08-11T15:51:21.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf75762640 1 --2- 172.21.15.27:0/2739572588 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 0x7fcf7006cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf75762640 1 --2- 172.21.15.27:0/2739572588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 0x7fcf700ca6f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf75762640 1 -- 172.21.15.27:0/2739572588 >> 172.21.15.27:0/2739572588 conn(0x7fcf700afa90 msgr2=0x7fcf700afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:21.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf75762640 1 -- 172.21.15.27:0/2739572588 shutdown_connections 2023-08-11T15:51:21.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.522+0000 7fcf75762640 1 -- 172.21.15.27:0/2739572588 wait complete. 2023-08-11T15:51:21.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.524+0000 7fcf75762640 1 Processor -- start 2023-08-11T15:51:21.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.524+0000 7fcf75762640 1 -- start start 2023-08-11T15:51:21.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.525+0000 7fcf75762640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 0x7fcf7014f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.525+0000 7fcf75762640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 0x7fcf7015a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.525+0000 7fcf75762640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf7015d760 0x7fcf7014fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.525+0000 7fcf75762640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf700a7560 con 0x7fcf701590d0 2023-08-11T15:51:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.525+0000 7fcf75762640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7015bce0 con 0x7fcf7015d760 2023-08-11T15:51:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.525+0000 7fcf75762640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7015c500 con 0x7fcf7006c9a0 2023-08-11T15:51:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 0x7fcf7015a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 0x7fcf7015a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46180/0 (socket says 172.21.15.27:46180) 2023-08-11T15:51:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf677fe640 1 -- 172.21.15.27:0/3647159939 learned_addr learned my addr 172.21.15.27:0/3647159939 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf74f61640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf7015d760 0x7fcf7014fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf6ffff640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 0x7fcf7014f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf677fe640 1 -- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 msgr2=0x7fcf7014f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf677fe640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 0x7fcf7014f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf677fe640 1 -- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf7015d760 msgr2=0x7fcf7014fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf677fe640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf7015d760 0x7fcf7014fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf677fe640 1 -- 172.21.15.27:0/3647159939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf60009a70 con 0x7fcf701590d0 2023-08-11T15:51:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf6ffff640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 0x7fcf7014f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.526+0000 7fcf74f61640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf7015d760 0x7fcf7014fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.527+0000 7fcf677fe640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 0x7fcf7015a7f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800c780 tx=0x7fcf5800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.527+0000 7fcf6d7fa640 1 -- 172.21.15.27:0/3647159939 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf580072d0 con 0x7fcf701590d0 2023-08-11T15:51:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.527+0000 7fcf6d7fa640 1 -- 172.21.15.27:0/3647159939 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf58007450 con 0x7fcf701590d0 2023-08-11T15:51:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.527+0000 7fcf6d7fa640 1 -- 172.21.15.27:0/3647159939 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf580047e0 con 0x7fcf701590d0 2023-08-11T15:51:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.527+0000 7fcf75762640 1 -- 172.21.15.27:0/3647159939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7015bf40 con 0x7fcf701590d0 2023-08-11T15:51:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.527+0000 7fcf75762640 1 -- 172.21.15.27:0/3647159939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf70159ff0 con 0x7fcf701590d0 2023-08-11T15:51:21.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.528+0000 7fcf75762640 1 -- 172.21.15.27:0/3647159939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7006cdf0 con 0x7fcf701590d0 2023-08-11T15:51:21.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.529+0000 7fcf6d7fa640 1 -- 172.21.15.27:0/3647159939 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcf58004960 con 0x7fcf701590d0 2023-08-11T15:51:21.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.530+0000 7fcf6d7fa640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf5006a050 0x7fcf5006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:21.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.530+0000 7fcf6d7fa640 1 -- 172.21.15.27:0/3647159939 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcf58087a30 con 0x7fcf701590d0 2023-08-11T15:51:21.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.531+0000 7fcf6ffff640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf5006a050 0x7fcf5006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:21.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.532+0000 7fcf6ffff640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf5006a050 0x7fcf5006c510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf60036000 tx=0x7fcf60030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:21.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.535+0000 7fcf6d7fa640 1 -- 172.21.15.27:0/3647159939 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fcf58052e20 con 0x7fcf701590d0 2023-08-11T15:51:21.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.728+0000 7fcf75762640 1 -- 172.21.15.27:0/3647159939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf7006d330 con 0x7fcf701590d0 2023-08-11T15:51:21.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.729+0000 7fcf6d7fa640 1 -- 172.21.15.27:0/3647159939 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf7006d330 con 0x7fcf701590d0 2023-08-11T15:51:21.732 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:51:21.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 -- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf5006a050 msgr2=0x7fcf5006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf5006a050 0x7fcf5006c510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf60036000 tx=0x7fcf60030000 comp rx=0 tx=0).stop 2023-08-11T15:51:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 -- 172.21.15.27:0/3647159939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 msgr2=0x7fcf7015a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 0x7fcf7015a7f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800c780 tx=0x7fcf5800cc40 comp rx=0 tx=0).stop 2023-08-11T15:51:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 -- 172.21.15.27:0/3647159939 shutdown_connections 2023-08-11T15:51:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf5006a050 0x7fcf5006c510 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf7015d760 0x7fcf7014fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf701590d0 0x7fcf7015a7f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 --2- 172.21.15.27:0/3647159939 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf7006c9a0 0x7fcf7014f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 -- 172.21.15.27:0/3647159939 >> 172.21.15.27:0/3647159939 conn(0x7fcf700afa90 msgr2=0x7fcf700adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 -- 172.21.15.27:0/3647159939 shutdown_connections 2023-08-11T15:51:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.732+0000 7fcf75762640 1 -- 172.21.15.27:0/3647159939 wait complete. 2023-08-11T15:51:21.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:51:22.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:21.999+0000 7fedd63a4640 1 Processor -- start 2023-08-11T15:51:22.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.000+0000 7fedd63a4640 1 -- start start 2023-08-11T15:51:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.001+0000 7fedd63a4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fedd00b38f0 0x7fedd00d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.001+0000 7fedd63a4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fedd0161210 0x7fedd00b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.001+0000 7fedd63a4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd00af250 con 0x7fedd00b3090 2023-08-11T15:51:22.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.001+0000 7fedd63a4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd00b0190 con 0x7fedd0161210 2023-08-11T15:51:22.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.001+0000 7fedd63a4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd00af510 con 0x7fedd00b38f0 2023-08-11T15:51:22.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedcf7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fedd00b3090 0x7fedd0160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48954/0 (socket says 172.21.15.27:48954) 2023-08-11T15:51:22.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedcf7fe640 1 -- 172.21.15.27:0/925357109 learned_addr learned my addr 172.21.15.27:0/925357109 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:22.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedd4941640 1 --2- 172.21.15.27:0/925357109 >> v2:172.21.15.27:3300/0 conn(0x7fedd0161210 0x7fedd00b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedcffff640 1 --2- 172.21.15.27:0/925357109 >> v2:172.21.15.119:3300/0 conn(0x7fedd00b38f0 0x7fedd00d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedceffd640 1 -- 172.21.15.27:0/925357109 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735504933 0 0) 0x7fedd00af250 con 0x7fedd00b3090 2023-08-11T15:51:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedceffd640 1 -- 172.21.15.27:0/925357109 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedbc003520 con 0x7fedd00b3090 2023-08-11T15:51:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedd4941640 1 -- 172.21.15.27:0/925357109 >> v2:172.21.15.119:3300/0 conn(0x7fedd00b38f0 msgr2=0x7fedd00d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedd4941640 1 --2- 172.21.15.27:0/925357109 >> v2:172.21.15.119:3300/0 conn(0x7fedd00b38f0 0x7fedd00d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedd4941640 1 -- 172.21.15.27:0/925357109 >> v1:172.21.15.27:6789/0 conn(0x7fedd00b3090 legacy=0x7fedd0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedd4941640 1 -- 172.21.15.27:0/925357109 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd00af7f0 con 0x7fedd0161210 2023-08-11T15:51:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.002+0000 7fedcffff640 1 --2- 172.21.15.27:0/925357109 >> v2:172.21.15.119:3300/0 conn(0x7fedd00b38f0 0x7fedd00d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.004+0000 7fedd4941640 1 --2- 172.21.15.27:0/925357109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd0161210 0x7fedd00b05b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fedd008b230 tx=0x7fedc802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6ba9a40ef10584 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.004+0000 7fedceffd640 1 -- 172.21.15.27:0/925357109 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8031830 con 0x7fedd0161210 2023-08-11T15:51:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.004+0000 7fedceffd640 1 -- 172.21.15.27:0/925357109 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc8031990 con 0x7fedd0161210 2023-08-11T15:51:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.004+0000 7fedceffd640 1 -- 172.21.15.27:0/925357109 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8031d20 con 0x7fedd0161210 2023-08-11T15:51:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.004+0000 7fedd63a4640 1 -- 172.21.15.27:0/925357109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd0161210 msgr2=0x7fedd00b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.004+0000 7fedd63a4640 1 --2- 172.21.15.27:0/925357109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd0161210 0x7fedd00b05b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fedd008b230 tx=0x7fedc802b8d0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.005+0000 7fedd63a4640 1 -- 172.21.15.27:0/925357109 shutdown_connections 2023-08-11T15:51:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.005+0000 7fedd63a4640 1 --2- 172.21.15.27:0/925357109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd0161210 0x7fedd00b05b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.005+0000 7fedd63a4640 1 --2- 172.21.15.27:0/925357109 >> v2:172.21.15.119:3300/0 conn(0x7fedd00b38f0 0x7fedd00d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.005+0000 7fedd63a4640 1 -- 172.21.15.27:0/925357109 >> 172.21.15.27:0/925357109 conn(0x7fedd009b960 msgr2=0x7fedd00d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:22.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.005+0000 7fedd63a4640 1 -- 172.21.15.27:0/925357109 shutdown_connections 2023-08-11T15:51:22.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.005+0000 7fedd63a4640 1 -- 172.21.15.27:0/925357109 wait complete. 2023-08-11T15:51:22.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.007+0000 7fedd63a4640 1 Processor -- start 2023-08-11T15:51:22.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.007+0000 7fedd63a4640 1 -- start start 2023-08-11T15:51:22.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.008+0000 7fedd63a4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fedd00b3090 0x7fedd00cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.008+0000 7fedd63a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd00b38f0 0x7fedd01f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.008+0000 7fedd63a4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd0161210 0x7fedd00ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.008+0000 7fedd63a4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00d0040 con 0x7fedd00b38f0 2023-08-11T15:51:22.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.008+0000 7fedd63a4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd01f26c0 con 0x7fedd0161210 2023-08-11T15:51:22.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.008+0000 7fedd63a4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd01f29a0 con 0x7fedd00b3090 2023-08-11T15:51:22.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.009+0000 7fedcf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd00b38f0 0x7fedd01f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.009+0000 7fedcf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd00b38f0 0x7fedd01f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46190/0 (socket says 172.21.15.27:46190) 2023-08-11T15:51:22.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.009+0000 7fedcf7fe640 1 -- 172.21.15.27:0/4107996578 learned_addr learned my addr 172.21.15.27:0/4107996578 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:22.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.009+0000 7fedcffff640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fedd00b3090 0x7fedd00cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.009+0000 7fedd4941640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd0161210 0x7fedd00ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.009+0000 7fedcf7fe640 1 -- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fedd00b3090 msgr2=0x7fedd00cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.009+0000 7fedcf7fe640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fedd00b3090 0x7fedd00cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.009+0000 7fedcf7fe640 1 -- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd0161210 msgr2=0x7fedd00ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.009+0000 7fedcf7fe640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd0161210 0x7fedd00ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.009+0000 7fedcf7fe640 1 -- 172.21.15.27:0/4107996578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc8009400 con 0x7fedd00b38f0 2023-08-11T15:51:22.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.010+0000 7fedcf7fe640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd00b38f0 0x7fedd01f1c00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c500 tx=0x7fedb800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:22.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.010+0000 7fedccff9640 1 -- 172.21.15.27:0/4107996578 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8005ce0 con 0x7fedd00b38f0 2023-08-11T15:51:22.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.010+0000 7fedccff9640 1 -- 172.21.15.27:0/4107996578 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8005e40 con 0x7fedd00b38f0 2023-08-11T15:51:22.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.010+0000 7fedd63a4640 1 -- 172.21.15.27:0/4107996578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd00cd0e0 con 0x7fedd00b38f0 2023-08-11T15:51:22.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.010+0000 7fedccff9640 1 -- 172.21.15.27:0/4107996578 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8007330 con 0x7fedd00b38f0 2023-08-11T15:51:22.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.010+0000 7fedd63a4640 1 -- 172.21.15.27:0/4107996578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd00c99c0 con 0x7fedd00b38f0 2023-08-11T15:51:22.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.011+0000 7fedd63a4640 1 -- 172.21.15.27:0/4107996578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd004e420 con 0x7fedd00b38f0 2023-08-11T15:51:22.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.012+0000 7fedccff9640 1 -- 172.21.15.27:0/4107996578 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fedb8007550 con 0x7fedd00b38f0 2023-08-11T15:51:22.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.013+0000 7fedccff9640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fedb006a050 0x7fedb006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.013+0000 7fedccff9640 1 -- 172.21.15.27:0/4107996578 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fedb8087ed0 con 0x7fedd00b38f0 2023-08-11T15:51:22.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.014+0000 7fedcffff640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fedb006a050 0x7fedb006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.015+0000 7fedcffff640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fedb006a050 0x7fedb006c510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedd01f62b0 tx=0x7fedc0003a10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:22.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.018+0000 7fedccff9640 1 -- 172.21.15.27:0/4107996578 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fedb8053420 con 0x7fedd00b38f0 2023-08-11T15:51:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.214+0000 7fedd63a4640 1 -- 172.21.15.27:0/4107996578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fedd00b86b0 con 0x7fedd00b38f0 2023-08-11T15:51:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.215+0000 7fedccff9640 1 -- 172.21.15.27:0/4107996578 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fedb8017020 con 0x7fedd00b38f0 2023-08-11T15:51:22.218 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:51:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 -- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fedb006a050 msgr2=0x7fedb006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fedb006a050 0x7fedb006c510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedd01f62b0 tx=0x7fedc0003a10 comp rx=0 tx=0).stop 2023-08-11T15:51:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 -- 172.21.15.27:0/4107996578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd00b38f0 msgr2=0x7fedd01f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd00b38f0 0x7fedd01f1c00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c500 tx=0x7fedb800c9c0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 -- 172.21.15.27:0/4107996578 shutdown_connections 2023-08-11T15:51:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fedb006a050 0x7fedb006c510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd0161210 0x7fedd00ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd00b38f0 0x7fedd01f1c00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 --2- 172.21.15.27:0/4107996578 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fedd00b3090 0x7fedd00cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 -- 172.21.15.27:0/4107996578 >> 172.21.15.27:0/4107996578 conn(0x7fedd009b960 msgr2=0x7fedd006bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 -- 172.21.15.27:0/4107996578 shutdown_connections 2023-08-11T15:51:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.219+0000 7fedd63a4640 1 -- 172.21.15.27:0/4107996578 wait complete. 2023-08-11T15:51:22.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:51:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.482+0000 7f218619b640 1 Processor -- start 2023-08-11T15:51:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.482+0000 7f218619b640 1 -- start start 2023-08-11T15:51:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.483+0000 7f218619b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 0x7f21800d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.484+0000 7f218619b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f21800cf240 0x7f21800b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.484+0000 7f218619b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800d0430 con 0x7f21800d1cf0 2023-08-11T15:51:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.484+0000 7f218619b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800d0710 con 0x7f21800cf240 2023-08-11T15:51:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.484+0000 7f218619b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21800d09f0 con 0x7f218015c3d0 2023-08-11T15:51:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.484+0000 7f2184998640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f21800d1cf0 0x7f21800ceb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48956/0 (socket says 172.21.15.27:48956) 2023-08-11T15:51:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.484+0000 7f2184998640 1 -- 172.21.15.27:0/842702043 learned_addr learned my addr 172.21.15.27:0/842702043 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.484+0000 7f2185199640 1 --2- 172.21.15.27:0/842702043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 0x7f21800d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.484+0000 7f218599a640 1 --2- 172.21.15.27:0/842702043 >> v2:172.21.15.119:3300/0 conn(0x7f21800cf240 0x7f21800b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.485+0000 7f2177fff640 1 -- 172.21.15.27:0/842702043 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126859358 0 0) 0x7f21800d0430 con 0x7f21800d1cf0 2023-08-11T15:51:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.485+0000 7f2177fff640 1 -- 172.21.15.27:0/842702043 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f216c003520 con 0x7f21800d1cf0 2023-08-11T15:51:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.485+0000 7f2185199640 1 -- 172.21.15.27:0/842702043 >> v2:172.21.15.119:3300/0 conn(0x7f21800cf240 msgr2=0x7f21800b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.485+0000 7f2185199640 1 --2- 172.21.15.27:0/842702043 >> v2:172.21.15.119:3300/0 conn(0x7f21800cf240 0x7f21800b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.485+0000 7f2185199640 1 -- 172.21.15.27:0/842702043 >> v1:172.21.15.27:6789/0 conn(0x7f21800d1cf0 legacy=0x7f21800ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.485+0000 7f2177fff640 1 -- 172.21.15.27:0/842702043 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3881181205 0 0) 0x7f216c003520 con 0x7f21800d1cf0 2023-08-11T15:51:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.485+0000 7f2185199640 1 -- 172.21.15.27:0/842702043 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800d0cd0 con 0x7f218015c3d0 2023-08-11T15:51:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.485+0000 7f218599a640 1 --2- 172.21.15.27:0/842702043 >> v2:172.21.15.119:3300/0 conn(0x7f21800cf240 0x7f21800b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:22.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.486+0000 7f2185199640 1 --2- 172.21.15.27:0/842702043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 0x7f21800d17b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f217c0097a0 tx=0x7f217c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b7b683338075e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:22.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.487+0000 7f2177fff640 1 -- 172.21.15.27:0/842702043 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170001670 con 0x7f218015c3d0 2023-08-11T15:51:22.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.487+0000 7f2177fff640 1 -- 172.21.15.27:0/842702043 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c02e050 con 0x7f218015c3d0 2023-08-11T15:51:22.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.487+0000 7f2177fff640 1 -- 172.21.15.27:0/842702043 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c030730 con 0x7f218015c3d0 2023-08-11T15:51:22.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.487+0000 7f218619b640 1 -- 172.21.15.27:0/842702043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 msgr2=0x7f21800d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.487+0000 7f218619b640 1 --2- 172.21.15.27:0/842702043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 0x7f21800d17b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f217c0097a0 tx=0x7f217c02b790 comp rx=0 tx=0).stop 2023-08-11T15:51:22.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.487+0000 7f218619b640 1 -- 172.21.15.27:0/842702043 shutdown_connections 2023-08-11T15:51:22.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.487+0000 7f218619b640 1 --2- 172.21.15.27:0/842702043 >> v2:172.21.15.119:3300/0 conn(0x7f21800cf240 0x7f21800b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.487+0000 7f218619b640 1 --2- 172.21.15.27:0/842702043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 0x7f21800d17b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.487+0000 7f218619b640 1 -- 172.21.15.27:0/842702043 >> 172.21.15.27:0/842702043 conn(0x7f218006a270 msgr2=0x7f218006a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:22.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.488+0000 7f218619b640 1 -- 172.21.15.27:0/842702043 shutdown_connections 2023-08-11T15:51:22.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.488+0000 7f218619b640 1 -- 172.21.15.27:0/842702043 wait complete. 2023-08-11T15:51:22.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.489+0000 7f218619b640 1 Processor -- start 2023-08-11T15:51:22.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.489+0000 7f218619b640 1 -- start start 2023-08-11T15:51:22.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.490+0000 7f218619b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21800cf240 0x7f21801f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.490+0000 7f218619b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21800d1cf0 0x7f21800ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.490+0000 7f218619b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 0x7f2180150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.490+0000 7f218619b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21801542b0 con 0x7f21800cf240 2023-08-11T15:51:22.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.490+0000 7f218619b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180154590 con 0x7f21800d1cf0 2023-08-11T15:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.490+0000 7f218619b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2180154870 con 0x7f218015c3d0 2023-08-11T15:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.491+0000 7f2185199640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21800cf240 0x7f21801f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.491+0000 7f2185199640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21800cf240 0x7f21801f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46196/0 (socket says 172.21.15.27:46196) 2023-08-11T15:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.491+0000 7f2185199640 1 -- 172.21.15.27:0/3062720769 learned_addr learned my addr 172.21.15.27:0/3062720769 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.491+0000 7f218599a640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 0x7f2180150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.491+0000 7f2184998640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21800d1cf0 0x7f21800ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.491+0000 7f2185199640 1 -- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 msgr2=0x7f2180150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.491+0000 7f2185199640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 0x7f2180150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.491+0000 7f2185199640 1 -- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21800d1cf0 msgr2=0x7f21800ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.491+0000 7f2185199640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21800d1cf0 0x7f21800ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.491+0000 7f2185199640 1 -- 172.21.15.27:0/3062720769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2178009a70 con 0x7f21800cf240 2023-08-11T15:51:22.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.492+0000 7f2185199640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21800cf240 0x7f21801f53a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f217c035040 tx=0x7f217c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:22.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.492+0000 7f2175ffb640 1 -- 172.21.15.27:0/3062720769 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c03d890 con 0x7f21800cf240 2023-08-11T15:51:22.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.492+0000 7f2175ffb640 1 -- 172.21.15.27:0/3062720769 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c03da30 con 0x7f21800cf240 2023-08-11T15:51:22.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.492+0000 7f2175ffb640 1 -- 172.21.15.27:0/3062720769 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c03dde0 con 0x7f21800cf240 2023-08-11T15:51:22.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.492+0000 7f218619b640 1 -- 172.21.15.27:0/3062720769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c009400 con 0x7f21800cf240 2023-08-11T15:51:22.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.492+0000 7f218619b640 1 -- 172.21.15.27:0/3062720769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2180154e30 con 0x7f21800cf240 2023-08-11T15:51:22.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.493+0000 7f218619b640 1 -- 172.21.15.27:0/3062720769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2180066e80 con 0x7f21800cf240 2023-08-11T15:51:22.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.494+0000 7f2175ffb640 1 -- 172.21.15.27:0/3062720769 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f217c039040 con 0x7f21800cf240 2023-08-11T15:51:22.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.495+0000 7f2175ffb640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f216406a050 0x7f216406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:22.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.495+0000 7f2175ffb640 1 -- 172.21.15.27:0/3062720769 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f217c0aa7e0 con 0x7f21800cf240 2023-08-11T15:51:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.496+0000 7f2184998640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f216406a050 0x7f216406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.497+0000 7f2184998640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f216406a050 0x7f216406c510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f21801f5090 tx=0x7f2170009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.500+0000 7f2175ffb640 1 -- 172.21.15.27:0/3062720769 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f217c075c60 con 0x7f21800cf240 2023-08-11T15:51:22.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.710+0000 7f218619b640 1 -- 172.21.15.27:0/3062720769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f21800cf790 con 0x7f21800cf240 2023-08-11T15:51:22.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.711+0000 7f2175ffb640 1 -- 172.21.15.27:0/3062720769 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f217c079280 con 0x7f21800cf240 2023-08-11T15:51:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:51:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.714+0000 7f218619b640 1 -- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f216406a050 msgr2=0x7f216406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.714+0000 7f218619b640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f216406a050 0x7f216406c510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f21801f5090 tx=0x7f2170009040 comp rx=0 tx=0).stop 2023-08-11T15:51:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.714+0000 7f218619b640 1 -- 172.21.15.27:0/3062720769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21800cf240 msgr2=0x7f21801f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.714+0000 7f218619b640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21800cf240 0x7f21801f53a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f217c035040 tx=0x7f217c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.715+0000 7f218619b640 1 -- 172.21.15.27:0/3062720769 shutdown_connections 2023-08-11T15:51:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.715+0000 7f218619b640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f216406a050 0x7f216406c510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.715+0000 7f218619b640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f218015c3d0 0x7f2180150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.715+0000 7f218619b640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21800d1cf0 0x7f21800ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.715+0000 7f218619b640 1 --2- 172.21.15.27:0/3062720769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21800cf240 0x7f21801f53a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.715+0000 7f218619b640 1 -- 172.21.15.27:0/3062720769 >> 172.21.15.27:0/3062720769 conn(0x7f218006a270 msgr2=0x7f21800b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.715+0000 7f218619b640 1 -- 172.21.15.27:0/3062720769 shutdown_connections 2023-08-11T15:51:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:22.715+0000 7f218619b640 1 -- 172.21.15.27:0/3062720769 wait complete. 2023-08-11T15:51:22.737 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-08-11T15:51:22.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:22.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:51:22.834 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[95419]: fuse finished with error 0 and tester_r 0 2023-08-11T15:51:22.840 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:51:22.895 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:51:22.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:22.896 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-08-11T15:51:22.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:23.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.221+0000 7f829a742640 1 Processor -- start 2023-08-11T15:51:23.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.221+0000 7f829a742640 1 -- start start 2023-08-11T15:51:23.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f829a742640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82940cb430 0x7f82940b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f829a742640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f829406f5d0 0x7f829406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f829a742640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82940c8a30 con 0x7f829415a140 2023-08-11T15:51:23.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f829a742640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829406ffa0 con 0x7f82940cb430 2023-08-11T15:51:23.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f829a742640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829406c0c0 con 0x7f829406f5d0 2023-08-11T15:51:23.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8293fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f829415a140 0x7f829406b870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48964/0 (socket says 172.21.15.27:48964) 2023-08-11T15:51:23.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8293fff640 1 -- 172.21.15.27:0/3721894211 learned_addr learned my addr 172.21.15.27:0/3721894211 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:23.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8298cdf640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f829406f5d0 0x7f829406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f82937fe640 1 --2- 172.21.15.27:0/3721894211 >> v2:172.21.15.119:3300/0 conn(0x7f82940cb430 0x7f82940b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023270914 0 0) 0x7f82940c8a30 con 0x7f829415a140 2023-08-11T15:51:23.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f827c003520 con 0x7f829415a140 2023-08-11T15:51:23.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3411041224 0 0) 0x7f827c003520 con 0x7f829415a140 2023-08-11T15:51:23.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f829406f5d0 msgr2=0x7f829406fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:23.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f829406f5d0 0x7f829406fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 >> v2:172.21.15.119:3300/0 conn(0x7f82940cb430 msgr2=0x7f82940b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:23.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 --2- 172.21.15.27:0/3721894211 >> v2:172.21.15.119:3300/0 conn(0x7f82940cb430 0x7f82940b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829406c3a0 con 0x7f829415a140 2023-08-11T15:51:23.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f82840017f0 con 0x7f829415a140 2023-08-11T15:51:23.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 >> v1:172.21.15.27:6789/0 conn(0x7f829415a140 legacy=0x7f829406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f829406f5d0 0x7f827c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f827c003ef0 0x7f827c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f827c004d00 0x7f827c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829406ffa0 con 0x7f827c004d00 2023-08-11T15:51:23.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829406c0c0 con 0x7f827c003ef0 2023-08-11T15:51:23.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.223+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c00fd90 con 0x7f829406f5d0 2023-08-11T15:51:23.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.224+0000 7f8298cdf640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f827c004d00 0x7f827c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.224+0000 7f8293fff640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f829406f5d0 0x7f827c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.224+0000 7f82937fe640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f827c003ef0 0x7f827c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.224+0000 7f8298cdf640 1 -- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f829406f5d0 msgr2=0x7f827c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:23.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.224+0000 7f8298cdf640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f829406f5d0 0x7f827c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.224+0000 7f8298cdf640 1 -- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f827c003ef0 msgr2=0x7f827c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:23.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.224+0000 7f8298cdf640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f827c003ef0 0x7f827c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.224+0000 7f8298cdf640 1 -- 172.21.15.27:0/3721894211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c003520 con 0x7f827c004d00 2023-08-11T15:51:23.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.225+0000 7f8298cdf640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f827c004d00 0x7f827c005180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f828c009fc0 tx=0x7f827c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcd9d01e8c9d80a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:23.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.226+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c02cc00 con 0x7f827c004d00 2023-08-11T15:51:23.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.226+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c00f040 con 0x7f827c004d00 2023-08-11T15:51:23.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.226+0000 7f8292ffd640 1 -- 172.21.15.27:0/3721894211 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c034440 con 0x7f827c004d00 2023-08-11T15:51:23.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.226+0000 7f829a742640 1 -- 172.21.15.27:0/3721894211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f827c004d00 msgr2=0x7f827c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.226+0000 7f829a742640 1 --2- 172.21.15.27:0/3721894211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f827c004d00 0x7f827c005180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f828c009fc0 tx=0x7f827c0057f0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.226+0000 7f8298cdf640 1 -- 172.21.15.27:0/3721894211 reap_dead start 2023-08-11T15:51:23.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.226+0000 7f829a742640 1 -- 172.21.15.27:0/3721894211 shutdown_connections 2023-08-11T15:51:23.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.226+0000 7f829a742640 1 -- 172.21.15.27:0/3721894211 >> 172.21.15.27:0/3721894211 conn(0x7f82940afb70 msgr2=0x7f82940aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:23.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.227+0000 7f829a742640 1 -- 172.21.15.27:0/3721894211 shutdown_connections 2023-08-11T15:51:23.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.227+0000 7f829a742640 1 -- 172.21.15.27:0/3721894211 wait complete. 2023-08-11T15:51:23.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.228+0000 7f829a742640 1 Processor -- start 2023-08-11T15:51:23.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.229+0000 7f829a742640 1 -- start start 2023-08-11T15:51:23.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f829a742640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f827c004d00 0x7f82940c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f829a742640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82940c2b20 0x7f8294145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f829a742640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8294158fe0 0x7f8294159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f829a742640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82941599a0 con 0x7f8294158fe0 2023-08-11T15:51:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f829a742640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82940c3b80 con 0x7f82940c2b20 2023-08-11T15:51:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f829a742640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82940c32e0 con 0x7f827c004d00 2023-08-11T15:51:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f8298cdf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8294158fe0 0x7f8294159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f8298cdf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8294158fe0 0x7f8294159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46206/0 (socket says 172.21.15.27:46206) 2023-08-11T15:51:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f8298cdf640 1 -- 172.21.15.27:0/2505738559 learned_addr learned my addr 172.21.15.27:0/2505738559 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f8293fff640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f827c004d00 0x7f82940c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f82937fe640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82940c2b20 0x7f8294145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f8298cdf640 1 -- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f827c004d00 msgr2=0x7f82940c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f8298cdf640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f827c004d00 0x7f82940c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f8298cdf640 1 -- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82940c2b20 msgr2=0x7f8294145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f8298cdf640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82940c2b20 0x7f8294145280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.230+0000 7f8298cdf640 1 -- 172.21.15.27:0/2505738559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82940c35c0 con 0x7f8294158fe0 2023-08-11T15:51:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.231+0000 7f8298cdf640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8294158fe0 0x7f8294159460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f828c00e230 tx=0x7f828c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.231+0000 7f8290ff9640 1 -- 172.21.15.27:0/2505738559 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c034d90 con 0x7f8294158fe0 2023-08-11T15:51:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.231+0000 7f8290ff9640 1 -- 172.21.15.27:0/2505738559 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c003990 con 0x7f8294158fe0 2023-08-11T15:51:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.231+0000 7f8290ff9640 1 -- 172.21.15.27:0/2505738559 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c003d20 con 0x7f8294158fe0 2023-08-11T15:51:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.231+0000 7f829a742640 1 -- 172.21.15.27:0/2505738559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82940c38a0 con 0x7f8294158fe0 2023-08-11T15:51:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.231+0000 7f829a742640 1 -- 172.21.15.27:0/2505738559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82940c2fa0 con 0x7f8294158fe0 2023-08-11T15:51:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.232+0000 7f829a742640 1 -- 172.21.15.27:0/2505738559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82940ca8b0 con 0x7f8294158fe0 2023-08-11T15:51:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.234+0000 7f8290ff9640 1 -- 172.21.15.27:0/2505738559 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f828c004030 con 0x7f8294158fe0 2023-08-11T15:51:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.234+0000 7f8290ff9640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f828006a050 0x7f828006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.234+0000 7f8290ff9640 1 -- 172.21.15.27:0/2505738559 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f828c0aa240 con 0x7f8294158fe0 2023-08-11T15:51:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.235+0000 7f8293fff640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f828006a050 0x7f828006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.236+0000 7f8293fff640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f828006a050 0x7f828006c510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82840011d0 tx=0x7f8284009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.239+0000 7f8290ff9640 1 -- 172.21.15.27:0/2505738559 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f82940ca8b0 con 0x7f8294158fe0 2023-08-11T15:51:23.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.435+0000 7f829a742640 1 -- 172.21.15.27:0/2505738559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82940ca8b0 con 0x7f8294158fe0 2023-08-11T15:51:23.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.438+0000 7f8290ff9640 1 -- 172.21.15.27:0/2505738559 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7f82940ca8b0 con 0x7f8294158fe0 2023-08-11T15:51:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:23.440 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:23.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 -- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f828006a050 msgr2=0x7f828006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f828006a050 0x7f828006c510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82840011d0 tx=0x7f8284009040 comp rx=0 tx=0).stop 2023-08-11T15:51:23.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 -- 172.21.15.27:0/2505738559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8294158fe0 msgr2=0x7f8294159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8294158fe0 0x7f8294159460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f828c00e230 tx=0x7f828c009750 comp rx=0 tx=0).stop 2023-08-11T15:51:23.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 -- 172.21.15.27:0/2505738559 shutdown_connections 2023-08-11T15:51:23.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f828006a050 0x7f828006c510 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8294158fe0 0x7f8294159460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82940c2b20 0x7f8294145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 --2- 172.21.15.27:0/2505738559 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f827c004d00 0x7f82940c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 -- 172.21.15.27:0/2505738559 >> 172.21.15.27:0/2505738559 conn(0x7f82940afb70 msgr2=0x7f8294158720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:23.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 -- 172.21.15.27:0/2505738559 shutdown_connections 2023-08-11T15:51:23.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.441+0000 7f829a742640 1 -- 172.21.15.27:0/2505738559 wait complete. 2023-08-11T15:51:23.461 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:23.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.693+0000 7f4747cec640 1 Processor -- start 2023-08-11T15:51:23.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.693+0000 7f4747cec640 1 -- start start 2023-08-11T15:51:23.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.694+0000 7f4747cec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47400cb0c0 0x7f474015a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.695+0000 7f4747cec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f47400cbb70 0x7f47400708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.695+0000 7f4747cec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47400cc020 con 0x7f4740156e50 2023-08-11T15:51:23.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.695+0000 7f4747cec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47400ca460 con 0x7f47400cbb70 2023-08-11T15:51:23.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.695+0000 7f4747cec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4740070470 con 0x7f47400cb0c0 2023-08-11T15:51:23.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.695+0000 7f4746289640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f47400cbb70 0x7f47400708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.695+0000 7f4746289640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f47400cbb70 0x7f47400708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46210/0 (socket says 172.21.15.27:46210) 2023-08-11T15:51:23.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.695+0000 7f4746289640 1 -- 172.21.15.27:0/149086335 learned_addr learned my addr 172.21.15.27:0/149086335 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:23.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.695+0000 7f4745287640 1 --2- 172.21.15.27:0/149086335 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47400cb0c0 0x7f474015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.696+0000 7f4746289640 1 -- 172.21.15.27:0/149086335 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47400cb0c0 msgr2=0x7f474015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:23.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.696+0000 7f4746289640 1 --2- 172.21.15.27:0/149086335 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47400cb0c0 0x7f474015a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.696+0000 7f4746289640 1 -- 172.21.15.27:0/149086335 >> v1:172.21.15.119:6789/0 conn(0x7f4740156e50 legacy=0x7f47400ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.696+0000 7f4746289640 1 -- 172.21.15.27:0/149086335 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47400cc270 con 0x7f47400cbb70 2023-08-11T15:51:23.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.697+0000 7f4746289640 1 --2- 172.21.15.27:0/149086335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47400cbb70 0x7f47400708c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4738009e50 tx=0x7f473802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2f2f6eeaeb3a4d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:23.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.697+0000 7f4744a86640 1 -- 172.21.15.27:0/149086335 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473802cb00 con 0x7f47400cbb70 2023-08-11T15:51:23.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.697+0000 7f4744a86640 1 -- 172.21.15.27:0/149086335 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473802cc60 con 0x7f47400cbb70 2023-08-11T15:51:23.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.697+0000 7f4744a86640 1 -- 172.21.15.27:0/149086335 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47380343f0 con 0x7f47400cbb70 2023-08-11T15:51:23.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.698+0000 7f4747cec640 1 -- 172.21.15.27:0/149086335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47400cbb70 msgr2=0x7f47400708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.698+0000 7f4747cec640 1 --2- 172.21.15.27:0/149086335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47400cbb70 0x7f47400708c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4738009e50 tx=0x7f473802bf60 comp rx=0 tx=0).stop 2023-08-11T15:51:23.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.698+0000 7f4747cec640 1 -- 172.21.15.27:0/149086335 shutdown_connections 2023-08-11T15:51:23.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.698+0000 7f4747cec640 1 --2- 172.21.15.27:0/149086335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47400cbb70 0x7f47400708c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.698+0000 7f4747cec640 1 --2- 172.21.15.27:0/149086335 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47400cb0c0 0x7f474015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.698+0000 7f4747cec640 1 -- 172.21.15.27:0/149086335 >> 172.21.15.27:0/149086335 conn(0x7f47400afb70 msgr2=0x7f47400aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:23.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.698+0000 7f4747cec640 1 -- 172.21.15.27:0/149086335 shutdown_connections 2023-08-11T15:51:23.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.698+0000 7f4747cec640 1 -- 172.21.15.27:0/149086335 wait complete. 2023-08-11T15:51:23.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.700+0000 7f4747cec640 1 Processor -- start 2023-08-11T15:51:23.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.700+0000 7f4747cec640 1 -- start start 2023-08-11T15:51:23.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.701+0000 7f4747cec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47400cb0c0 0x7f47401e3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.701+0000 7f4747cec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47400cbb70 0x7f4740071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.701+0000 7f4747cec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4740156e50 0x7f47400aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.701+0000 7f4747cec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c3890 con 0x7f4740156e50 2023-08-11T15:51:23.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.701+0000 7f4747cec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c2c50 con 0x7f47400cbb70 2023-08-11T15:51:23.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.701+0000 7f4747cec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c2ef0 con 0x7f47400cb0c0 2023-08-11T15:51:23.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4746289640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4740156e50 0x7f47400aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4746289640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4740156e50 0x7f47400aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46216/0 (socket says 172.21.15.27:46216) 2023-08-11T15:51:23.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4746289640 1 -- 172.21.15.27:0/3466204283 learned_addr learned my addr 172.21.15.27:0/3466204283 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:23.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4745287640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47400cbb70 0x7f4740071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4745a88640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47400cb0c0 0x7f47401e3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4746289640 1 -- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47400cb0c0 msgr2=0x7f47401e3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4746289640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47400cb0c0 0x7f47401e3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4746289640 1 -- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47400cbb70 msgr2=0x7f4740071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4746289640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47400cbb70 0x7f4740071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4746289640 1 -- 172.21.15.27:0/3466204283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4738009b30 con 0x7f4740156e50 2023-08-11T15:51:23.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.702+0000 7f4746289640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4740156e50 0x7f47400aea60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4738036040 tx=0x7f4738004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:23.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.703+0000 7f47367fc640 1 -- 172.21.15.27:0/3466204283 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738009510 con 0x7f4740156e50 2023-08-11T15:51:23.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.703+0000 7f47367fc640 1 -- 172.21.15.27:0/3466204283 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473803e450 con 0x7f4740156e50 2023-08-11T15:51:23.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.703+0000 7f47367fc640 1 -- 172.21.15.27:0/3466204283 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473803e7e0 con 0x7f4740156e50 2023-08-11T15:51:23.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.703+0000 7f4747cec640 1 -- 172.21.15.27:0/3466204283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47400c31d0 con 0x7f4740156e50 2023-08-11T15:51:23.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.703+0000 7f4747cec640 1 -- 172.21.15.27:0/3466204283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47401e3c60 con 0x7f4740156e50 2023-08-11T15:51:23.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.704+0000 7f4747cec640 1 -- 172.21.15.27:0/3466204283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4740073990 con 0x7f4740156e50 2023-08-11T15:51:23.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.705+0000 7f47367fc640 1 -- 172.21.15.27:0/3466204283 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f473803e940 con 0x7f4740156e50 2023-08-11T15:51:23.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.706+0000 7f47367fc640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f472406a050 0x7f472406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:23.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.706+0000 7f47367fc640 1 -- 172.21.15.27:0/3466204283 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f47380ab590 con 0x7f4740156e50 2023-08-11T15:51:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.707+0000 7f4745a88640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f472406a050 0x7f472406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.708+0000 7f4745a88640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f472406a050 0x7f472406c510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f473c000fb0 tx=0x7f473c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:23.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.711+0000 7f47367fc640 1 -- 172.21.15.27:0/3466204283 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f4738076980 con 0x7f4740156e50 2023-08-11T15:51:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.914+0000 7f4747cec640 1 -- 172.21.15.27:0/3466204283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47400cb8d0 con 0x7f4740156e50 2023-08-11T15:51:23.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.916+0000 7f47367fc640 1 -- 172.21.15.27:0/3466204283 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7f4738079fa0 con 0x7f4740156e50 2023-08-11T15:51:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:23.918 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:23.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 -- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f472406a050 msgr2=0x7f472406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f472406a050 0x7f472406c510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f473c000fb0 tx=0x7f473c009040 comp rx=0 tx=0).stop 2023-08-11T15:51:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 -- 172.21.15.27:0/3466204283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4740156e50 msgr2=0x7f47400aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4740156e50 0x7f47400aea60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4738036040 tx=0x7f4738004a40 comp rx=0 tx=0).stop 2023-08-11T15:51:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 -- 172.21.15.27:0/3466204283 shutdown_connections 2023-08-11T15:51:23.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f472406a050 0x7f472406c510 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4740156e50 0x7f47400aea60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47400cbb70 0x7f4740071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 --2- 172.21.15.27:0/3466204283 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f47400cb0c0 0x7f47401e3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:23.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 -- 172.21.15.27:0/3466204283 >> 172.21.15.27:0/3466204283 conn(0x7f47400afb70 msgr2=0x7f47401526c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:23.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.919+0000 7f4747cec640 1 -- 172.21.15.27:0/3466204283 shutdown_connections 2023-08-11T15:51:23.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:23.920+0000 7f4747cec640 1 -- 172.21.15.27:0/3466204283 wait complete. 2023-08-11T15:51:23.941 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-11T15:51:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.178+0000 7faa530ec640 1 Processor -- start 2023-08-11T15:51:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.179+0000 7faa530ec640 1 -- start start 2023-08-11T15:51:24.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.181+0000 7faa530ec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faa4c06b670 0x7faa4c0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:24.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.181+0000 7faa530ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa4c0d2850 0x7faa4c0af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.181+0000 7faa530ec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa4c06bc10 con 0x7faa4c15c760 2023-08-11T15:51:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.181+0000 7faa530ec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4c06c790 con 0x7faa4c06b670 2023-08-11T15:51:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.181+0000 7faa530ec640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4c0d13f0 con 0x7faa4c0d2850 2023-08-11T15:51:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.181+0000 7faa50e88640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faa4c15c760 0x7faa4c0d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48970/0 (socket says 172.21.15.27:48970) 2023-08-11T15:51:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.181+0000 7faa50e88640 1 -- 172.21.15.27:0/919294381 learned_addr learned my addr 172.21.15.27:0/919294381 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.181+0000 7faa4bfff640 1 --2- 172.21.15.27:0/919294381 >> v2:172.21.15.27:3300/0 conn(0x7faa4c06b670 0x7faa4c0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.181+0000 7faa51689640 1 --2- 172.21.15.27:0/919294381 >> v2:172.21.15.119:3300/0 conn(0x7faa4c0d2850 0x7faa4c0af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.182+0000 7faa4bfff640 1 -- 172.21.15.27:0/919294381 >> v2:172.21.15.119:3300/0 conn(0x7faa4c0d2850 msgr2=0x7faa4c0af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.182+0000 7faa4bfff640 1 --2- 172.21.15.27:0/919294381 >> v2:172.21.15.119:3300/0 conn(0x7faa4c0d2850 0x7faa4c0af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.182+0000 7faa4bfff640 1 -- 172.21.15.27:0/919294381 >> v1:172.21.15.27:6789/0 conn(0x7faa4c15c760 legacy=0x7faa4c0d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.182+0000 7faa4bfff640 1 -- 172.21.15.27:0/919294381 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c06bef0 con 0x7faa4c06b670 2023-08-11T15:51:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.183+0000 7faa4bfff640 1 --2- 172.21.15.27:0/919294381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c06b670 0x7faa4c0d2310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0c96f0 tx=0x7faa3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19bd4fe750c3e61 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.183+0000 7faa4b7fe640 1 -- 172.21.15.27:0/919294381 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38031830 con 0x7faa4c06b670 2023-08-11T15:51:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.183+0000 7faa4b7fe640 1 -- 172.21.15.27:0/919294381 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa38031990 con 0x7faa4c06b670 2023-08-11T15:51:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.183+0000 7faa4b7fe640 1 -- 172.21.15.27:0/919294381 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38031d20 con 0x7faa4c06b670 2023-08-11T15:51:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.184+0000 7faa530ec640 1 -- 172.21.15.27:0/919294381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c06b670 msgr2=0x7faa4c0d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.184+0000 7faa530ec640 1 --2- 172.21.15.27:0/919294381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c06b670 0x7faa4c0d2310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0c96f0 tx=0x7faa3802b8d0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.184+0000 7faa530ec640 1 -- 172.21.15.27:0/919294381 shutdown_connections 2023-08-11T15:51:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.184+0000 7faa530ec640 1 --2- 172.21.15.27:0/919294381 >> v2:172.21.15.119:3300/0 conn(0x7faa4c0d2850 0x7faa4c0af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.184+0000 7faa530ec640 1 --2- 172.21.15.27:0/919294381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c06b670 0x7faa4c0d2310 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.184+0000 7faa530ec640 1 -- 172.21.15.27:0/919294381 >> 172.21.15.27:0/919294381 conn(0x7faa4c089ff0 msgr2=0x7faa4c09b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.184+0000 7faa530ec640 1 -- 172.21.15.27:0/919294381 shutdown_connections 2023-08-11T15:51:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.184+0000 7faa530ec640 1 -- 172.21.15.27:0/919294381 wait complete. 2023-08-11T15:51:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.187+0000 7faa530ec640 1 Processor -- start 2023-08-11T15:51:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.187+0000 7faa530ec640 1 -- start start 2023-08-11T15:51:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.188+0000 7faa530ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4c06b670 0x7faa4c0cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.188+0000 7faa530ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa4c0d2850 0x7faa4c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.188+0000 7faa530ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c15c760 0x7faa4c0b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.188+0000 7faa530ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c0cdc70 con 0x7faa4c15c760 2023-08-11T15:51:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.188+0000 7faa530ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c0cd060 con 0x7faa4c0d2850 2023-08-11T15:51:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.188+0000 7faa530ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c0cd340 con 0x7faa4c06b670 2023-08-11T15:51:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.188+0000 7faa51689640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c15c760 0x7faa4c0b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa51689640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c15c760 0x7faa4c0b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46228/0 (socket says 172.21.15.27:46228) 2023-08-11T15:51:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa51689640 1 -- 172.21.15.27:0/2511615324 learned_addr learned my addr 172.21.15.27:0/2511615324 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa4bfff640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa4c0d2850 0x7faa4c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa50e88640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4c06b670 0x7faa4c0cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa51689640 1 -- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4c06b670 msgr2=0x7faa4c0cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa51689640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4c06b670 0x7faa4c0cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa51689640 1 -- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa4c0d2850 msgr2=0x7faa4c0b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:24.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa51689640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa4c0d2850 0x7faa4c0b7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa51689640 1 -- 172.21.15.27:0/2511615324 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38009400 con 0x7faa4c15c760 2023-08-11T15:51:24.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa50e88640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4c06b670 0x7faa4c0cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:24.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.189+0000 7faa51689640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c15c760 0x7faa4c0b7db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faa44007e40 tx=0x7faa4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:24.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.190+0000 7faa497fa640 1 -- 172.21.15.27:0/2511615324 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44014620 con 0x7faa4c15c760 2023-08-11T15:51:24.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.190+0000 7faa497fa640 1 -- 172.21.15.27:0/2511615324 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4400ee90 con 0x7faa4c15c760 2023-08-11T15:51:24.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.190+0000 7faa497fa640 1 -- 172.21.15.27:0/2511615324 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa440149e0 con 0x7faa4c15c760 2023-08-11T15:51:24.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.190+0000 7faa530ec640 1 -- 172.21.15.27:0/2511615324 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4c0cd620 con 0x7faa4c15c760 2023-08-11T15:51:24.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.190+0000 7faa530ec640 1 -- 172.21.15.27:0/2511615324 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c0cf300 con 0x7faa4c15c760 2023-08-11T15:51:24.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.191+0000 7faa530ec640 1 -- 172.21.15.27:0/2511615324 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c0cdde0 con 0x7faa4c15c760 2023-08-11T15:51:24.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.192+0000 7faa497fa640 1 -- 172.21.15.27:0/2511615324 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faa44014b40 con 0x7faa4c15c760 2023-08-11T15:51:24.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.193+0000 7faa497fa640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa3006a050 0x7faa3006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:24.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.193+0000 7faa497fa640 1 -- 172.21.15.27:0/2511615324 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faa44089320 con 0x7faa4c15c760 2023-08-11T15:51:24.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.194+0000 7faa50e88640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa3006a050 0x7faa3006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:24.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.195+0000 7faa50e88640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa3006a050 0x7faa3006c510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faa40005880 tx=0x7faa40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:24.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.198+0000 7faa497fa640 1 -- 172.21.15.27:0/2511615324 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7faa44054680 con 0x7faa4c15c760 2023-08-11T15:51:24.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.431+0000 7faa530ec640 1 -- 172.21.15.27:0/2511615324 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7faa4c0cf490 con 0x7faa4c15c760 2023-08-11T15:51:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.836+0000 7faa497fa640 1 -- 172.21.15.27:0/2511615324 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7faa44057ca0 con 0x7faa4c15c760 2023-08-11T15:51:24.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.840+0000 7faa530ec640 1 -- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa3006a050 msgr2=0x7faa3006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:24.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.840+0000 7faa530ec640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa3006a050 0x7faa3006c510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faa40005880 tx=0x7faa40009040 comp rx=0 tx=0).stop 2023-08-11T15:51:24.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.840+0000 7faa530ec640 1 -- 172.21.15.27:0/2511615324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c15c760 msgr2=0x7faa4c0b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:24.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.840+0000 7faa530ec640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c15c760 0x7faa4c0b7db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faa44007e40 tx=0x7faa4400e4b0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.841+0000 7faa530ec640 1 -- 172.21.15.27:0/2511615324 shutdown_connections 2023-08-11T15:51:24.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.841+0000 7faa530ec640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa3006a050 0x7faa3006c510 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.841+0000 7faa530ec640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa4c15c760 0x7faa4c0b7db0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.841+0000 7faa530ec640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa4c0d2850 0x7faa4c0b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.841+0000 7faa530ec640 1 --2- 172.21.15.27:0/2511615324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4c06b670 0x7faa4c0cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:24.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.841+0000 7faa530ec640 1 -- 172.21.15.27:0/2511615324 >> 172.21.15.27:0/2511615324 conn(0x7faa4c089ff0 msgr2=0x7faa4c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:24.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.841+0000 7faa530ec640 1 -- 172.21.15.27:0/2511615324 shutdown_connections 2023-08-11T15:51:24.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:24.841+0000 7faa530ec640 1 -- 172.21.15.27:0/2511615324 wait complete. 2023-08-11T15:51:24.863 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-08-11T15:51:24.863 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-11T15:51:25.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.110+0000 7f76715bf640 1 Processor -- start 2023-08-11T15:51:25.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.110+0000 7f76715bf640 1 -- start start 2023-08-11T15:51:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.112+0000 7f76715bf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f766c142890 0x7f766c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.112+0000 7f76715bf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f766c0b2000 0x7f766c0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.112+0000 7f76715bf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c15a5f0 con 0x7f766c15cc20 2023-08-11T15:51:25.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.112+0000 7f76715bf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c0b29c0 con 0x7f766c0b2000 2023-08-11T15:51:25.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.112+0000 7f76715bf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c0b0ff0 con 0x7f766c142890 2023-08-11T15:51:25.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.113+0000 7f7670dbe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f766c15cc20 0x7f766c15d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48972/0 (socket says 172.21.15.27:48972) 2023-08-11T15:51:25.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.113+0000 7f7670dbe640 1 -- 172.21.15.27:0/1460732046 learned_addr learned my addr 172.21.15.27:0/1460732046 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:25.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.113+0000 7f766b7fe640 1 --2- 172.21.15.27:0/1460732046 >> v2:172.21.15.27:3300/0 conn(0x7f766c0b2000 0x7f766c0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.113+0000 7f766bfff640 1 --2- 172.21.15.27:0/1460732046 >> v2:172.21.15.119:3300/0 conn(0x7f766c142890 0x7f766c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.113+0000 7f766b7fe640 1 -- 172.21.15.27:0/1460732046 >> v2:172.21.15.119:3300/0 conn(0x7f766c142890 msgr2=0x7f766c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:25.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.113+0000 7f766b7fe640 1 --2- 172.21.15.27:0/1460732046 >> v2:172.21.15.119:3300/0 conn(0x7f766c142890 0x7f766c15a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.113+0000 7f766b7fe640 1 -- 172.21.15.27:0/1460732046 >> v1:172.21.15.27:6789/0 conn(0x7f766c15cc20 legacy=0x7f766c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.113+0000 7f766b7fe640 1 -- 172.21.15.27:0/1460732046 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c0b12d0 con 0x7f766c0b2000 2023-08-11T15:51:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.113+0000 7f766bfff640 1 --2- 172.21.15.27:0/1460732046 >> v2:172.21.15.119:3300/0 conn(0x7f766c142890 0x7f766c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.113+0000 7f766affd640 1 -- 172.21.15.27:0/1460732046 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030608141 0 0) 0x7f766c15a5f0 con 0x7f766c15cc20 2023-08-11T15:51:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.115+0000 7f766b7fe640 1 --2- 172.21.15.27:0/1460732046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c0b2000 0x7f766c0b2480 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f766c0b33d0 tx=0x7f766002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eec780ef1cbc94d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.115+0000 7f766affd640 1 -- 172.21.15.27:0/1460732046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660031830 con 0x7f766c0b2000 2023-08-11T15:51:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.115+0000 7f766affd640 1 -- 172.21.15.27:0/1460732046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660031990 con 0x7f766c0b2000 2023-08-11T15:51:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.115+0000 7f766affd640 1 -- 172.21.15.27:0/1460732046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660031d20 con 0x7f766c0b2000 2023-08-11T15:51:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.116+0000 7f76715bf640 1 -- 172.21.15.27:0/1460732046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c0b2000 msgr2=0x7f766c0b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.116+0000 7f76715bf640 1 --2- 172.21.15.27:0/1460732046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c0b2000 0x7f766c0b2480 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f766c0b33d0 tx=0x7f766002b8d0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.116+0000 7f76715bf640 1 -- 172.21.15.27:0/1460732046 shutdown_connections 2023-08-11T15:51:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.116+0000 7f76715bf640 1 --2- 172.21.15.27:0/1460732046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c0b2000 0x7f766c0b2480 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.116+0000 7f76715bf640 1 --2- 172.21.15.27:0/1460732046 >> v2:172.21.15.119:3300/0 conn(0x7f766c142890 0x7f766c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.116+0000 7f76715bf640 1 -- 172.21.15.27:0/1460732046 >> 172.21.15.27:0/1460732046 conn(0x7f766c069800 msgr2=0x7f766c069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.116+0000 7f76715bf640 1 -- 172.21.15.27:0/1460732046 shutdown_connections 2023-08-11T15:51:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.116+0000 7f76715bf640 1 -- 172.21.15.27:0/1460732046 wait complete. 2023-08-11T15:51:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.118+0000 7f76715bf640 1 Processor -- start 2023-08-11T15:51:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.118+0000 7f76715bf640 1 -- start start 2023-08-11T15:51:25.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f76715bf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f766c0b2000 0x7f766c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f76715bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c142890 0x7f766c157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f76715bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f766c15cc20 0x7f766c139030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f76715bf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c138650 con 0x7f766c142890 2023-08-11T15:51:25.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f76715bf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c138370 con 0x7f766c15cc20 2023-08-11T15:51:25.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f76715bf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c157410 con 0x7f766c0b2000 2023-08-11T15:51:25.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f766b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c142890 0x7f766c157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f766b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c142890 0x7f766c157ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46250/0 (socket says 172.21.15.27:46250) 2023-08-11T15:51:25.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f766b7fe640 1 -- 172.21.15.27:0/2081347268 learned_addr learned my addr 172.21.15.27:0/2081347268 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f766bfff640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f766c0b2000 0x7f766c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.119+0000 7f7670dbe640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f766c15cc20 0x7f766c139030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.120+0000 7f766b7fe640 1 -- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f766c0b2000 msgr2=0x7f766c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.120+0000 7f766b7fe640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f766c0b2000 0x7f766c15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.120+0000 7f766b7fe640 1 -- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f766c15cc20 msgr2=0x7f766c139030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.120+0000 7f766b7fe640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f766c15cc20 0x7f766c139030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.120+0000 7f766b7fe640 1 -- 172.21.15.27:0/2081347268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7660009400 con 0x7f766c142890 2023-08-11T15:51:25.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.120+0000 7f766b7fe640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c142890 0x7f766c157ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7660000c00 tx=0x7f76600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:25.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.120+0000 7f7668ff9640 1 -- 172.21.15.27:0/2081347268 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660004230 con 0x7f766c142890 2023-08-11T15:51:25.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.120+0000 7f7668ff9640 1 -- 172.21.15.27:0/2081347268 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76600043b0 con 0x7f766c142890 2023-08-11T15:51:25.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.120+0000 7f7668ff9640 1 -- 172.21.15.27:0/2081347268 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660004520 con 0x7f766c142890 2023-08-11T15:51:25.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.120+0000 7f76715bf640 1 -- 172.21.15.27:0/2081347268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c1576f0 con 0x7f766c142890 2023-08-11T15:51:25.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.121+0000 7f76715bf640 1 -- 172.21.15.27:0/2081347268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c138000 con 0x7f766c142890 2023-08-11T15:51:25.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.122+0000 7f76715bf640 1 -- 172.21.15.27:0/2081347268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766c066e80 con 0x7f766c142890 2023-08-11T15:51:25.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.123+0000 7f7668ff9640 1 -- 172.21.15.27:0/2081347268 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f766003a8a0 con 0x7f766c142890 2023-08-11T15:51:25.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.124+0000 7f7668ff9640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f765806a080 0x7f765806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.124+0000 7f7668ff9640 1 -- 172.21.15.27:0/2081347268 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f76600aaf40 con 0x7f766c142890 2023-08-11T15:51:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.125+0000 7f766bfff640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f765806a080 0x7f765806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.126+0000 7f766bfff640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f765806a080 0x7f765806c540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f765c006fd0 tx=0x7f765c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:25.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.129+0000 7f7668ff9640 1 -- 172.21.15.27:0/2081347268 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f76600762a0 con 0x7f766c142890 2023-08-11T15:51:25.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.357+0000 7f76715bf640 1 -- 172.21.15.27:0/2081347268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f766c0af690 con 0x7f766c142890 2023-08-11T15:51:25.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.839+0000 7f7668ff9640 1 -- 172.21.15.27:0/2081347268 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f76600350c0 con 0x7f766c142890 2023-08-11T15:51:25.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 -- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f765806a080 msgr2=0x7f765806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f765806a080 0x7f765806c540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f765c006fd0 tx=0x7f765c008040 comp rx=0 tx=0).stop 2023-08-11T15:51:25.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 -- 172.21.15.27:0/2081347268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c142890 msgr2=0x7f766c157ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c142890 0x7f766c157ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7660000c00 tx=0x7f76600046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 -- 172.21.15.27:0/2081347268 shutdown_connections 2023-08-11T15:51:25.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f765806a080 0x7f765806c540 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f766c15cc20 0x7f766c139030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f766c142890 0x7f766c157ed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 --2- 172.21.15.27:0/2081347268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f766c0b2000 0x7f766c15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 -- 172.21.15.27:0/2081347268 >> 172.21.15.27:0/2081347268 conn(0x7f766c069800 msgr2=0x7f766c15b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 -- 172.21.15.27:0/2081347268 shutdown_connections 2023-08-11T15:51:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:25.843+0000 7f76715bf640 1 -- 172.21.15.27:0/2081347268 wait complete. 2023-08-11T15:51:25.868 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-11T15:51:25.868 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:51:26.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.110+0000 7f3164bf5640 1 Processor -- start 2023-08-11T15:51:26.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.111+0000 7f3164bf5640 1 -- start start 2023-08-11T15:51:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.112+0000 7f3164bf5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f316015c1f0 0x7f31600b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.112+0000 7f3164bf5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f316015c5d0 0x7f316015ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f3164bf5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31600b5170 con 0x7f31600d04b0 2023-08-11T15:51:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f3164bf5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316015d080 con 0x7f316015c5d0 2023-08-11T15:51:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f3164bf5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316015d360 con 0x7f316015c1f0 2023-08-11T15:51:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f3157fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f316015c5d0 0x7f316015ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f315ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f31600d04b0 0x7f3160160510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48974/0 (socket says 172.21.15.27:48974) 2023-08-11T15:51:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f315ffff640 1 -- 172.21.15.27:0/3578981222 learned_addr learned my addr 172.21.15.27:0/3578981222 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f315f7fe640 1 --2- 172.21.15.27:0/3578981222 >> v2:172.21.15.119:3300/0 conn(0x7f316015c1f0 0x7f31600b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f3157fff640 1 -- 172.21.15.27:0/3578981222 >> v2:172.21.15.119:3300/0 conn(0x7f316015c1f0 msgr2=0x7f31600b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:26.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f3157fff640 1 --2- 172.21.15.27:0/3578981222 >> v2:172.21.15.119:3300/0 conn(0x7f316015c1f0 0x7f31600b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f3157fff640 1 -- 172.21.15.27:0/3578981222 >> v1:172.21.15.27:6789/0 conn(0x7f31600d04b0 legacy=0x7f3160160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f3157fff640 1 -- 172.21.15.27:0/3578981222 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316015d640 con 0x7f316015c5d0 2023-08-11T15:51:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.113+0000 7f315effd640 1 -- 172.21.15.27:0/3578981222 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534706076 0 0) 0x7f31600b5170 con 0x7f31600d04b0 2023-08-11T15:51:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.115+0000 7f3157fff640 1 --2- 172.21.15.27:0/3578981222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 0x7f316015ca30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f316008b310 tx=0x7f314802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db04347676133ee6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.115+0000 7f315effd640 1 -- 172.21.15.27:0/3578981222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148030610 con 0x7f316015c5d0 2023-08-11T15:51:26.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.115+0000 7f315effd640 1 -- 172.21.15.27:0/3578981222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3148030770 con 0x7f316015c5d0 2023-08-11T15:51:26.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.115+0000 7f315effd640 1 -- 172.21.15.27:0/3578981222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148030b00 con 0x7f316015c5d0 2023-08-11T15:51:26.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.115+0000 7f3164bf5640 1 -- 172.21.15.27:0/3578981222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 msgr2=0x7f316015ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.115+0000 7f3164bf5640 1 --2- 172.21.15.27:0/3578981222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 0x7f316015ca30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f316008b310 tx=0x7f314802b960 comp rx=0 tx=0).stop 2023-08-11T15:51:26.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.116+0000 7f3164bf5640 1 -- 172.21.15.27:0/3578981222 shutdown_connections 2023-08-11T15:51:26.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.116+0000 7f3164bf5640 1 --2- 172.21.15.27:0/3578981222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 0x7f316015ca30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.116+0000 7f3164bf5640 1 --2- 172.21.15.27:0/3578981222 >> v2:172.21.15.119:3300/0 conn(0x7f316015c1f0 0x7f31600b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.116+0000 7f3164bf5640 1 -- 172.21.15.27:0/3578981222 >> 172.21.15.27:0/3578981222 conn(0x7f316006a250 msgr2=0x7f316006a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:26.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.116+0000 7f3164bf5640 1 -- 172.21.15.27:0/3578981222 shutdown_connections 2023-08-11T15:51:26.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.116+0000 7f3164bf5640 1 -- 172.21.15.27:0/3578981222 wait complete. 2023-08-11T15:51:26.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.117+0000 7f3164bf5640 1 Processor -- start 2023-08-11T15:51:26.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.118+0000 7f3164bf5640 1 -- start start 2023-08-11T15:51:26.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f3164bf5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f31600d04b0 0x7f316015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f3164bf5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f316015c1f0 0x7f316015e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f3164bf5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 0x7f31600b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f3164bf5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600b6510 con 0x7f316015c5d0 2023-08-11T15:51:26.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f3164bf5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316015ddc0 con 0x7f316015c1f0 2023-08-11T15:51:26.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f3164bf5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3160157960 con 0x7f31600d04b0 2023-08-11T15:51:26.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f315ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 0x7f31600b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f315ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 0x7f31600b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46274/0 (socket says 172.21.15.27:46274) 2023-08-11T15:51:26.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f315ffff640 1 -- 172.21.15.27:0/1719724730 learned_addr learned my addr 172.21.15.27:0/1719724730 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:26.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f315f7fe640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f31600d04b0 0x7f316015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f3157fff640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f316015c1f0 0x7f316015e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f315ffff640 1 -- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f31600d04b0 msgr2=0x7f316015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f315ffff640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f31600d04b0 0x7f316015d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.119+0000 7f315ffff640 1 -- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f316015c1f0 msgr2=0x7f316015e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.120+0000 7f315ffff640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f316015c1f0 0x7f316015e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.120+0000 7f315ffff640 1 -- 172.21.15.27:0/1719724730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3148009400 con 0x7f316015c5d0 2023-08-11T15:51:26.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.120+0000 7f315ffff640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 0x7f31600b6930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f315800c7e0 tx=0x7f315800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:26.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.120+0000 7f315cff9640 1 -- 172.21.15.27:0/1719724730 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315800a5e0 con 0x7f316015c5d0 2023-08-11T15:51:26.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.120+0000 7f315cff9640 1 -- 172.21.15.27:0/1719724730 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315800a760 con 0x7f316015c5d0 2023-08-11T15:51:26.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.120+0000 7f3164bf5640 1 -- 172.21.15.27:0/1719724730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316015ef20 con 0x7f316015c5d0 2023-08-11T15:51:26.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.120+0000 7f315cff9640 1 -- 172.21.15.27:0/1719724730 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315800ab10 con 0x7f316015c5d0 2023-08-11T15:51:26.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.120+0000 7f3164bf5640 1 -- 172.21.15.27:0/1719724730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3160157730 con 0x7f316015c5d0 2023-08-11T15:51:26.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.122+0000 7f3164bf5640 1 -- 172.21.15.27:0/1719724730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31600b6650 con 0x7f316015c5d0 2023-08-11T15:51:26.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.122+0000 7f315cff9640 1 -- 172.21.15.27:0/1719724730 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f315800ac70 con 0x7f316015c5d0 2023-08-11T15:51:26.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.124+0000 7f315cff9640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f314006a050 0x7f314006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.124+0000 7f315cff9640 1 -- 172.21.15.27:0/1719724730 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3158088080 con 0x7f316015c5d0 2023-08-11T15:51:26.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.125+0000 7f315f7fe640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f314006a050 0x7f314006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.126+0000 7f315f7fe640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f314006a050 0x7f314006c510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3150006fd0 tx=0x7f3150008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:26.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.129+0000 7f315cff9640 1 -- 172.21.15.27:0/1719724730 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f31580ba7c0 con 0x7f316015c5d0 2023-08-11T15:51:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.359+0000 7f3164bf5640 1 -- 172.21.15.27:0/1719724730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f316004e420 con 0x7f316015c5d0 2023-08-11T15:51:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.361+0000 7f315cff9640 1 -- 172.21.15.27:0/1719724730 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f3158018020 con 0x7f316015c5d0 2023-08-11T15:51:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:26.363 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:51:01.220970+0000","modified":"2023-08-11T15:51:25.832166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4315":{"gid":4315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:51:26.363 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2023-08-11T15:51:26.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 -- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f314006a050 msgr2=0x7f314006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f314006a050 0x7f314006c510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3150006fd0 tx=0x7f3150008040 comp rx=0 tx=0).stop 2023-08-11T15:51:26.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 -- 172.21.15.27:0/1719724730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 msgr2=0x7f31600b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 0x7f31600b6930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f315800c7e0 tx=0x7f315800cca0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 -- 172.21.15.27:0/1719724730 shutdown_connections 2023-08-11T15:51:26.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f314006a050 0x7f314006c510 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316015c5d0 0x7f31600b6930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f316015c1f0 0x7f316015e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 --2- 172.21.15.27:0/1719724730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f31600d04b0 0x7f316015d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 -- 172.21.15.27:0/1719724730 >> 172.21.15.27:0/1719724730 conn(0x7f316006a250 msgr2=0x7f31600ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:26.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.363+0000 7f3164bf5640 1 -- 172.21.15.27:0/1719724730 shutdown_connections 2023-08-11T15:51:26.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.364+0000 7f3164bf5640 1 -- 172.21.15.27:0/1719724730 wait complete. 2023-08-11T15:51:26.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:26.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.608+0000 7f6de9399640 1 Processor -- start 2023-08-11T15:51:26.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.609+0000 7f6de9399640 1 -- start start 2023-08-11T15:51:26.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.610+0000 7f6de9399640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6de415c620 0x7f6de40b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.610+0000 7f6de9399640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6de40b82f0 0x7f6de40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.610+0000 7f6de9399640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de40cda30 con 0x7f6de40b55c0 2023-08-11T15:51:26.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.611+0000 7f6de9399640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de40ccc10 con 0x7f6de415c620 2023-08-11T15:51:26.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.611+0000 7f6de9399640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de4161110 con 0x7f6de40b82f0 2023-08-11T15:51:26.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.611+0000 7f6de2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6de415c620 0x7f6de40b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.611+0000 7f6de37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6de40b55c0 0x7f6de40b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48976/0 (socket says 172.21.15.27:48976) 2023-08-11T15:51:26.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.611+0000 7f6de37fe640 1 -- 172.21.15.27:0/1122636863 learned_addr learned my addr 172.21.15.27:0/1122636863 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:26.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.611+0000 7f6de27fc640 1 --2- 172.21.15.27:0/1122636863 >> v2:172.21.15.119:3300/0 conn(0x7f6de40b82f0 0x7f6de40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.611+0000 7f6de2ffd640 1 -- 172.21.15.27:0/1122636863 >> v2:172.21.15.119:3300/0 conn(0x7f6de40b82f0 msgr2=0x7f6de40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:26.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.611+0000 7f6de2ffd640 1 --2- 172.21.15.27:0/1122636863 >> v2:172.21.15.119:3300/0 conn(0x7f6de40b82f0 0x7f6de40cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.611+0000 7f6de2ffd640 1 -- 172.21.15.27:0/1122636863 >> v1:172.21.15.27:6789/0 conn(0x7f6de40b55c0 legacy=0x7f6de40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.612+0000 7f6de2ffd640 1 -- 172.21.15.27:0/1122636863 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de40cce20 con 0x7f6de415c620 2023-08-11T15:51:26.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.612+0000 7f6de27fc640 1 --2- 172.21.15.27:0/1122636863 >> v2:172.21.15.119:3300/0 conn(0x7f6de40b82f0 0x7f6de40cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:26.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.613+0000 7f6de2ffd640 1 --2- 172.21.15.27:0/1122636863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 0x7f6de40b7db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6de40b50e0 tx=0x7f6dd402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8995f55c81e99ae9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:26.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.613+0000 7f6de1ffb640 1 -- 172.21.15.27:0/1122636863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd402cc80 con 0x7f6de415c620 2023-08-11T15:51:26.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.613+0000 7f6de1ffb640 1 -- 172.21.15.27:0/1122636863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd402cde0 con 0x7f6de415c620 2023-08-11T15:51:26.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.613+0000 7f6de1ffb640 1 -- 172.21.15.27:0/1122636863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4034660 con 0x7f6de415c620 2023-08-11T15:51:26.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.614+0000 7f6de9399640 1 -- 172.21.15.27:0/1122636863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 msgr2=0x7f6de40b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.614+0000 7f6de9399640 1 --2- 172.21.15.27:0/1122636863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 0x7f6de40b7db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6de40b50e0 tx=0x7f6dd402bed0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.614+0000 7f6de9399640 1 -- 172.21.15.27:0/1122636863 shutdown_connections 2023-08-11T15:51:26.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.614+0000 7f6de9399640 1 --2- 172.21.15.27:0/1122636863 >> v2:172.21.15.119:3300/0 conn(0x7f6de40b82f0 0x7f6de40cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.614+0000 7f6de9399640 1 --2- 172.21.15.27:0/1122636863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 0x7f6de40b7db0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.614+0000 7f6de9399640 1 -- 172.21.15.27:0/1122636863 >> 172.21.15.27:0/1122636863 conn(0x7f6de4071110 msgr2=0x7f6de40714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:26.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.614+0000 7f6de9399640 1 -- 172.21.15.27:0/1122636863 shutdown_connections 2023-08-11T15:51:26.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.614+0000 7f6de9399640 1 -- 172.21.15.27:0/1122636863 wait complete. 2023-08-11T15:51:26.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.616+0000 7f6de9399640 1 Processor -- start 2023-08-11T15:51:26.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.616+0000 7f6de9399640 1 -- start start 2023-08-11T15:51:26.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.617+0000 7f6de9399640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6de40b55c0 0x7f6de41f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.617+0000 7f6de9399640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6de40b82f0 0x7f6de40cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.617+0000 7f6de9399640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 0x7f6de40b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.617+0000 7f6de9399640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40ca330 con 0x7f6de415c620 2023-08-11T15:51:26.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.617+0000 7f6de9399640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40ca610 con 0x7f6de40b55c0 2023-08-11T15:51:26.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.617+0000 7f6de9399640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40ca8f0 con 0x7f6de40b82f0 2023-08-11T15:51:26.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.618+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 0x7f6de40b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.618+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 0x7f6de40b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46294/0 (socket says 172.21.15.27:46294) 2023-08-11T15:51:26.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.618+0000 7f6de37fe640 1 -- 172.21.15.27:0/89130119 learned_addr learned my addr 172.21.15.27:0/89130119 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.618+0000 7f6de2ffd640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6de40b55c0 0x7f6de41f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.618+0000 7f6de27fc640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6de40b82f0 0x7f6de40cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.618+0000 7f6de37fe640 1 -- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6de40b82f0 msgr2=0x7f6de40cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.618+0000 7f6de37fe640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6de40b82f0 0x7f6de40cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.618+0000 7f6de37fe640 1 -- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6de40b55c0 msgr2=0x7f6de41f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.618+0000 7f6de37fe640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6de40b55c0 0x7f6de41f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.618+0000 7f6de37fe640 1 -- 172.21.15.27:0/89130119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd4009a70 con 0x7f6de415c620 2023-08-11T15:51:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.619+0000 7f6de37fe640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 0x7f6de40b4230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c340 tx=0x7f6ddc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:26.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.619+0000 7f6ddb7fe640 1 -- 172.21.15.27:0/89130119 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc005ce0 con 0x7f6de415c620 2023-08-11T15:51:26.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.619+0000 7f6ddb7fe640 1 -- 172.21.15.27:0/89130119 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc005e40 con 0x7f6de415c620 2023-08-11T15:51:26.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.619+0000 7f6ddb7fe640 1 -- 172.21.15.27:0/89130119 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0072d0 con 0x7f6de415c620 2023-08-11T15:51:26.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.619+0000 7f6de9399640 1 -- 172.21.15.27:0/89130119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de40cabd0 con 0x7f6de415c620 2023-08-11T15:51:26.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.619+0000 7f6de9399640 1 -- 172.21.15.27:0/89130119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de40cb910 con 0x7f6de415c620 2023-08-11T15:51:26.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.620+0000 7f6de9399640 1 -- 172.21.15.27:0/89130119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de4073990 con 0x7f6de415c620 2023-08-11T15:51:26.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.621+0000 7f6ddb7fe640 1 -- 172.21.15.27:0/89130119 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6ddc0074f0 con 0x7f6de415c620 2023-08-11T15:51:26.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.622+0000 7f6ddb7fe640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6dc406a050 0x7f6dc406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:26.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.622+0000 7f6ddb7fe640 1 -- 172.21.15.27:0/89130119 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6ddc087c00 con 0x7f6de415c620 2023-08-11T15:51:26.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.623+0000 7f6de2ffd640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6dc406a050 0x7f6dc406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:26.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.624+0000 7f6de2ffd640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6dc406a050 0x7f6dc406c510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4036000 tx=0x7f6dd4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:26.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.626+0000 7f6ddb7fe640 1 -- 172.21.15.27:0/89130119 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6ddc0530a0 con 0x7f6de415c620 2023-08-11T15:51:26.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.825+0000 7f6de9399640 1 -- 172.21.15.27:0/89130119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6de406b6f0 con 0x7f6de415c620 2023-08-11T15:51:26.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.828+0000 7f6ddb7fe640 1 -- 172.21.15.27:0/89130119 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7f6ddc0566c0 con 0x7f6de415c620 2023-08-11T15:51:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:26.830 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:26.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 -- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6dc406a050 msgr2=0x7f6dc406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6dc406a050 0x7f6dc406c510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4036000 tx=0x7f6dd4030000 comp rx=0 tx=0).stop 2023-08-11T15:51:26.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 -- 172.21.15.27:0/89130119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 msgr2=0x7f6de40b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 0x7f6de40b4230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c340 tx=0x7f6ddc00c800 comp rx=0 tx=0).stop 2023-08-11T15:51:26.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 -- 172.21.15.27:0/89130119 shutdown_connections 2023-08-11T15:51:26.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6dc406a050 0x7f6dc406c510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6de40b82f0 0x7f6de40cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6de415c620 0x7f6de40b4230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 --2- 172.21.15.27:0/89130119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6de40b55c0 0x7f6de41f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 -- 172.21.15.27:0/89130119 >> 172.21.15.27:0/89130119 conn(0x7f6de4071110 msgr2=0x7f6de40b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:26.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 -- 172.21.15.27:0/89130119 shutdown_connections 2023-08-11T15:51:26.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:26.831+0000 7f6de9399640 1 -- 172.21.15.27:0/89130119 wait complete. 2023-08-11T15:51:26.851 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-11T15:51:26.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:51:27.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.080+0000 7f2aa274f640 1 Processor -- start 2023-08-11T15:51:27.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.080+0000 7f2aa274f640 1 -- start start 2023-08-11T15:51:27.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.082+0000 7f2aa274f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2a9c15a140 0x7f2a9c06b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.082+0000 7f2aa274f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a9c06f5d0 0x7f2a9c06fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.082+0000 7f2aa274f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9c0c8a30 con 0x7f2a9c0cb430 2023-08-11T15:51:27.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.082+0000 7f2aa274f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c06ffa0 con 0x7f2a9c15a140 2023-08-11T15:51:27.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.082+0000 7f2aa274f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0b1580 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.082+0000 7f2a9bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2a9c15a140 0x7f2a9c06b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.083+0000 7f2aa0cec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a9c06f5d0 0x7f2a9c06fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.083+0000 7f2a9bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2a9c15a140 0x7f2a9c06b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35874/0 (socket says 172.21.15.27:35874) 2023-08-11T15:51:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.083+0000 7f2a9bfff640 1 -- 172.21.15.27:0/339996518 learned_addr learned my addr 172.21.15.27:0/339996518 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:27.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.083+0000 7f2a93fff640 1 --1- 172.21.15.27:0/339996518 >> v1:172.21.15.119:6789/0 conn(0x7f2a9c0cb430 0x7f2a9c0b0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:55770/0 (socket says 172.21.15.27:55770) 2023-08-11T15:51:27.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.083+0000 7f2a9bfff640 1 -- 172.21.15.27:0/339996518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a9c06f5d0 msgr2=0x7f2a9c06fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:27.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.083+0000 7f2a9bfff640 1 --2- 172.21.15.27:0/339996518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a9c06f5d0 0x7f2a9c06fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.083+0000 7f2a9bfff640 1 -- 172.21.15.27:0/339996518 >> v1:172.21.15.119:6789/0 conn(0x7f2a9c0cb430 legacy=0x7f2a9c0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.084+0000 7f2a9b7fe640 1 -- 172.21.15.27:0/339996518 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416573780 0 0) 0x7f2a9c0c8a30 con 0x7f2a9c0cb430 2023-08-11T15:51:27.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.084+0000 7f2a9bfff640 1 -- 172.21.15.27:0/339996518 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c06c0c0 con 0x7f2a9c15a140 2023-08-11T15:51:27.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.085+0000 7f2a9bfff640 1 --2- 172.21.15.27:0/339996518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a9c15a140 0x7f2a9c06b870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009880 tx=0x7f2a8c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c6ab3bd30d3009d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:27.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.086+0000 7f2a9b7fe640 1 -- 172.21.15.27:0/339996518 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c004230 con 0x7f2a9c15a140 2023-08-11T15:51:27.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.086+0000 7f2a9b7fe640 1 -- 172.21.15.27:0/339996518 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c004390 con 0x7f2a9c15a140 2023-08-11T15:51:27.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.086+0000 7f2a9b7fe640 1 -- 172.21.15.27:0/339996518 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c003810 con 0x7f2a9c15a140 2023-08-11T15:51:27.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.087+0000 7f2aa274f640 1 -- 172.21.15.27:0/339996518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a9c15a140 msgr2=0x7f2a9c06b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.087+0000 7f2aa274f640 1 --2- 172.21.15.27:0/339996518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a9c15a140 0x7f2a9c06b870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009880 tx=0x7f2a8c02b960 comp rx=0 tx=0).stop 2023-08-11T15:51:27.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.087+0000 7f2aa274f640 1 -- 172.21.15.27:0/339996518 shutdown_connections 2023-08-11T15:51:27.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.087+0000 7f2aa274f640 1 --2- 172.21.15.27:0/339996518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a9c06f5d0 0x7f2a9c06fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.087+0000 7f2aa274f640 1 --2- 172.21.15.27:0/339996518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a9c15a140 0x7f2a9c06b870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.087+0000 7f2aa274f640 1 -- 172.21.15.27:0/339996518 >> 172.21.15.27:0/339996518 conn(0x7f2a9c0afb70 msgr2=0x7f2a9c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:27.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.087+0000 7f2aa274f640 1 -- 172.21.15.27:0/339996518 shutdown_connections 2023-08-11T15:51:27.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.087+0000 7f2aa274f640 1 -- 172.21.15.27:0/339996518 wait complete. 2023-08-11T15:51:27.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.089+0000 7f2aa274f640 1 Processor -- start 2023-08-11T15:51:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.089+0000 7f2aa274f640 1 -- start start 2023-08-11T15:51:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.090+0000 7f2aa274f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a9c06f5d0 0x7f2a9c151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.090+0000 7f2aa274f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a9c0cb430 0x7f2a9c151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2aa274f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a9c15a140 0x7f2a9c151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2aa274f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0c5070 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2aa274f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c153040 con 0x7f2a9c0cb430 2023-08-11T15:51:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2aa274f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c153290 con 0x7f2a9c15a140 2023-08-11T15:51:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a9c06f5d0 0x7f2a9c151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a9c06f5d0 0x7f2a9c151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46308/0 (socket says 172.21.15.27:46308) 2023-08-11T15:51:27.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2a9bfff640 1 -- 172.21.15.27:0/4235967745 learned_addr learned my addr 172.21.15.27:0/4235967745 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:27.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2a93fff640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a9c0cb430 0x7f2a9c151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2aa0cec640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a9c15a140 0x7f2a9c151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2a9bfff640 1 -- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a9c15a140 msgr2=0x7f2a9c151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2a9bfff640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a9c15a140 0x7f2a9c151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2a9bfff640 1 -- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a9c0cb430 msgr2=0x7f2a9c151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.091+0000 7f2a9bfff640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a9c0cb430 0x7f2a9c151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.092+0000 7f2a9bfff640 1 -- 172.21.15.27:0/4235967745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c009460 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.092+0000 7f2a9bfff640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a9c06f5d0 0x7f2a9c151020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000c00 tx=0x7f2a8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:27.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.092+0000 7f2a997fa640 1 -- 172.21.15.27:0/4235967745 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c03d070 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.092+0000 7f2a997fa640 1 -- 172.21.15.27:0/4235967745 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c004500 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.092+0000 7f2a997fa640 1 -- 172.21.15.27:0/4235967745 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c003e10 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.092+0000 7f2aa274f640 1 -- 172.21.15.27:0/4235967745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c153570 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.093+0000 7f2aa274f640 1 -- 172.21.15.27:0/4235967745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c153b30 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.094+0000 7f2aa274f640 1 -- 172.21.15.27:0/4235967745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c073990 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.095+0000 7f2a997fa640 1 -- 172.21.15.27:0/4235967745 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2a8c006040 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.096+0000 7f2a997fa640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2a7c06a050 0x7f2a7c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.096+0000 7f2a997fa640 1 -- 172.21.15.27:0/4235967745 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2a8c0aa540 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.096+0000 7f2a93fff640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2a7c06a050 0x7f2a7c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.097+0000 7f2a93fff640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2a7c06a050 0x7f2a7c06c510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0cd0e0 tx=0x7f2a84007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:27.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.101+0000 7f2a997fa640 1 -- 172.21.15.27:0/4235967745 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2a8c075af0 con 0x7f2a9c06f5d0 2023-08-11T15:51:27.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.292+0000 7f2aa274f640 1 -- 172.21.15.27:0/4235967745 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a9c06c680 con 0x7f2a7c06a050 2023-08-11T15:51:27.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.326+0000 7f2a997fa640 1 -- 172.21.15.27:0/4235967745 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367129 (secure 0 0 0) 0x7f2a84007170 con 0x7f2a7c06a050 2023-08-11T15:51:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:27.333 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:51:27.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 -- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2a7c06a050 msgr2=0x7f2a7c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2a7c06a050 0x7f2a7c06c510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0cd0e0 tx=0x7f2a84007b50 comp rx=0 tx=0).stop 2023-08-11T15:51:27.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 -- 172.21.15.27:0/4235967745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a9c06f5d0 msgr2=0x7f2a9c151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a9c06f5d0 0x7f2a9c151020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000c00 tx=0x7f2a8c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 -- 172.21.15.27:0/4235967745 shutdown_connections 2023-08-11T15:51:27.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2a7c06a050 0x7f2a7c06c510 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a9c15a140 0x7f2a9c151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a9c0cb430 0x7f2a9c151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 --2- 172.21.15.27:0/4235967745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a9c06f5d0 0x7f2a9c151020 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 -- 172.21.15.27:0/4235967745 >> 172.21.15.27:0/4235967745 conn(0x7f2a9c0afb70 msgr2=0x7f2a9c1590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:27.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.334+0000 7f2aa274f640 1 -- 172.21.15.27:0/4235967745 shutdown_connections 2023-08-11T15:51:27.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.335+0000 7f2aa274f640 1 -- 172.21.15.27:0/4235967745 wait complete. 2023-08-11T15:51:27.358 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-08-11T15:51:26.719348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232028,"kb_used_data":3560,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754742692,"statfs":{"total":773094113280,"available":772856516608,"internally_reserved":0,"allocated":3645440,"data_stored":2175300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005024"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449971+0000","last_change":"2023-08-11T15:51:05.843998+0000","last_active":"2023-08-11T15:51:07.449971+0000","last_peered":"2023-08-11T15:51:07.449971+0000","last_clean":"2023-08-11T15:51:07.449971+0000","last_became_active":"2023-08-11T15:51:05.843613+0000","last_became_peered":"2023-08-11T15:51:05.843613+0000","last_unstale":"2023-08-11T15:51:07.449971+0000","last_undegraded":"2023-08-11T15:51:07.449971+0000","last_fullsized":"2023-08-11T15:51:07.449971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:35:45.480000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842354+0000","last_change":"2023-08-11T15:51:00.746010+0000","last_active":"2023-08-11T15:51:06.842354+0000","last_peered":"2023-08-11T15:51:06.842354+0000","last_clean":"2023-08-11T15:51:06.842354+0000","last_became_active":"2023-08-11T15:51:00.745732+0000","last_became_peered":"2023-08-11T15:51:00.745732+0000","last_unstale":"2023-08-11T15:51:06.842354+0000","last_undegraded":"2023-08-11T15:51:06.842354+0000","last_fullsized":"2023-08-11T15:51:06.842354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:02.960249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028896+0000","last_change":"2023-08-11T15:51:01.746812+0000","last_active":"2023-08-11T15:51:08.028896+0000","last_peered":"2023-08-11T15:51:08.028896+0000","last_clean":"2023-08-11T15:51:08.028896+0000","last_became_active":"2023-08-11T15:51:01.745729+0000","last_became_peered":"2023-08-11T15:51:01.745729+0000","last_unstale":"2023-08-11T15:51:08.028896+0000","last_undegraded":"2023-08-11T15:51:08.028896+0000","last_fullsized":"2023-08-11T15:51:08.028896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:46.849896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028532+0000","last_change":"2023-08-11T15:51:05.846118+0000","last_active":"2023-08-11T15:51:08.028532+0000","last_peered":"2023-08-11T15:51:08.028532+0000","last_clean":"2023-08-11T15:51:08.028532+0000","last_became_active":"2023-08-11T15:51:05.845703+0000","last_became_peered":"2023-08-11T15:51:05.845703+0000","last_unstale":"2023-08-11T15:51:08.028532+0000","last_undegraded":"2023-08-11T15:51:08.028532+0000","last_fullsized":"2023-08-11T15:51:08.028532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:15.472354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278289+0000","last_change":"2023-08-11T15:51:00.732689+0000","last_active":"2023-08-11T15:51:07.278289+0000","last_peered":"2023-08-11T15:51:07.278289+0000","last_clean":"2023-08-11T15:51:07.278289+0000","last_became_active":"2023-08-11T15:51:00.732185+0000","last_became_peered":"2023-08-11T15:51:00.732185+0000","last_unstale":"2023-08-11T15:51:07.278289+0000","last_undegraded":"2023-08-11T15:51:07.278289+0000","last_fullsized":"2023-08-11T15:51:07.278289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:19.559486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277505+0000","last_change":"2023-08-11T15:51:01.740441+0000","last_active":"2023-08-11T15:51:07.277505+0000","last_peered":"2023-08-11T15:51:07.277505+0000","last_clean":"2023-08-11T15:51:07.277505+0000","last_became_active":"2023-08-11T15:51:01.740104+0000","last_became_peered":"2023-08-11T15:51:01.740104+0000","last_unstale":"2023-08-11T15:51:07.277505+0000","last_undegraded":"2023-08-11T15:51:07.277505+0000","last_fullsized":"2023-08-11T15:51:07.277505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842536+0000","last_change":"2023-08-11T15:51:05.850642+0000","last_active":"2023-08-11T15:51:06.842536+0000","last_peered":"2023-08-11T15:51:06.842536+0000","last_clean":"2023-08-11T15:51:06.842536+0000","last_became_active":"2023-08-11T15:51:05.850276+0000","last_became_peered":"2023-08-11T15:51:05.850276+0000","last_unstale":"2023-08-11T15:51:06.842536+0000","last_undegraded":"2023-08-11T15:51:06.842536+0000","last_fullsized":"2023-08-11T15:51:06.842536+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:06:02.062687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845184+0000","last_change":"2023-08-11T15:51:00.753395+0000","last_active":"2023-08-11T15:51:06.845184+0000","last_peered":"2023-08-11T15:51:06.845184+0000","last_clean":"2023-08-11T15:51:06.845184+0000","last_became_active":"2023-08-11T15:51:00.753094+0000","last_became_peered":"2023-08-11T15:51:00.753094+0000","last_unstale":"2023-08-11T15:51:06.845184+0000","last_undegraded":"2023-08-11T15:51:06.845184+0000","last_fullsized":"2023-08-11T15:51:06.845184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:32.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029427+0000","last_change":"2023-08-11T15:51:01.748176+0000","last_active":"2023-08-11T15:51:08.029427+0000","last_peered":"2023-08-11T15:51:08.029427+0000","last_clean":"2023-08-11T15:51:08.029427+0000","last_became_active":"2023-08-11T15:51:01.747880+0000","last_became_peered":"2023-08-11T15:51:01.747880+0000","last_unstale":"2023-08-11T15:51:08.029427+0000","last_undegraded":"2023-08-11T15:51:08.029427+0000","last_fullsized":"2023-08-11T15:51:08.029427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:40:28.656635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842819+0000","last_change":"2023-08-11T15:51:05.854549+0000","last_active":"2023-08-11T15:51:06.842819+0000","last_peered":"2023-08-11T15:51:06.842819+0000","last_clean":"2023-08-11T15:51:06.842819+0000","last_became_active":"2023-08-11T15:51:05.853942+0000","last_became_peered":"2023-08-11T15:51:05.853942+0000","last_unstale":"2023-08-11T15:51:06.842819+0000","last_undegraded":"2023-08-11T15:51:06.842819+0000","last_fullsized":"2023-08-11T15:51:06.842819+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:12.616756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844270+0000","last_change":"2023-08-11T15:51:00.754010+0000","last_active":"2023-08-11T15:51:06.844270+0000","last_peered":"2023-08-11T15:51:06.844270+0000","last_clean":"2023-08-11T15:51:06.844270+0000","last_became_active":"2023-08-11T15:51:00.753640+0000","last_became_peered":"2023-08-11T15:51:00.753640+0000","last_unstale":"2023-08-11T15:51:06.844270+0000","last_undegraded":"2023-08-11T15:51:06.844270+0000","last_fullsized":"2023-08-11T15:51:06.844270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:59.887617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029380+0000","last_change":"2023-08-11T15:51:01.748240+0000","last_active":"2023-08-11T15:51:08.029380+0000","last_peered":"2023-08-11T15:51:08.029380+0000","last_clean":"2023-08-11T15:51:08.029380+0000","last_became_active":"2023-08-11T15:51:01.747946+0000","last_became_peered":"2023-08-11T15:51:01.747946+0000","last_unstale":"2023-08-11T15:51:08.029380+0000","last_undegraded":"2023-08-11T15:51:08.029380+0000","last_fullsized":"2023-08-11T15:51:08.029380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:08:40.170218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450286+0000","last_change":"2023-08-11T15:51:05.843794+0000","last_active":"2023-08-11T15:51:07.450286+0000","last_peered":"2023-08-11T15:51:07.450286+0000","last_clean":"2023-08-11T15:51:07.450286+0000","last_became_active":"2023-08-11T15:51:05.843006+0000","last_became_peered":"2023-08-11T15:51:05.843006+0000","last_unstale":"2023-08-11T15:51:07.450286+0000","last_undegraded":"2023-08-11T15:51:07.450286+0000","last_fullsized":"2023-08-11T15:51:07.450286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:53:03.920014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841384+0000","last_change":"2023-08-11T15:51:00.753341+0000","last_active":"2023-08-11T15:51:06.841384+0000","last_peered":"2023-08-11T15:51:06.841384+0000","last_clean":"2023-08-11T15:51:06.841384+0000","last_became_active":"2023-08-11T15:51:00.753070+0000","last_became_peered":"2023-08-11T15:51:00.753070+0000","last_unstale":"2023-08-11T15:51:06.841384+0000","last_undegraded":"2023-08-11T15:51:06.841384+0000","last_fullsized":"2023-08-11T15:51:06.841384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:07.508785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575377+0000","last_change":"2023-08-11T15:51:01.745198+0000","last_active":"2023-08-11T15:51:08.575377+0000","last_peered":"2023-08-11T15:51:08.575377+0000","last_clean":"2023-08-11T15:51:08.575377+0000","last_became_active":"2023-08-11T15:51:01.744845+0000","last_became_peered":"2023-08-11T15:51:01.744845+0000","last_unstale":"2023-08-11T15:51:08.575377+0000","last_undegraded":"2023-08-11T15:51:08.575377+0000","last_fullsized":"2023-08-11T15:51:08.575377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:07:54.433272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029435+0000","last_change":"2023-08-11T15:51:05.846476+0000","last_active":"2023-08-11T15:51:08.029435+0000","last_peered":"2023-08-11T15:51:08.029435+0000","last_clean":"2023-08-11T15:51:08.029435+0000","last_became_active":"2023-08-11T15:51:05.846010+0000","last_became_peered":"2023-08-11T15:51:05.846010+0000","last_unstale":"2023-08-11T15:51:08.029435+0000","last_undegraded":"2023-08-11T15:51:08.029435+0000","last_fullsized":"2023-08-11T15:51:08.029435+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:55:13.061478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029848+0000","last_change":"2023-08-11T15:51:00.748391+0000","last_active":"2023-08-11T15:51:08.029848+0000","last_peered":"2023-08-11T15:51:08.029848+0000","last_clean":"2023-08-11T15:51:08.029848+0000","last_became_active":"2023-08-11T15:51:00.744340+0000","last_became_peered":"2023-08-11T15:51:00.744340+0000","last_unstale":"2023-08-11T15:51:08.029848+0000","last_undegraded":"2023-08-11T15:51:08.029848+0000","last_fullsized":"2023-08-11T15:51:08.029848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:56:37.817338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278162+0000","last_change":"2023-08-11T15:51:01.749456+0000","last_active":"2023-08-11T15:51:07.278162+0000","last_peered":"2023-08-11T15:51:07.278162+0000","last_clean":"2023-08-11T15:51:07.278162+0000","last_became_active":"2023-08-11T15:51:01.749042+0000","last_became_peered":"2023-08-11T15:51:01.749042+0000","last_unstale":"2023-08-11T15:51:07.278162+0000","last_undegraded":"2023-08-11T15:51:07.278162+0000","last_fullsized":"2023-08-11T15:51:07.278162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:03.472522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644834+0000","last_change":"2023-08-11T15:51:05.849122+0000","last_active":"2023-08-11T15:51:08.644834+0000","last_peered":"2023-08-11T15:51:08.644834+0000","last_clean":"2023-08-11T15:51:08.644834+0000","last_became_active":"2023-08-11T15:51:05.848698+0000","last_became_peered":"2023-08-11T15:51:05.848698+0000","last_unstale":"2023-08-11T15:51:08.644834+0000","last_undegraded":"2023-08-11T15:51:08.644834+0000","last_fullsized":"2023-08-11T15:51:08.644834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:46:03.919402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449963+0000","last_change":"2023-08-11T15:51:00.743735+0000","last_active":"2023-08-11T15:51:07.449963+0000","last_peered":"2023-08-11T15:51:07.449963+0000","last_clean":"2023-08-11T15:51:07.449963+0000","last_became_active":"2023-08-11T15:51:00.743412+0000","last_became_peered":"2023-08-11T15:51:00.743412+0000","last_unstale":"2023-08-11T15:51:07.449963+0000","last_undegraded":"2023-08-11T15:51:07.449963+0000","last_fullsized":"2023-08-11T15:51:07.449963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:07.487770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841592+0000","last_change":"2023-08-11T15:51:01.740683+0000","last_active":"2023-08-11T15:51:06.841592+0000","last_peered":"2023-08-11T15:51:06.841592+0000","last_clean":"2023-08-11T15:51:06.841592+0000","last_became_active":"2023-08-11T15:51:01.740230+0000","last_became_peered":"2023-08-11T15:51:01.740230+0000","last_unstale":"2023-08-11T15:51:06.841592+0000","last_undegraded":"2023-08-11T15:51:06.841592+0000","last_fullsized":"2023-08-11T15:51:06.841592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:57.966913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841129+0000","last_change":"2023-08-11T15:51:05.843204+0000","last_active":"2023-08-11T15:51:06.841129+0000","last_peered":"2023-08-11T15:51:06.841129+0000","last_clean":"2023-08-11T15:51:06.841129+0000","last_became_active":"2023-08-11T15:51:05.842830+0000","last_became_peered":"2023-08-11T15:51:05.842830+0000","last_unstale":"2023-08-11T15:51:06.841129+0000","last_undegraded":"2023-08-11T15:51:06.841129+0000","last_fullsized":"2023-08-11T15:51:06.841129+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:13.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277138+0000","last_change":"2023-08-11T15:51:00.735703+0000","last_active":"2023-08-11T15:51:07.277138+0000","last_peered":"2023-08-11T15:51:07.277138+0000","last_clean":"2023-08-11T15:51:07.277138+0000","last_became_active":"2023-08-11T15:51:00.735384+0000","last_became_peered":"2023-08-11T15:51:00.735384+0000","last_unstale":"2023-08-11T15:51:07.277138+0000","last_undegraded":"2023-08-11T15:51:07.277138+0000","last_fullsized":"2023-08-11T15:51:07.277138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:59.887617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644911+0000","last_change":"2023-08-11T15:51:01.742945+0000","last_active":"2023-08-11T15:51:08.644911+0000","last_peered":"2023-08-11T15:51:08.644911+0000","last_clean":"2023-08-11T15:51:08.644911+0000","last_became_active":"2023-08-11T15:51:01.742477+0000","last_became_peered":"2023-08-11T15:51:01.742477+0000","last_unstale":"2023-08-11T15:51:08.644911+0000","last_undegraded":"2023-08-11T15:51:08.644911+0000","last_fullsized":"2023-08-11T15:51:08.644911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:17:49.108624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276720+0000","last_change":"2023-08-11T15:51:01.755598+0000","last_active":"2023-08-11T15:51:07.276720+0000","last_peered":"2023-08-11T15:51:07.276720+0000","last_clean":"2023-08-11T15:51:07.276720+0000","last_became_active":"2023-08-11T15:51:01.755329+0000","last_became_peered":"2023-08-11T15:51:01.755329+0000","last_unstale":"2023-08-11T15:51:07.276720+0000","last_undegraded":"2023-08-11T15:51:07.276720+0000","last_fullsized":"2023-08-11T15:51:07.276720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:09.551106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842608+0000","last_change":"2023-08-11T15:51:00.747156+0000","last_active":"2023-08-11T15:51:06.842608+0000","last_peered":"2023-08-11T15:51:06.842608+0000","last_clean":"2023-08-11T15:51:06.842608+0000","last_became_active":"2023-08-11T15:51:00.745449+0000","last_became_peered":"2023-08-11T15:51:00.745449+0000","last_unstale":"2023-08-11T15:51:06.842608+0000","last_undegraded":"2023-08-11T15:51:06.842608+0000","last_fullsized":"2023-08-11T15:51:06.842608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643560+0000","last_change":"2023-08-11T15:51:05.839469+0000","last_active":"2023-08-11T15:51:08.643560+0000","last_peered":"2023-08-11T15:51:08.643560+0000","last_clean":"2023-08-11T15:51:08.643560+0000","last_became_active":"2023-08-11T15:51:05.839035+0000","last_became_peered":"2023-08-11T15:51:05.839035+0000","last_unstale":"2023-08-11T15:51:08.643560+0000","last_undegraded":"2023-08-11T15:51:08.643560+0000","last_fullsized":"2023-08-11T15:51:08.643560+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:11.122033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276959+0000","last_change":"2023-08-11T15:51:01.748158+0000","last_active":"2023-08-11T15:51:07.276959+0000","last_peered":"2023-08-11T15:51:07.276959+0000","last_clean":"2023-08-11T15:51:07.276959+0000","last_became_active":"2023-08-11T15:51:01.747870+0000","last_became_peered":"2023-08-11T15:51:01.747870+0000","last_unstale":"2023-08-11T15:51:07.276959+0000","last_undegraded":"2023-08-11T15:51:07.276959+0000","last_fullsized":"2023-08-11T15:51:07.276959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:57.966913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845778+0000","last_change":"2023-08-11T15:51:00.739033+0000","last_active":"2023-08-11T15:51:06.845778+0000","last_peered":"2023-08-11T15:51:06.845778+0000","last_clean":"2023-08-11T15:51:06.845778+0000","last_became_active":"2023-08-11T15:51:00.738746+0000","last_became_peered":"2023-08-11T15:51:00.738746+0000","last_unstale":"2023-08-11T15:51:06.845778+0000","last_undegraded":"2023-08-11T15:51:06.845778+0000","last_fullsized":"2023-08-11T15:51:06.845778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841377+0000","last_change":"2023-08-11T15:51:05.853925+0000","last_active":"2023-08-11T15:51:06.841377+0000","last_peered":"2023-08-11T15:51:06.841377+0000","last_clean":"2023-08-11T15:51:06.841377+0000","last_became_active":"2023-08-11T15:51:05.853531+0000","last_became_peered":"2023-08-11T15:51:05.853531+0000","last_unstale":"2023-08-11T15:51:06.841377+0000","last_undegraded":"2023-08-11T15:51:06.841377+0000","last_fullsized":"2023-08-11T15:51:06.841377+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:13.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843271+0000","last_change":"2023-08-11T15:51:00.733126+0000","last_active":"2023-08-11T15:51:06.843271+0000","last_peered":"2023-08-11T15:51:06.843271+0000","last_clean":"2023-08-11T15:51:06.843271+0000","last_became_active":"2023-08-11T15:51:00.731695+0000","last_became_peered":"2023-08-11T15:51:00.731695+0000","last_unstale":"2023-08-11T15:51:06.843271+0000","last_undegraded":"2023-08-11T15:51:06.843271+0000","last_fullsized":"2023-08-11T15:51:06.843271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:07.914582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449817+0000","last_change":"2023-08-11T15:51:01.750498+0000","last_active":"2023-08-11T15:51:07.449817+0000","last_peered":"2023-08-11T15:51:07.449817+0000","last_clean":"2023-08-11T15:51:07.449817+0000","last_became_active":"2023-08-11T15:51:01.750204+0000","last_became_peered":"2023-08-11T15:51:01.750204+0000","last_unstale":"2023-08-11T15:51:07.449817+0000","last_undegraded":"2023-08-11T15:51:07.449817+0000","last_fullsized":"2023-08-11T15:51:07.449817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:30:40.844864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277300+0000","last_change":"2023-08-11T15:51:05.849252+0000","last_active":"2023-08-11T15:51:07.277300+0000","last_peered":"2023-08-11T15:51:07.277300+0000","last_clean":"2023-08-11T15:51:07.277300+0000","last_became_active":"2023-08-11T15:51:05.848107+0000","last_became_peered":"2023-08-11T15:51:05.848107+0000","last_unstale":"2023-08-11T15:51:07.277300+0000","last_undegraded":"2023-08-11T15:51:07.277300+0000","last_fullsized":"2023-08-11T15:51:07.277300+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:13.022552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:13.714518+0000","last_change":"2023-08-11T15:51:00.743771+0000","last_active":"2023-08-11T15:51:13.714518+0000","last_peered":"2023-08-11T15:51:13.714518+0000","last_clean":"2023-08-11T15:51:13.714518+0000","last_became_active":"2023-08-11T15:51:00.741410+0000","last_became_peered":"2023-08-11T15:51:00.741410+0000","last_unstale":"2023-08-11T15:51:13.714518+0000","last_undegraded":"2023-08-11T15:51:13.714518+0000","last_fullsized":"2023-08-11T15:51:13.714518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:46:01.024956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029670+0000","last_change":"2023-08-11T15:51:01.742960+0000","last_active":"2023-08-11T15:51:08.029670+0000","last_peered":"2023-08-11T15:51:08.029670+0000","last_clean":"2023-08-11T15:51:08.029670+0000","last_became_active":"2023-08-11T15:51:01.742674+0000","last_became_peered":"2023-08-11T15:51:01.742674+0000","last_unstale":"2023-08-11T15:51:08.029670+0000","last_undegraded":"2023-08-11T15:51:08.029670+0000","last_fullsized":"2023-08-11T15:51:08.029670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:16:42.978408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450187+0000","last_change":"2023-08-11T15:51:05.850946+0000","last_active":"2023-08-11T15:51:07.450187+0000","last_peered":"2023-08-11T15:51:07.450187+0000","last_clean":"2023-08-11T15:51:07.450187+0000","last_became_active":"2023-08-11T15:51:05.850520+0000","last_became_peered":"2023-08-11T15:51:05.850520+0000","last_unstale":"2023-08-11T15:51:07.450187+0000","last_undegraded":"2023-08-11T15:51:07.450187+0000","last_fullsized":"2023-08-11T15:51:07.450187+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:19:49.259478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029612+0000","last_change":"2023-08-11T15:51:01.733689+0000","last_active":"2023-08-11T15:51:08.029612+0000","last_peered":"2023-08-11T15:51:08.029612+0000","last_clean":"2023-08-11T15:51:08.029612+0000","last_became_active":"2023-08-11T15:51:01.732609+0000","last_became_peered":"2023-08-11T15:51:01.732609+0000","last_unstale":"2023-08-11T15:51:08.029612+0000","last_undegraded":"2023-08-11T15:51:08.029612+0000","last_fullsized":"2023-08-11T15:51:08.029612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:37:47.670825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.645014+0000","last_change":"2023-08-11T15:50:54.667601+0000","last_active":"2023-08-11T15:51:08.645014+0000","last_peered":"2023-08-11T15:51:08.645014+0000","last_clean":"2023-08-11T15:51:08.645014+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:51:08.645014+0000","last_undegraded":"2023-08-11T15:51:08.645014+0000","last_fullsized":"2023-08-11T15:51:08.645014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:26.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842067+0000","last_change":"2023-08-11T15:51:00.744472+0000","last_active":"2023-08-11T15:51:06.842067+0000","last_peered":"2023-08-11T15:51:06.842067+0000","last_clean":"2023-08-11T15:51:06.842067+0000","last_became_active":"2023-08-11T15:51:00.744162+0000","last_became_peered":"2023-08-11T15:51:00.744162+0000","last_unstale":"2023-08-11T15:51:06.842067+0000","last_undegraded":"2023-08-11T15:51:06.842067+0000","last_fullsized":"2023-08-11T15:51:06.842067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:44.445917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276689+0000","last_change":"2023-08-11T15:51:05.840924+0000","last_active":"2023-08-11T15:51:07.276689+0000","last_peered":"2023-08-11T15:51:07.276689+0000","last_clean":"2023-08-11T15:51:07.276689+0000","last_became_active":"2023-08-11T15:51:05.840447+0000","last_became_peered":"2023-08-11T15:51:05.840447+0000","last_unstale":"2023-08-11T15:51:07.276689+0000","last_undegraded":"2023-08-11T15:51:07.276689+0000","last_fullsized":"2023-08-11T15:51:07.276689+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:15.105060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277414+0000","last_change":"2023-08-11T15:51:01.747032+0000","last_active":"2023-08-11T15:51:07.277414+0000","last_peered":"2023-08-11T15:51:07.277414+0000","last_clean":"2023-08-11T15:51:07.277414+0000","last_became_active":"2023-08-11T15:51:01.746677+0000","last_became_peered":"2023-08-11T15:51:01.746677+0000","last_unstale":"2023-08-11T15:51:07.277414+0000","last_undegraded":"2023-08-11T15:51:07.277414+0000","last_fullsized":"2023-08-11T15:51:07.277414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:45.458114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450746+0000","last_change":"2023-08-11T15:50:54.662968+0000","last_active":"2023-08-11T15:51:07.450746+0000","last_peered":"2023-08-11T15:51:07.450746+0000","last_clean":"2023-08-11T15:51:07.450746+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:51:07.450746+0000","last_undegraded":"2023-08-11T15:51:07.450746+0000","last_fullsized":"2023-08-11T15:51:07.450746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:21.192999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278963+0000","last_change":"2023-08-11T15:51:00.728408+0000","last_active":"2023-08-11T15:51:07.278963+0000","last_peered":"2023-08-11T15:51:07.278963+0000","last_clean":"2023-08-11T15:51:07.278963+0000","last_became_active":"2023-08-11T15:51:00.721838+0000","last_became_peered":"2023-08-11T15:51:00.721838+0000","last_unstale":"2023-08-11T15:51:07.278963+0000","last_undegraded":"2023-08-11T15:51:07.278963+0000","last_fullsized":"2023-08-11T15:51:07.278963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:10.364383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028778+0000","last_change":"2023-08-11T15:51:05.850931+0000","last_active":"2023-08-11T15:51:08.028778+0000","last_peered":"2023-08-11T15:51:08.028778+0000","last_clean":"2023-08-11T15:51:08.028778+0000","last_became_active":"2023-08-11T15:51:05.850227+0000","last_became_peered":"2023-08-11T15:51:05.850227+0000","last_unstale":"2023-08-11T15:51:08.028778+0000","last_undegraded":"2023-08-11T15:51:08.028778+0000","last_fullsized":"2023-08-11T15:51:08.028778+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.029041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840867+0000","last_change":"2023-08-11T15:51:01.739504+0000","last_active":"2023-08-11T15:51:06.840867+0000","last_peered":"2023-08-11T15:51:06.840867+0000","last_clean":"2023-08-11T15:51:06.840867+0000","last_became_active":"2023-08-11T15:51:01.739172+0000","last_became_peered":"2023-08-11T15:51:01.739172+0000","last_unstale":"2023-08-11T15:51:06.840867+0000","last_undegraded":"2023-08-11T15:51:06.840867+0000","last_fullsized":"2023-08-11T15:51:06.840867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:12:34.705521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844166+0000","last_change":"2023-08-11T15:50:55.272783+0000","last_active":"2023-08-11T15:51:06.844166+0000","last_peered":"2023-08-11T15:51:06.844166+0000","last_clean":"2023-08-11T15:51:06.844166+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:51:06.844166+0000","last_undegraded":"2023-08-11T15:51:06.844166+0000","last_fullsized":"2023-08-11T15:51:06.844166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:26.593873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450158+0000","last_change":"2023-08-11T15:51:00.737083+0000","last_active":"2023-08-11T15:51:07.450158+0000","last_peered":"2023-08-11T15:51:07.450158+0000","last_clean":"2023-08-11T15:51:07.450158+0000","last_became_active":"2023-08-11T15:51:00.736799+0000","last_became_peered":"2023-08-11T15:51:00.736799+0000","last_unstale":"2023-08-11T15:51:07.450158+0000","last_undegraded":"2023-08-11T15:51:07.450158+0000","last_fullsized":"2023-08-11T15:51:07.450158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:03:14.195345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643834+0000","last_change":"2023-08-11T15:51:05.842861+0000","last_active":"2023-08-11T15:51:08.643834+0000","last_peered":"2023-08-11T15:51:08.643834+0000","last_clean":"2023-08-11T15:51:08.643834+0000","last_became_active":"2023-08-11T15:51:05.842445+0000","last_became_peered":"2023-08-11T15:51:05.842445+0000","last_unstale":"2023-08-11T15:51:08.643834+0000","last_undegraded":"2023-08-11T15:51:08.643834+0000","last_fullsized":"2023-08-11T15:51:08.643834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:41.996042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449997+0000","last_change":"2023-08-11T15:51:01.746944+0000","last_active":"2023-08-11T15:51:07.449997+0000","last_peered":"2023-08-11T15:51:07.449997+0000","last_clean":"2023-08-11T15:51:07.449997+0000","last_became_active":"2023-08-11T15:51:01.745106+0000","last_became_peered":"2023-08-11T15:51:01.745106+0000","last_unstale":"2023-08-11T15:51:07.449997+0000","last_undegraded":"2023-08-11T15:51:07.449997+0000","last_fullsized":"2023-08-11T15:51:07.449997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:38:45.900704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450317+0000","last_change":"2023-08-11T15:50:55.270974+0000","last_active":"2023-08-11T15:51:07.450317+0000","last_peered":"2023-08-11T15:51:07.450317+0000","last_clean":"2023-08-11T15:51:07.450317+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:51:07.450317+0000","last_undegraded":"2023-08-11T15:51:07.450317+0000","last_fullsized":"2023-08-11T15:51:07.450317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:49:35.912143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845445+0000","last_change":"2023-08-11T15:51:00.736540+0000","last_active":"2023-08-11T15:51:06.845445+0000","last_peered":"2023-08-11T15:51:06.845445+0000","last_clean":"2023-08-11T15:51:06.845445+0000","last_became_active":"2023-08-11T15:51:00.736211+0000","last_became_peered":"2023-08-11T15:51:00.736211+0000","last_unstale":"2023-08-11T15:51:06.845445+0000","last_undegraded":"2023-08-11T15:51:06.845445+0000","last_fullsized":"2023-08-11T15:51:06.845445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.749276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840401+0000","last_change":"2023-08-11T15:51:05.834845+0000","last_active":"2023-08-11T15:51:06.840401+0000","last_peered":"2023-08-11T15:51:06.840401+0000","last_clean":"2023-08-11T15:51:06.840401+0000","last_became_active":"2023-08-11T15:51:05.834329+0000","last_became_peered":"2023-08-11T15:51:05.834329+0000","last_unstale":"2023-08-11T15:51:06.840401+0000","last_undegraded":"2023-08-11T15:51:06.840401+0000","last_fullsized":"2023-08-11T15:51:06.840401+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:27:54.750554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450003+0000","last_change":"2023-08-11T15:51:01.744716+0000","last_active":"2023-08-11T15:51:07.450003+0000","last_peered":"2023-08-11T15:51:07.450003+0000","last_clean":"2023-08-11T15:51:07.450003+0000","last_became_active":"2023-08-11T15:51:01.744413+0000","last_became_peered":"2023-08-11T15:51:01.744413+0000","last_unstale":"2023-08-11T15:51:07.450003+0000","last_undegraded":"2023-08-11T15:51:07.450003+0000","last_fullsized":"2023-08-11T15:51:07.450003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:43:31.654955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030156+0000","last_change":"2023-08-11T15:51:00.744357+0000","last_active":"2023-08-11T15:51:08.030156+0000","last_peered":"2023-08-11T15:51:08.030156+0000","last_clean":"2023-08-11T15:51:08.030156+0000","last_became_active":"2023-08-11T15:51:00.744005+0000","last_became_peered":"2023-08-11T15:51:00.744005+0000","last_unstale":"2023-08-11T15:51:08.030156+0000","last_undegraded":"2023-08-11T15:51:08.030156+0000","last_fullsized":"2023-08-11T15:51:08.030156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:40:52.791014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842501+0000","last_change":"2023-08-11T15:50:54.660862+0000","last_active":"2023-08-11T15:51:06.842501+0000","last_peered":"2023-08-11T15:51:06.842501+0000","last_clean":"2023-08-11T15:51:06.842501+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:51:06.842501+0000","last_undegraded":"2023-08-11T15:51:06.842501+0000","last_fullsized":"2023-08-11T15:51:06.842501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:01.459444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840964+0000","last_change":"2023-08-11T15:51:05.834675+0000","last_active":"2023-08-11T15:51:06.840964+0000","last_peered":"2023-08-11T15:51:06.840964+0000","last_clean":"2023-08-11T15:51:06.840964+0000","last_became_active":"2023-08-11T15:51:05.833754+0000","last_became_peered":"2023-08-11T15:51:05.833754+0000","last_unstale":"2023-08-11T15:51:06.840964+0000","last_undegraded":"2023-08-11T15:51:06.840964+0000","last_fullsized":"2023-08-11T15:51:06.840964+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:02:13.485965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450824+0000","last_change":"2023-08-11T15:51:01.747929+0000","last_active":"2023-08-11T15:51:07.450824+0000","last_peered":"2023-08-11T15:51:07.450824+0000","last_clean":"2023-08-11T15:51:07.450824+0000","last_became_active":"2023-08-11T15:51:01.747489+0000","last_became_peered":"2023-08-11T15:51:01.747489+0000","last_unstale":"2023-08-11T15:51:07.450824+0000","last_undegraded":"2023-08-11T15:51:07.450824+0000","last_fullsized":"2023-08-11T15:51:07.450824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:16:47.148363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449531+0000","last_change":"2023-08-11T15:51:00.735611+0000","last_active":"2023-08-11T15:51:07.449531+0000","last_peered":"2023-08-11T15:51:07.449531+0000","last_clean":"2023-08-11T15:51:07.449531+0000","last_became_active":"2023-08-11T15:51:00.735334+0000","last_became_peered":"2023-08-11T15:51:00.735334+0000","last_unstale":"2023-08-11T15:51:07.449531+0000","last_undegraded":"2023-08-11T15:51:07.449531+0000","last_fullsized":"2023-08-11T15:51:07.449531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:21:47.785823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574361+0000","last_change":"2023-08-11T15:50:54.660575+0000","last_active":"2023-08-11T15:51:08.574361+0000","last_peered":"2023-08-11T15:51:08.574361+0000","last_clean":"2023-08-11T15:51:08.574361+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:51:08.574361+0000","last_undegraded":"2023-08-11T15:51:08.574361+0000","last_fullsized":"2023-08-11T15:51:08.574361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:44:19.275242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574984+0000","last_change":"2023-08-11T15:51:05.851204+0000","last_active":"2023-08-11T15:51:08.574984+0000","last_peered":"2023-08-11T15:51:08.574984+0000","last_clean":"2023-08-11T15:51:08.574984+0000","last_became_active":"2023-08-11T15:51:05.850707+0000","last_became_peered":"2023-08-11T15:51:05.850707+0000","last_unstale":"2023-08-11T15:51:08.574984+0000","last_undegraded":"2023-08-11T15:51:08.574984+0000","last_fullsized":"2023-08-11T15:51:08.574984+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:46:46.279070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841686+0000","last_change":"2023-08-11T15:51:01.754726+0000","last_active":"2023-08-11T15:51:06.841686+0000","last_peered":"2023-08-11T15:51:06.841686+0000","last_clean":"2023-08-11T15:51:06.841686+0000","last_became_active":"2023-08-11T15:51:01.754139+0000","last_became_peered":"2023-08-11T15:51:01.754139+0000","last_unstale":"2023-08-11T15:51:06.841686+0000","last_undegraded":"2023-08-11T15:51:06.841686+0000","last_fullsized":"2023-08-11T15:51:06.841686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839697+0000","last_change":"2023-08-11T15:51:00.753588+0000","last_active":"2023-08-11T15:51:06.839697+0000","last_peered":"2023-08-11T15:51:06.839697+0000","last_clean":"2023-08-11T15:51:06.839697+0000","last_became_active":"2023-08-11T15:51:00.753328+0000","last_became_peered":"2023-08-11T15:51:00.753328+0000","last_unstale":"2023-08-11T15:51:06.839697+0000","last_undegraded":"2023-08-11T15:51:06.839697+0000","last_fullsized":"2023-08-11T15:51:06.839697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:32.917550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842875+0000","last_change":"2023-08-11T15:50:54.669675+0000","last_active":"2023-08-11T15:51:06.842875+0000","last_peered":"2023-08-11T15:51:06.842875+0000","last_clean":"2023-08-11T15:51:06.842875+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:51:06.842875+0000","last_undegraded":"2023-08-11T15:51:06.842875+0000","last_fullsized":"2023-08-11T15:51:06.842875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:50.905375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845192+0000","last_change":"2023-08-11T15:51:05.839222+0000","last_active":"2023-08-11T15:51:06.845192+0000","last_peered":"2023-08-11T15:51:06.845192+0000","last_clean":"2023-08-11T15:51:06.845192+0000","last_became_active":"2023-08-11T15:51:05.838710+0000","last_became_peered":"2023-08-11T15:51:05.838710+0000","last_unstale":"2023-08-11T15:51:06.845192+0000","last_undegraded":"2023-08-11T15:51:06.845192+0000","last_fullsized":"2023-08-11T15:51:06.845192+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:07.568296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644528+0000","last_change":"2023-08-11T15:51:01.744538+0000","last_active":"2023-08-11T15:51:08.644528+0000","last_peered":"2023-08-11T15:51:08.644528+0000","last_clean":"2023-08-11T15:51:08.644528+0000","last_became_active":"2023-08-11T15:51:01.744177+0000","last_became_peered":"2023-08-11T15:51:01.744177+0000","last_unstale":"2023-08-11T15:51:08.644528+0000","last_undegraded":"2023-08-11T15:51:08.644528+0000","last_fullsized":"2023-08-11T15:51:08.644528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:36.996581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277663+0000","last_change":"2023-08-11T15:51:00.736542+0000","last_active":"2023-08-11T15:51:07.277663+0000","last_peered":"2023-08-11T15:51:07.277663+0000","last_clean":"2023-08-11T15:51:07.277663+0000","last_became_active":"2023-08-11T15:51:00.736156+0000","last_became_peered":"2023-08-11T15:51:00.736156+0000","last_unstale":"2023-08-11T15:51:07.277663+0000","last_undegraded":"2023-08-11T15:51:07.277663+0000","last_fullsized":"2023-08-11T15:51:07.277663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843800+0000","last_change":"2023-08-11T15:50:54.663730+0000","last_active":"2023-08-11T15:51:06.843800+0000","last_peered":"2023-08-11T15:51:06.843800+0000","last_clean":"2023-08-11T15:51:06.843800+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:51:06.843800+0000","last_undegraded":"2023-08-11T15:51:06.843800+0000","last_fullsized":"2023-08-11T15:51:06.843800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:36.826532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843222+0000","last_change":"2023-08-11T15:50:48.426393+0000","last_active":"2023-08-11T15:51:06.843222+0000","last_peered":"2023-08-11T15:51:06.843222+0000","last_clean":"2023-08-11T15:51:06.843222+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:51:06.843222+0000","last_undegraded":"2023-08-11T15:51:06.843222+0000","last_fullsized":"2023-08-11T15:51:06.843222+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_clean_scrub_stamp":"2023-08-11T15:50:47.133560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:47.340557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843724+0000","last_change":"2023-08-11T15:51:05.850670+0000","last_active":"2023-08-11T15:51:06.843724+0000","last_peered":"2023-08-11T15:51:06.843724+0000","last_clean":"2023-08-11T15:51:06.843724+0000","last_became_active":"2023-08-11T15:51:05.850026+0000","last_became_peered":"2023-08-11T15:51:05.850026+0000","last_unstale":"2023-08-11T15:51:06.843724+0000","last_undegraded":"2023-08-11T15:51:06.843724+0000","last_fullsized":"2023-08-11T15:51:06.843724+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:08.068219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450801+0000","last_change":"2023-08-11T15:51:01.748517+0000","last_active":"2023-08-11T15:51:07.450801+0000","last_peered":"2023-08-11T15:51:07.450801+0000","last_clean":"2023-08-11T15:51:07.450801+0000","last_became_active":"2023-08-11T15:51:01.748222+0000","last_became_peered":"2023-08-11T15:51:01.748222+0000","last_unstale":"2023-08-11T15:51:07.450801+0000","last_undegraded":"2023-08-11T15:51:07.450801+0000","last_fullsized":"2023-08-11T15:51:07.450801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:24:23.784093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277502+0000","last_change":"2023-08-11T15:51:00.734516+0000","last_active":"2023-08-11T15:51:07.277502+0000","last_peered":"2023-08-11T15:51:07.277502+0000","last_clean":"2023-08-11T15:51:07.277502+0000","last_became_active":"2023-08-11T15:51:00.734143+0000","last_became_peered":"2023-08-11T15:51:00.734143+0000","last_unstale":"2023-08-11T15:51:07.277502+0000","last_undegraded":"2023-08-11T15:51:07.277502+0000","last_fullsized":"2023-08-11T15:51:07.277502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:02.960249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030585+0000","last_change":"2023-08-11T15:51:05.848396+0000","last_active":"2023-08-11T15:51:08.030585+0000","last_peered":"2023-08-11T15:51:08.030585+0000","last_clean":"2023-08-11T15:51:08.030585+0000","last_became_active":"2023-08-11T15:51:05.846255+0000","last_became_peered":"2023-08-11T15:51:05.846255+0000","last_unstale":"2023-08-11T15:51:08.030585+0000","last_undegraded":"2023-08-11T15:51:08.030585+0000","last_fullsized":"2023-08-11T15:51:08.030585+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:11:39.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.031055+0000","last_change":"2023-08-11T15:51:01.745387+0000","last_active":"2023-08-11T15:51:08.031055+0000","last_peered":"2023-08-11T15:51:08.031055+0000","last_clean":"2023-08-11T15:51:08.031055+0000","last_became_active":"2023-08-11T15:51:01.745076+0000","last_became_peered":"2023-08-11T15:51:01.745076+0000","last_unstale":"2023-08-11T15:51:08.031055+0000","last_undegraded":"2023-08-11T15:51:08.031055+0000","last_fullsized":"2023-08-11T15:51:08.031055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:46:26.462256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030159+0000","last_change":"2023-08-11T15:51:00.749075+0000","last_active":"2023-08-11T15:51:08.030159+0000","last_peered":"2023-08-11T15:51:08.030159+0000","last_clean":"2023-08-11T15:51:08.030159+0000","last_became_active":"2023-08-11T15:51:00.748709+0000","last_became_peered":"2023-08-11T15:51:00.748709+0000","last_unstale":"2023-08-11T15:51:08.030159+0000","last_undegraded":"2023-08-11T15:51:08.030159+0000","last_fullsized":"2023-08-11T15:51:08.030159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:18:35.477914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844801+0000","last_change":"2023-08-11T15:51:05.843837+0000","last_active":"2023-08-11T15:51:06.844801+0000","last_peered":"2023-08-11T15:51:06.844801+0000","last_clean":"2023-08-11T15:51:06.844801+0000","last_became_active":"2023-08-11T15:51:05.843261+0000","last_became_peered":"2023-08-11T15:51:05.843261+0000","last_unstale":"2023-08-11T15:51:06.844801+0000","last_undegraded":"2023-08-11T15:51:06.844801+0000","last_fullsized":"2023-08-11T15:51:06.844801+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:08.068219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030584+0000","last_change":"2023-08-11T15:51:01.735881+0000","last_active":"2023-08-11T15:51:08.030584+0000","last_peered":"2023-08-11T15:51:08.030584+0000","last_clean":"2023-08-11T15:51:08.030584+0000","last_became_active":"2023-08-11T15:51:01.735297+0000","last_became_peered":"2023-08-11T15:51:01.735297+0000","last_unstale":"2023-08-11T15:51:08.030584+0000","last_undegraded":"2023-08-11T15:51:08.030584+0000","last_fullsized":"2023-08-11T15:51:08.030584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:06:05.316781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644461+0000","last_change":"2023-08-11T15:51:00.754786+0000","last_active":"2023-08-11T15:51:08.644461+0000","last_peered":"2023-08-11T15:51:08.644461+0000","last_clean":"2023-08-11T15:51:08.644461+0000","last_became_active":"2023-08-11T15:51:00.754437+0000","last_became_peered":"2023-08-11T15:51:00.754437+0000","last_unstale":"2023-08-11T15:51:08.644461+0000","last_undegraded":"2023-08-11T15:51:08.644461+0000","last_fullsized":"2023-08-11T15:51:08.644461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:41:49.524739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449772+0000","last_change":"2023-08-11T15:51:05.833713+0000","last_active":"2023-08-11T15:51:07.449772+0000","last_peered":"2023-08-11T15:51:07.449772+0000","last_clean":"2023-08-11T15:51:07.449772+0000","last_became_active":"2023-08-11T15:51:05.833089+0000","last_became_peered":"2023-08-11T15:51:05.833089+0000","last_unstale":"2023-08-11T15:51:07.449772+0000","last_undegraded":"2023-08-11T15:51:07.449772+0000","last_fullsized":"2023-08-11T15:51:07.449772+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:17:01.770576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030037+0000","last_change":"2023-08-11T15:51:01.748944+0000","last_active":"2023-08-11T15:51:08.030037+0000","last_peered":"2023-08-11T15:51:08.030037+0000","last_clean":"2023-08-11T15:51:08.030037+0000","last_became_active":"2023-08-11T15:51:01.748661+0000","last_became_peered":"2023-08-11T15:51:01.748661+0000","last_unstale":"2023-08-11T15:51:08.030037+0000","last_undegraded":"2023-08-11T15:51:08.030037+0000","last_fullsized":"2023-08-11T15:51:08.030037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:02:46.657720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278376+0000","last_change":"2023-08-11T15:51:00.743809+0000","last_active":"2023-08-11T15:51:07.278376+0000","last_peered":"2023-08-11T15:51:07.278376+0000","last_clean":"2023-08-11T15:51:07.278376+0000","last_became_active":"2023-08-11T15:51:00.743475+0000","last_became_peered":"2023-08-11T15:51:00.743475+0000","last_unstale":"2023-08-11T15:51:07.278376+0000","last_undegraded":"2023-08-11T15:51:07.278376+0000","last_fullsized":"2023-08-11T15:51:07.278376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574614+0000","last_change":"2023-08-11T15:51:05.845552+0000","last_active":"2023-08-11T15:51:08.574614+0000","last_peered":"2023-08-11T15:51:08.574614+0000","last_clean":"2023-08-11T15:51:08.574614+0000","last_became_active":"2023-08-11T15:51:05.845089+0000","last_became_peered":"2023-08-11T15:51:05.845089+0000","last_unstale":"2023-08-11T15:51:08.574614+0000","last_undegraded":"2023-08-11T15:51:08.574614+0000","last_fullsized":"2023-08-11T15:51:08.574614+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:14:44.850189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840418+0000","last_change":"2023-08-11T15:51:01.754448+0000","last_active":"2023-08-11T15:51:06.840418+0000","last_peered":"2023-08-11T15:51:06.840418+0000","last_clean":"2023-08-11T15:51:06.840418+0000","last_became_active":"2023-08-11T15:51:01.754103+0000","last_became_peered":"2023-08-11T15:51:01.754103+0000","last_unstale":"2023-08-11T15:51:06.840418+0000","last_undegraded":"2023-08-11T15:51:06.840418+0000","last_fullsized":"2023-08-11T15:51:06.840418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:48:19.707007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844315+0000","last_change":"2023-08-11T15:51:00.740399+0000","last_active":"2023-08-11T15:51:06.844315+0000","last_peered":"2023-08-11T15:51:06.844315+0000","last_clean":"2023-08-11T15:51:06.844315+0000","last_became_active":"2023-08-11T15:51:00.740100+0000","last_became_peered":"2023-08-11T15:51:00.740100+0000","last_unstale":"2023-08-11T15:51:06.844315+0000","last_undegraded":"2023-08-11T15:51:06.844315+0000","last_fullsized":"2023-08-11T15:51:06.844315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:07.914582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574234+0000","last_change":"2023-08-11T15:51:05.857518+0000","last_active":"2023-08-11T15:51:08.574234+0000","last_peered":"2023-08-11T15:51:08.574234+0000","last_clean":"2023-08-11T15:51:08.574234+0000","last_became_active":"2023-08-11T15:51:05.857113+0000","last_became_peered":"2023-08-11T15:51:05.857113+0000","last_unstale":"2023-08-11T15:51:08.574234+0000","last_undegraded":"2023-08-11T15:51:08.574234+0000","last_fullsized":"2023-08-11T15:51:08.574234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:27:22.211685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643846+0000","last_change":"2023-08-11T15:51:01.754343+0000","last_active":"2023-08-11T15:51:08.643846+0000","last_peered":"2023-08-11T15:51:08.643846+0000","last_clean":"2023-08-11T15:51:08.643846+0000","last_became_active":"2023-08-11T15:51:01.754068+0000","last_became_peered":"2023-08-11T15:51:01.754068+0000","last_unstale":"2023-08-11T15:51:08.643846+0000","last_undegraded":"2023-08-11T15:51:08.643846+0000","last_fullsized":"2023-08-11T15:51:08.643846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:17.417849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843859+0000","last_change":"2023-08-11T15:51:00.739404+0000","last_active":"2023-08-11T15:51:06.843859+0000","last_peered":"2023-08-11T15:51:06.843859+0000","last_clean":"2023-08-11T15:51:06.843859+0000","last_became_active":"2023-08-11T15:51:00.738824+0000","last_became_peered":"2023-08-11T15:51:00.738824+0000","last_unstale":"2023-08-11T15:51:06.843859+0000","last_undegraded":"2023-08-11T15:51:06.843859+0000","last_fullsized":"2023-08-11T15:51:06.843859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574756+0000","last_change":"2023-08-11T15:51:05.852495+0000","last_active":"2023-08-11T15:51:08.574756+0000","last_peered":"2023-08-11T15:51:08.574756+0000","last_clean":"2023-08-11T15:51:08.574756+0000","last_became_active":"2023-08-11T15:51:05.852028+0000","last_became_peered":"2023-08-11T15:51:05.852028+0000","last_unstale":"2023-08-11T15:51:08.574756+0000","last_undegraded":"2023-08-11T15:51:08.574756+0000","last_fullsized":"2023-08-11T15:51:08.574756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:00:50.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030008+0000","last_change":"2023-08-11T15:51:01.746332+0000","last_active":"2023-08-11T15:51:08.030008+0000","last_peered":"2023-08-11T15:51:08.030008+0000","last_clean":"2023-08-11T15:51:08.030008+0000","last_became_active":"2023-08-11T15:51:01.746037+0000","last_became_peered":"2023-08-11T15:51:01.746037+0000","last_unstale":"2023-08-11T15:51:08.030008+0000","last_undegraded":"2023-08-11T15:51:08.030008+0000","last_fullsized":"2023-08-11T15:51:08.030008+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:01:53.294858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846006+0000","last_change":"2023-08-11T15:51:00.750241+0000","last_active":"2023-08-11T15:51:06.846006+0000","last_peered":"2023-08-11T15:51:06.846006+0000","last_clean":"2023-08-11T15:51:06.846006+0000","last_became_active":"2023-08-11T15:51:00.749526+0000","last_became_peered":"2023-08-11T15:51:00.749526+0000","last_unstale":"2023-08-11T15:51:06.846006+0000","last_undegraded":"2023-08-11T15:51:06.846006+0000","last_fullsized":"2023-08-11T15:51:06.846006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.749276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574297+0000","last_change":"2023-08-11T15:51:05.852589+0000","last_active":"2023-08-11T15:51:08.574297+0000","last_peered":"2023-08-11T15:51:08.574297+0000","last_clean":"2023-08-11T15:51:08.574297+0000","last_became_active":"2023-08-11T15:51:05.852123+0000","last_became_peered":"2023-08-11T15:51:05.852123+0000","last_unstale":"2023-08-11T15:51:08.574297+0000","last_undegraded":"2023-08-11T15:51:08.574297+0000","last_fullsized":"2023-08-11T15:51:08.574297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:52.516197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841591+0000","last_change":"2023-08-11T15:51:01.740461+0000","last_active":"2023-08-11T15:51:06.841591+0000","last_peered":"2023-08-11T15:51:06.841591+0000","last_clean":"2023-08-11T15:51:06.841591+0000","last_became_active":"2023-08-11T15:51:01.740152+0000","last_became_peered":"2023-08-11T15:51:01.740152+0000","last_unstale":"2023-08-11T15:51:06.841591+0000","last_undegraded":"2023-08-11T15:51:06.841591+0000","last_fullsized":"2023-08-11T15:51:06.841591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:09.551106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846676+0000","last_change":"2023-08-11T15:51:00.749804+0000","last_active":"2023-08-11T15:51:06.846676+0000","last_peered":"2023-08-11T15:51:06.846676+0000","last_clean":"2023-08-11T15:51:06.846676+0000","last_became_active":"2023-08-11T15:51:00.749397+0000","last_became_peered":"2023-08-11T15:51:00.749397+0000","last_unstale":"2023-08-11T15:51:06.846676+0000","last_undegraded":"2023-08-11T15:51:06.846676+0000","last_fullsized":"2023-08-11T15:51:06.846676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843683+0000","last_change":"2023-08-11T15:51:05.857250+0000","last_active":"2023-08-11T15:51:06.843683+0000","last_peered":"2023-08-11T15:51:06.843683+0000","last_clean":"2023-08-11T15:51:06.843683+0000","last_became_active":"2023-08-11T15:51:05.856843+0000","last_became_peered":"2023-08-11T15:51:05.856843+0000","last_unstale":"2023-08-11T15:51:06.843683+0000","last_undegraded":"2023-08-11T15:51:06.843683+0000","last_fullsized":"2023-08-11T15:51:06.843683+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:30:03.847116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846267+0000","last_change":"2023-08-11T15:51:05.849584+0000","last_active":"2023-08-11T15:51:06.846267+0000","last_peered":"2023-08-11T15:51:06.846267+0000","last_clean":"2023-08-11T15:51:06.846267+0000","last_became_active":"2023-08-11T15:51:05.849144+0000","last_became_peered":"2023-08-11T15:51:05.849144+0000","last_unstale":"2023-08-11T15:51:06.846267+0000","last_undegraded":"2023-08-11T15:51:06.846267+0000","last_fullsized":"2023-08-11T15:51:06.846267+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:19:51.734914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643988+0000","last_change":"2023-08-11T15:51:00.743098+0000","last_active":"2023-08-11T15:51:08.643988+0000","last_peered":"2023-08-11T15:51:08.643988+0000","last_clean":"2023-08-11T15:51:08.643988+0000","last_became_active":"2023-08-11T15:51:00.742606+0000","last_became_peered":"2023-08-11T15:51:00.742606+0000","last_unstale":"2023-08-11T15:51:08.643988+0000","last_undegraded":"2023-08-11T15:51:08.643988+0000","last_fullsized":"2023-08-11T15:51:08.643988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:38:17.546002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278685+0000","last_change":"2023-08-11T15:51:01.746170+0000","last_active":"2023-08-11T15:51:07.278685+0000","last_peered":"2023-08-11T15:51:07.278685+0000","last_clean":"2023-08-11T15:51:07.278685+0000","last_became_active":"2023-08-11T15:51:01.745672+0000","last_became_peered":"2023-08-11T15:51:01.745672+0000","last_unstale":"2023-08-11T15:51:07.278685+0000","last_undegraded":"2023-08-11T15:51:07.278685+0000","last_fullsized":"2023-08-11T15:51:07.278685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:20:27.436792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278398+0000","last_change":"2023-08-11T15:51:05.844332+0000","last_active":"2023-08-11T15:51:07.278398+0000","last_peered":"2023-08-11T15:51:07.278398+0000","last_clean":"2023-08-11T15:51:07.278398+0000","last_became_active":"2023-08-11T15:51:05.841336+0000","last_became_peered":"2023-08-11T15:51:05.841336+0000","last_unstale":"2023-08-11T15:51:07.278398+0000","last_undegraded":"2023-08-11T15:51:07.278398+0000","last_fullsized":"2023-08-11T15:51:07.278398+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:19:51.734914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844582+0000","last_change":"2023-08-11T15:51:00.750609+0000","last_active":"2023-08-11T15:51:06.844582+0000","last_peered":"2023-08-11T15:51:06.844582+0000","last_clean":"2023-08-11T15:51:06.844582+0000","last_became_active":"2023-08-11T15:51:00.750277+0000","last_became_peered":"2023-08-11T15:51:00.750277+0000","last_unstale":"2023-08-11T15:51:06.844582+0000","last_undegraded":"2023-08-11T15:51:06.844582+0000","last_fullsized":"2023-08-11T15:51:06.844582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:19.559486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841973+0000","last_change":"2023-08-11T15:51:01.750332+0000","last_active":"2023-08-11T15:51:06.841973+0000","last_peered":"2023-08-11T15:51:06.841973+0000","last_clean":"2023-08-11T15:51:06.841973+0000","last_became_active":"2023-08-11T15:51:01.749234+0000","last_became_peered":"2023-08-11T15:51:01.749234+0000","last_unstale":"2023-08-11T15:51:06.841973+0000","last_undegraded":"2023-08-11T15:51:06.841973+0000","last_fullsized":"2023-08-11T15:51:06.841973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:08.926778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839995+0000","last_change":"2023-08-11T15:51:00.745865+0000","last_active":"2023-08-11T15:51:06.839995+0000","last_peered":"2023-08-11T15:51:06.839995+0000","last_clean":"2023-08-11T15:51:06.839995+0000","last_became_active":"2023-08-11T15:51:00.745469+0000","last_became_peered":"2023-08-11T15:51:00.745469+0000","last_unstale":"2023-08-11T15:51:06.839995+0000","last_undegraded":"2023-08-11T15:51:06.839995+0000","last_fullsized":"2023-08-11T15:51:06.839995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:25:15.827962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845118+0000","last_change":"2023-08-11T15:51:01.740388+0000","last_active":"2023-08-11T15:51:06.845118+0000","last_peered":"2023-08-11T15:51:06.845118+0000","last_clean":"2023-08-11T15:51:06.845118+0000","last_became_active":"2023-08-11T15:51:01.739972+0000","last_became_peered":"2023-08-11T15:51:01.739972+0000","last_unstale":"2023-08-11T15:51:06.845118+0000","last_undegraded":"2023-08-11T15:51:06.845118+0000","last_fullsized":"2023-08-11T15:51:06.845118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:41.231851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277756+0000","last_change":"2023-08-11T15:51:05.847377+0000","last_active":"2023-08-11T15:51:07.277756+0000","last_peered":"2023-08-11T15:51:07.277756+0000","last_clean":"2023-08-11T15:51:07.277756+0000","last_became_active":"2023-08-11T15:51:05.843902+0000","last_became_peered":"2023-08-11T15:51:05.843902+0000","last_unstale":"2023-08-11T15:51:07.277756+0000","last_undegraded":"2023-08-11T15:51:07.277756+0000","last_fullsized":"2023-08-11T15:51:07.277756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:45.327626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843318+0000","last_change":"2023-08-11T15:51:00.753432+0000","last_active":"2023-08-11T15:51:06.843318+0000","last_peered":"2023-08-11T15:51:06.843318+0000","last_clean":"2023-08-11T15:51:06.843318+0000","last_became_active":"2023-08-11T15:51:00.753099+0000","last_became_peered":"2023-08-11T15:51:00.753099+0000","last_unstale":"2023-08-11T15:51:06.843318+0000","last_undegraded":"2023-08-11T15:51:06.843318+0000","last_fullsized":"2023-08-11T15:51:06.843318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:44.445917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278374+0000","last_change":"2023-08-11T15:51:01.754975+0000","last_active":"2023-08-11T15:51:07.278374+0000","last_peered":"2023-08-11T15:51:07.278374+0000","last_clean":"2023-08-11T15:51:07.278374+0000","last_became_active":"2023-08-11T15:51:01.754617+0000","last_became_peered":"2023-08-11T15:51:01.754617+0000","last_unstale":"2023-08-11T15:51:07.278374+0000","last_undegraded":"2023-08-11T15:51:07.278374+0000","last_fullsized":"2023-08-11T15:51:07.278374+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:20.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844645+0000","last_change":"2023-08-11T15:51:05.841963+0000","last_active":"2023-08-11T15:51:06.844645+0000","last_peered":"2023-08-11T15:51:06.844645+0000","last_clean":"2023-08-11T15:51:06.844645+0000","last_became_active":"2023-08-11T15:51:05.841100+0000","last_became_peered":"2023-08-11T15:51:05.841100+0000","last_unstale":"2023-08-11T15:51:06.844645+0000","last_undegraded":"2023-08-11T15:51:06.844645+0000","last_fullsized":"2023-08-11T15:51:06.844645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:49.553888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643596+0000","last_change":"2023-08-11T15:51:01.740691+0000","last_active":"2023-08-11T15:51:08.643596+0000","last_peered":"2023-08-11T15:51:08.643596+0000","last_clean":"2023-08-11T15:51:08.643596+0000","last_became_active":"2023-08-11T15:51:01.740411+0000","last_became_peered":"2023-08-11T15:51:01.740411+0000","last_unstale":"2023-08-11T15:51:08.643596+0000","last_undegraded":"2023-08-11T15:51:08.643596+0000","last_fullsized":"2023-08-11T15:51:08.643596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:14:24.513369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276978+0000","last_change":"2023-08-11T15:51:01.754967+0000","last_active":"2023-08-11T15:51:07.276978+0000","last_peered":"2023-08-11T15:51:07.276978+0000","last_clean":"2023-08-11T15:51:07.276978+0000","last_became_active":"2023-08-11T15:51:01.754670+0000","last_became_peered":"2023-08-11T15:51:01.754670+0000","last_unstale":"2023-08-11T15:51:07.276978+0000","last_undegraded":"2023-08-11T15:51:07.276978+0000","last_fullsized":"2023-08-11T15:51:07.276978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029518+0000","last_change":"2023-08-11T15:51:01.743911+0000","last_active":"2023-08-11T15:51:08.029518+0000","last_peered":"2023-08-11T15:51:08.029518+0000","last_clean":"2023-08-11T15:51:08.029518+0000","last_became_active":"2023-08-11T15:51:01.743608+0000","last_became_peered":"2023-08-11T15:51:01.743608+0000","last_unstale":"2023-08-11T15:51:08.029518+0000","last_undegraded":"2023-08-11T15:51:08.029518+0000","last_fullsized":"2023-08-11T15:51:08.029518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:45:30.132181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277768+0000","last_change":"2023-08-11T15:51:01.748054+0000","last_active":"2023-08-11T15:51:07.277768+0000","last_peered":"2023-08-11T15:51:07.277768+0000","last_clean":"2023-08-11T15:51:07.277768+0000","last_became_active":"2023-08-11T15:51:01.747751+0000","last_became_peered":"2023-08-11T15:51:01.747751+0000","last_unstale":"2023-08-11T15:51:07.277768+0000","last_undegraded":"2023-08-11T15:51:07.277768+0000","last_fullsized":"2023-08-11T15:51:07.277768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:33.655411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450215+0000","last_change":"2023-08-11T15:51:01.745398+0000","last_active":"2023-08-11T15:51:07.450215+0000","last_peered":"2023-08-11T15:51:07.450215+0000","last_clean":"2023-08-11T15:51:07.450215+0000","last_became_active":"2023-08-11T15:51:01.745013+0000","last_became_peered":"2023-08-11T15:51:01.745013+0000","last_unstale":"2023-08-11T15:51:07.450215+0000","last_undegraded":"2023-08-11T15:51:07.450215+0000","last_fullsized":"2023-08-11T15:51:07.450215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:37:45.341896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844793+0000","last_change":"2023-08-11T15:51:01.747243+0000","last_active":"2023-08-11T15:51:06.844793+0000","last_peered":"2023-08-11T15:51:06.844793+0000","last_clean":"2023-08-11T15:51:06.844793+0000","last_became_active":"2023-08-11T15:51:01.746905+0000","last_became_peered":"2023-08-11T15:51:01.746905+0000","last_unstale":"2023-08-11T15:51:06.844793+0000","last_undegraded":"2023-08-11T15:51:06.844793+0000","last_fullsized":"2023-08-11T15:51:06.844793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450600+0000","last_change":"2023-08-11T15:51:01.743655+0000","last_active":"2023-08-11T15:51:07.450600+0000","last_peered":"2023-08-11T15:51:07.450600+0000","last_clean":"2023-08-11T15:51:07.450600+0000","last_became_active":"2023-08-11T15:51:01.743308+0000","last_became_peered":"2023-08-11T15:51:01.743308+0000","last_unstale":"2023-08-11T15:51:07.450600+0000","last_undegraded":"2023-08-11T15:51:07.450600+0000","last_fullsized":"2023-08-11T15:51:07.450600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:08:59.509573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450686+0000","last_change":"2023-08-11T15:51:01.747270+0000","last_active":"2023-08-11T15:51:07.450686+0000","last_peered":"2023-08-11T15:51:07.450686+0000","last_clean":"2023-08-11T15:51:07.450686+0000","last_became_active":"2023-08-11T15:51:01.746924+0000","last_became_peered":"2023-08-11T15:51:01.746924+0000","last_unstale":"2023-08-11T15:51:07.450686+0000","last_undegraded":"2023-08-11T15:51:07.450686+0000","last_fullsized":"2023-08-11T15:51:07.450686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:07:01.100072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842527+0000","last_change":"2023-08-11T15:51:01.750707+0000","last_active":"2023-08-11T15:51:06.842527+0000","last_peered":"2023-08-11T15:51:06.842527+0000","last_clean":"2023-08-11T15:51:06.842527+0000","last_became_active":"2023-08-11T15:51:01.750243+0000","last_became_peered":"2023-08-11T15:51:01.750243+0000","last_unstale":"2023-08-11T15:51:06.842527+0000","last_undegraded":"2023-08-11T15:51:06.842527+0000","last_fullsized":"2023-08-11T15:51:06.842527+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276812+0000","last_change":"2023-08-11T15:51:01.744825+0000","last_active":"2023-08-11T15:51:07.276812+0000","last_peered":"2023-08-11T15:51:07.276812+0000","last_clean":"2023-08-11T15:51:07.276812+0000","last_became_active":"2023-08-11T15:51:01.744446+0000","last_became_peered":"2023-08-11T15:51:01.744446+0000","last_unstale":"2023-08-11T15:51:07.276812+0000","last_undegraded":"2023-08-11T15:51:07.276812+0000","last_fullsized":"2023-08-11T15:51:07.276812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:08.520981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839890+0000","last_change":"2023-08-11T15:51:01.754226+0000","last_active":"2023-08-11T15:51:06.839890+0000","last_peered":"2023-08-11T15:51:06.839890+0000","last_clean":"2023-08-11T15:51:06.839890+0000","last_became_active":"2023-08-11T15:51:01.753864+0000","last_became_peered":"2023-08-11T15:51:01.753864+0000","last_unstale":"2023-08-11T15:51:06.839890+0000","last_undegraded":"2023-08-11T15:51:06.839890+0000","last_fullsized":"2023-08-11T15:51:06.839890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:42:42.499555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843198+0000","last_change":"2023-08-11T15:51:01.751698+0000","last_active":"2023-08-11T15:51:06.843198+0000","last_peered":"2023-08-11T15:51:06.843198+0000","last_clean":"2023-08-11T15:51:06.843198+0000","last_became_active":"2023-08-11T15:51:01.751183+0000","last_became_peered":"2023-08-11T15:51:01.751183+0000","last_unstale":"2023-08-11T15:51:06.843198+0000","last_undegraded":"2023-08-11T15:51:06.843198+0000","last_fullsized":"2023-08-11T15:51:06.843198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029934+0000","last_change":"2023-08-11T15:51:01.743064+0000","last_active":"2023-08-11T15:51:08.029934+0000","last_peered":"2023-08-11T15:51:08.029934+0000","last_clean":"2023-08-11T15:51:08.029934+0000","last_became_active":"2023-08-11T15:51:01.742783+0000","last_became_peered":"2023-08-11T15:51:01.742783+0000","last_unstale":"2023-08-11T15:51:08.029934+0000","last_undegraded":"2023-08-11T15:51:08.029934+0000","last_fullsized":"2023-08-11T15:51:08.029934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:25:56.529094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574852+0000","last_change":"2023-08-11T15:51:01.746080+0000","last_active":"2023-08-11T15:51:08.574852+0000","last_peered":"2023-08-11T15:51:08.574852+0000","last_clean":"2023-08-11T15:51:08.574852+0000","last_became_active":"2023-08-11T15:51:01.745366+0000","last_became_peered":"2023-08-11T15:51:01.745366+0000","last_unstale":"2023-08-11T15:51:08.574852+0000","last_undegraded":"2023-08-11T15:51:08.574852+0000","last_fullsized":"2023-08-11T15:51:08.574852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:58:33.514033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030489+0000","last_change":"2023-08-11T15:51:01.735183+0000","last_active":"2023-08-11T15:51:08.030489+0000","last_peered":"2023-08-11T15:51:08.030489+0000","last_clean":"2023-08-11T15:51:08.030489+0000","last_became_active":"2023-08-11T15:51:01.734771+0000","last_became_peered":"2023-08-11T15:51:01.734771+0000","last_unstale":"2023-08-11T15:51:08.030489+0000","last_undegraded":"2023-08-11T15:51:08.030489+0000","last_fullsized":"2023-08-11T15:51:08.030489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:28:21.255388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842288+0000","last_change":"2023-08-11T15:51:01.753111+0000","last_active":"2023-08-11T15:51:06.842288+0000","last_peered":"2023-08-11T15:51:06.842288+0000","last_clean":"2023-08-11T15:51:06.842288+0000","last_became_active":"2023-08-11T15:51:01.752792+0000","last_became_peered":"2023-08-11T15:51:01.752792+0000","last_unstale":"2023-08-11T15:51:06.842288+0000","last_undegraded":"2023-08-11T15:51:06.842288+0000","last_fullsized":"2023-08-11T15:51:06.842288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028692+0000","last_change":"2023-08-11T15:51:01.737005+0000","last_active":"2023-08-11T15:51:08.028692+0000","last_peered":"2023-08-11T15:51:08.028692+0000","last_clean":"2023-08-11T15:51:08.028692+0000","last_became_active":"2023-08-11T15:51:01.736669+0000","last_became_peered":"2023-08-11T15:51:01.736669+0000","last_unstale":"2023-08-11T15:51:08.028692+0000","last_undegraded":"2023-08-11T15:51:08.028692+0000","last_fullsized":"2023-08-11T15:51:08.028692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:10:00.491572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844298+0000","last_change":"2023-08-11T15:51:01.755893+0000","last_active":"2023-08-11T15:51:06.844298+0000","last_peered":"2023-08-11T15:51:06.844298+0000","last_clean":"2023-08-11T15:51:06.844298+0000","last_became_active":"2023-08-11T15:51:01.755182+0000","last_became_peered":"2023-08-11T15:51:01.755182+0000","last_unstale":"2023-08-11T15:51:06.844298+0000","last_undegraded":"2023-08-11T15:51:06.844298+0000","last_fullsized":"2023-08-11T15:51:06.844298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:03.472522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842189+0000","last_change":"2023-08-11T15:51:01.739785+0000","last_active":"2023-08-11T15:51:06.842189+0000","last_peered":"2023-08-11T15:51:06.842189+0000","last_clean":"2023-08-11T15:51:06.842189+0000","last_became_active":"2023-08-11T15:51:01.739500+0000","last_became_peered":"2023-08-11T15:51:01.739500+0000","last_unstale":"2023-08-11T15:51:06.842189+0000","last_undegraded":"2023-08-11T15:51:06.842189+0000","last_fullsized":"2023-08-11T15:51:06.842189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574594+0000","last_change":"2023-08-11T15:51:01.732565+0000","last_active":"2023-08-11T15:51:08.574594+0000","last_peered":"2023-08-11T15:51:08.574594+0000","last_clean":"2023-08-11T15:51:08.574594+0000","last_became_active":"2023-08-11T15:51:01.732076+0000","last_became_peered":"2023-08-11T15:51:01.732076+0000","last_unstale":"2023-08-11T15:51:08.574594+0000","last_undegraded":"2023-08-11T15:51:08.574594+0000","last_fullsized":"2023-08-11T15:51:08.574594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:20.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575381+0000","last_change":"2023-08-11T15:51:01.736603+0000","last_active":"2023-08-11T15:51:08.575381+0000","last_peered":"2023-08-11T15:51:08.575381+0000","last_clean":"2023-08-11T15:51:08.575381+0000","last_became_active":"2023-08-11T15:51:01.735835+0000","last_became_peered":"2023-08-11T15:51:01.735835+0000","last_unstale":"2023-08-11T15:51:08.575381+0000","last_undegraded":"2023-08-11T15:51:08.575381+0000","last_fullsized":"2023-08-11T15:51:08.575381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:07.026259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030580+0000","last_change":"2023-08-11T15:51:01.752520+0000","last_active":"2023-08-11T15:51:08.030580+0000","last_peered":"2023-08-11T15:51:08.030580+0000","last_clean":"2023-08-11T15:51:08.030580+0000","last_became_active":"2023-08-11T15:51:01.752203+0000","last_became_peered":"2023-08-11T15:51:01.752203+0000","last_unstale":"2023-08-11T15:51:08.030580+0000","last_undegraded":"2023-08-11T15:51:08.030580+0000","last_fullsized":"2023-08-11T15:51:08.030580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:29:45.923242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845432+0000","last_change":"2023-08-11T15:51:01.737002+0000","last_active":"2023-08-11T15:51:06.845432+0000","last_peered":"2023-08-11T15:51:06.845432+0000","last_clean":"2023-08-11T15:51:06.845432+0000","last_became_active":"2023-08-11T15:51:01.736720+0000","last_became_peered":"2023-08-11T15:51:01.736720+0000","last_unstale":"2023-08-11T15:51:06.845432+0000","last_undegraded":"2023-08-11T15:51:06.845432+0000","last_fullsized":"2023-08-11T15:51:06.845432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:20.571682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841857+0000","last_change":"2023-08-11T15:51:01.752888+0000","last_active":"2023-08-11T15:51:06.841857+0000","last_peered":"2023-08-11T15:51:06.841857+0000","last_clean":"2023-08-11T15:51:06.841857+0000","last_became_active":"2023-08-11T15:51:01.752593+0000","last_became_peered":"2023-08-11T15:51:01.752593+0000","last_unstale":"2023-08-11T15:51:06.841857+0000","last_undegraded":"2023-08-11T15:51:06.841857+0000","last_fullsized":"2023-08-11T15:51:06.841857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:39:00.899813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841369+0000","last_change":"2023-08-11T15:51:01.741200+0000","last_active":"2023-08-11T15:51:06.841369+0000","last_peered":"2023-08-11T15:51:06.841369+0000","last_clean":"2023-08-11T15:51:06.841369+0000","last_became_active":"2023-08-11T15:51:01.738593+0000","last_became_peered":"2023-08-11T15:51:01.738593+0000","last_unstale":"2023-08-11T15:51:06.841369+0000","last_undegraded":"2023-08-11T15:51:06.841369+0000","last_fullsized":"2023-08-11T15:51:06.841369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:02:09.390191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844003+0000","last_change":"2023-08-11T15:51:01.756072+0000","last_active":"2023-08-11T15:51:06.844003+0000","last_peered":"2023-08-11T15:51:06.844003+0000","last_clean":"2023-08-11T15:51:06.844003+0000","last_became_active":"2023-08-11T15:51:01.755522+0000","last_became_peered":"2023-08-11T15:51:01.755522+0000","last_unstale":"2023-08-11T15:51:06.844003+0000","last_undegraded":"2023-08-11T15:51:06.844003+0000","last_fullsized":"2023-08-11T15:51:06.844003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:41.231851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029178+0000","last_change":"2023-08-11T15:51:01.737005+0000","last_active":"2023-08-11T15:51:08.029178+0000","last_peered":"2023-08-11T15:51:08.029178+0000","last_clean":"2023-08-11T15:51:08.029178+0000","last_became_active":"2023-08-11T15:51:01.736657+0000","last_became_peered":"2023-08-11T15:51:01.736657+0000","last_unstale":"2023-08-11T15:51:08.029178+0000","last_undegraded":"2023-08-11T15:51:08.029178+0000","last_fullsized":"2023-08-11T15:51:08.029178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:20:52.818361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841216+0000","last_change":"2023-08-11T15:51:01.749800+0000","last_active":"2023-08-11T15:51:06.841216+0000","last_peered":"2023-08-11T15:51:06.841216+0000","last_clean":"2023-08-11T15:51:06.841216+0000","last_became_active":"2023-08-11T15:51:01.749436+0000","last_became_peered":"2023-08-11T15:51:01.749436+0000","last_unstale":"2023-08-11T15:51:06.841216+0000","last_undegraded":"2023-08-11T15:51:06.841216+0000","last_fullsized":"2023-08-11T15:51:06.841216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644342+0000","last_change":"2023-08-11T15:51:01.739883+0000","last_active":"2023-08-11T15:51:08.644342+0000","last_peered":"2023-08-11T15:51:08.644342+0000","last_clean":"2023-08-11T15:51:08.644342+0000","last_became_active":"2023-08-11T15:51:01.739490+0000","last_became_peered":"2023-08-11T15:51:01.739490+0000","last_unstale":"2023-08-11T15:51:08.644342+0000","last_undegraded":"2023-08-11T15:51:08.644342+0000","last_fullsized":"2023-08-11T15:51:08.644342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:19.428228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030296+0000","last_change":"2023-08-11T15:51:01.737093+0000","last_active":"2023-08-11T15:51:08.030296+0000","last_peered":"2023-08-11T15:51:08.030296+0000","last_clean":"2023-08-11T15:51:08.030296+0000","last_became_active":"2023-08-11T15:51:01.736803+0000","last_became_peered":"2023-08-11T15:51:01.736803+0000","last_unstale":"2023-08-11T15:51:08.030296+0000","last_undegraded":"2023-08-11T15:51:08.030296+0000","last_fullsized":"2023-08-11T15:51:08.030296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:50:46.864696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844939+0000","last_change":"2023-08-11T15:51:01.753163+0000","last_active":"2023-08-11T15:51:06.844939+0000","last_peered":"2023-08-11T15:51:06.844939+0000","last_clean":"2023-08-11T15:51:06.844939+0000","last_became_active":"2023-08-11T15:51:01.752836+0000","last_became_peered":"2023-08-11T15:51:01.752836+0000","last_unstale":"2023-08-11T15:51:06.844939+0000","last_undegraded":"2023-08-11T15:51:06.844939+0000","last_fullsized":"2023-08-11T15:51:06.844939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:15.195226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575171+0000","last_change":"2023-08-11T15:51:01.739059+0000","last_active":"2023-08-11T15:51:08.575171+0000","last_peered":"2023-08-11T15:51:08.575171+0000","last_clean":"2023-08-11T15:51:08.575171+0000","last_became_active":"2023-08-11T15:51:01.738739+0000","last_became_peered":"2023-08-11T15:51:01.738739+0000","last_unstale":"2023-08-11T15:51:08.575171+0000","last_undegraded":"2023-08-11T15:51:08.575171+0000","last_fullsized":"2023-08-11T15:51:08.575171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:11:40.167154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":921600,"data_stored":717480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]}]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":123848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.827}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.917}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1219999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1629999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":319488,"data_stored":126952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":123286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5469999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7070000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.605}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5369999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":897024,"data_stored":714146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:51:27.366 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:51:27.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.632+0000 7f5ff7857640 1 Processor -- start 2023-08-11T15:51:27.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.632+0000 7f5ff7857640 1 -- start start 2023-08-11T15:51:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.633+0000 7f5ff7857640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ff0160d20 0x7f5ff00cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.633+0000 7f5ff7857640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5ff00cc350 0x7f5ff015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.633+0000 7f5ff7857640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ff015d1e0 0x7f5ff006b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff7857640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff00cc7d0 con 0x7f5ff00cc350 2023-08-11T15:51:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff7857640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff0156fa0 con 0x7f5ff015d1e0 2023-08-11T15:51:27.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff7857640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff015da30 con 0x7f5ff0160d20 2023-08-11T15:51:27.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff4df2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5ff00cc350 0x7f5ff015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff4df2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5ff00cc350 0x7f5ff015cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46314/0 (socket says 172.21.15.27:46314) 2023-08-11T15:51:27.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff4df2640 1 -- 172.21.15.27:0/2199166461 learned_addr learned my addr 172.21.15.27:0/2199166461 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:27.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff55f3640 1 --2- 172.21.15.27:0/2199166461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ff0160d20 0x7f5ff00cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff5df4640 1 --2- 172.21.15.27:0/2199166461 >> v2:172.21.15.119:3300/0 conn(0x7f5ff015d1e0 0x7f5ff006b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff4df2640 1 -- 172.21.15.27:0/2199166461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ff0160d20 msgr2=0x7f5ff00cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:27.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff4df2640 1 --2- 172.21.15.27:0/2199166461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ff0160d20 0x7f5ff00cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff4df2640 1 -- 172.21.15.27:0/2199166461 >> v2:172.21.15.119:3300/0 conn(0x7f5ff015d1e0 msgr2=0x7f5ff006b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:27.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff4df2640 1 --2- 172.21.15.27:0/2199166461 >> v2:172.21.15.119:3300/0 conn(0x7f5ff015d1e0 0x7f5ff006b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.634+0000 7f5ff4df2640 1 -- 172.21.15.27:0/2199166461 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00ca570 con 0x7f5ff00cc350 2023-08-11T15:51:27.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.636+0000 7f5ff4df2640 1 --2- 172.21.15.27:0/2199166461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 0x7f5ff015cca0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0066b00 tx=0x7f5fe002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4286fd14502617 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:27.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.636+0000 7f5fe7fff640 1 -- 172.21.15.27:0/2199166461 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0031870 con 0x7f5ff00cc350 2023-08-11T15:51:27.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.636+0000 7f5fe7fff640 1 -- 172.21.15.27:0/2199166461 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe00319d0 con 0x7f5ff00cc350 2023-08-11T15:51:27.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.636+0000 7f5fe7fff640 1 -- 172.21.15.27:0/2199166461 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0031d60 con 0x7f5ff00cc350 2023-08-11T15:51:27.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.636+0000 7f5ff7857640 1 -- 172.21.15.27:0/2199166461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 msgr2=0x7f5ff015cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.636+0000 7f5ff7857640 1 --2- 172.21.15.27:0/2199166461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 0x7f5ff015cca0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0066b00 tx=0x7f5fe002b910 comp rx=0 tx=0).stop 2023-08-11T15:51:27.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.637+0000 7f5ff7857640 1 -- 172.21.15.27:0/2199166461 shutdown_connections 2023-08-11T15:51:27.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.637+0000 7f5ff7857640 1 --2- 172.21.15.27:0/2199166461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 0x7f5ff015cca0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.637+0000 7f5ff7857640 1 --2- 172.21.15.27:0/2199166461 >> v2:172.21.15.119:3300/0 conn(0x7f5ff015d1e0 0x7f5ff006b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.637+0000 7f5ff7857640 1 --2- 172.21.15.27:0/2199166461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ff0160d20 0x7f5ff00cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.637+0000 7f5ff7857640 1 -- 172.21.15.27:0/2199166461 >> 172.21.15.27:0/2199166461 conn(0x7f5ff00afb70 msgr2=0x7f5ff00aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:27.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.637+0000 7f5ff7857640 1 -- 172.21.15.27:0/2199166461 shutdown_connections 2023-08-11T15:51:27.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.637+0000 7f5ff7857640 1 -- 172.21.15.27:0/2199166461 wait complete. 2023-08-11T15:51:27.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.638+0000 7f5ff7857640 1 Processor -- start 2023-08-11T15:51:27.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.639+0000 7f5ff7857640 1 -- start start 2023-08-11T15:51:27.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.639+0000 7f5ff7857640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 0x7f5ff00c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.639+0000 7f5ff7857640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff015d1e0 0x7f5ff0159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.639+0000 7f5ff7857640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ff0160d20 0x7f5ff00af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.639+0000 7f5ff7857640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00c3340 con 0x7f5ff00cc350 2023-08-11T15:51:27.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.639+0000 7f5ff7857640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00c3620 con 0x7f5ff015d1e0 2023-08-11T15:51:27.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.639+0000 7f5ff7857640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00c3900 con 0x7f5ff0160d20 2023-08-11T15:51:27.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.640+0000 7f5ff55f3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 0x7f5ff00c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.640+0000 7f5ff55f3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 0x7f5ff00c43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46324/0 (socket says 172.21.15.27:46324) 2023-08-11T15:51:27.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.640+0000 7f5ff55f3640 1 -- 172.21.15.27:0/953150807 learned_addr learned my addr 172.21.15.27:0/953150807 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:27.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.640+0000 7f5ff5df4640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ff0160d20 0x7f5ff00af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.640+0000 7f5ff4df2640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff015d1e0 0x7f5ff0159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.640+0000 7f5ff55f3640 1 -- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ff0160d20 msgr2=0x7f5ff00af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.640+0000 7f5ff55f3640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ff0160d20 0x7f5ff00af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.640+0000 7f5ff55f3640 1 -- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff015d1e0 msgr2=0x7f5ff0159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.640+0000 7f5ff55f3640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff015d1e0 0x7f5ff0159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.640+0000 7f5ff55f3640 1 -- 172.21.15.27:0/953150807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe00094d0 con 0x7f5ff00cc350 2023-08-11T15:51:27.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.641+0000 7f5ff55f3640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 0x7f5ff00c43c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00d7d0 tx=0x7f5fec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:27.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.641+0000 7f5fe5ffb640 1 -- 172.21.15.27:0/953150807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0096c0 con 0x7f5ff00cc350 2023-08-11T15:51:27.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.641+0000 7f5fe5ffb640 1 -- 172.21.15.27:0/953150807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec009820 con 0x7f5ff00cc350 2023-08-11T15:51:27.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.641+0000 7f5fe5ffb640 1 -- 172.21.15.27:0/953150807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec009bd0 con 0x7f5ff00cc350 2023-08-11T15:51:27.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.641+0000 7f5ff7857640 1 -- 172.21.15.27:0/953150807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff00c3be0 con 0x7f5ff00cc350 2023-08-11T15:51:27.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.641+0000 7f5ff7857640 1 -- 172.21.15.27:0/953150807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff00c4900 con 0x7f5ff00cc350 2023-08-11T15:51:27.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.642+0000 7f5ff7857640 1 -- 172.21.15.27:0/953150807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff0066060 con 0x7f5ff00cc350 2023-08-11T15:51:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.643+0000 7f5fe5ffb640 1 -- 172.21.15.27:0/953150807 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5fec009d30 con 0x7f5ff00cc350 2023-08-11T15:51:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.644+0000 7f5fe5ffb640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5fd406a050 0x7f5fd406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.644+0000 7f5fe5ffb640 1 -- 172.21.15.27:0/953150807 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5fec089160 con 0x7f5ff00cc350 2023-08-11T15:51:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.645+0000 7f5ff4df2640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5fd406a050 0x7f5fd406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:27.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.646+0000 7f5ff4df2640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5fd406a050 0x7f5fd406c510 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0036000 tx=0x7f5fe003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:27.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.648+0000 7f5fe5ffb640 1 -- 172.21.15.27:0/953150807 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f5fec057f50 con 0x7f5ff00cc350 2023-08-11T15:51:27.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.845+0000 7f5ff7857640 1 -- 172.21.15.27:0/953150807 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ff00cab30 con 0x7f5fd406a050 2023-08-11T15:51:27.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.875+0000 7f5fe5ffb640 1 -- 172.21.15.27:0/953150807 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367129 (secure 0 0 0) 0x7f5fe0044400 con 0x7f5fd406a050 2023-08-11T15:51:27.880 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:27.881 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:51:27.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.881+0000 7f5ff7857640 1 -- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5fd406a050 msgr2=0x7f5fd406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.881+0000 7f5ff7857640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5fd406a050 0x7f5fd406c510 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0036000 tx=0x7f5fe003c040 comp rx=0 tx=0).stop 2023-08-11T15:51:27.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.881+0000 7f5ff7857640 1 -- 172.21.15.27:0/953150807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 msgr2=0x7f5ff00c43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:27.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.881+0000 7f5ff7857640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 0x7f5ff00c43c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00d7d0 tx=0x7f5fec00dc90 comp rx=0 tx=0).stop 2023-08-11T15:51:27.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.881+0000 7f5ff7857640 1 -- 172.21.15.27:0/953150807 shutdown_connections 2023-08-11T15:51:27.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.881+0000 7f5ff7857640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5fd406a050 0x7f5fd406c510 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.881+0000 7f5ff7857640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ff0160d20 0x7f5ff00af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.881+0000 7f5ff7857640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff015d1e0 0x7f5ff0159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.881+0000 7f5ff7857640 1 --2- 172.21.15.27:0/953150807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff00cc350 0x7f5ff00c43c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:27.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.881+0000 7f5ff7857640 1 -- 172.21.15.27:0/953150807 >> 172.21.15.27:0/953150807 conn(0x7f5ff00afb70 msgr2=0x7f5ff00cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:27.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.882+0000 7f5ff7857640 1 -- 172.21.15.27:0/953150807 shutdown_connections 2023-08-11T15:51:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:27.882+0000 7f5ff7857640 1 -- 172.21.15.27:0/953150807 wait complete. 2023-08-11T15:51:27.903 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-08-11T15:51:26.719348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232028,"kb_used_data":3560,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754742692,"statfs":{"total":773094113280,"available":772856516608,"internally_reserved":0,"allocated":3645440,"data_stored":2175300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005024"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449971+0000","last_change":"2023-08-11T15:51:05.843998+0000","last_active":"2023-08-11T15:51:07.449971+0000","last_peered":"2023-08-11T15:51:07.449971+0000","last_clean":"2023-08-11T15:51:07.449971+0000","last_became_active":"2023-08-11T15:51:05.843613+0000","last_became_peered":"2023-08-11T15:51:05.843613+0000","last_unstale":"2023-08-11T15:51:07.449971+0000","last_undegraded":"2023-08-11T15:51:07.449971+0000","last_fullsized":"2023-08-11T15:51:07.449971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:35:45.480000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842354+0000","last_change":"2023-08-11T15:51:00.746010+0000","last_active":"2023-08-11T15:51:06.842354+0000","last_peered":"2023-08-11T15:51:06.842354+0000","last_clean":"2023-08-11T15:51:06.842354+0000","last_became_active":"2023-08-11T15:51:00.745732+0000","last_became_peered":"2023-08-11T15:51:00.745732+0000","last_unstale":"2023-08-11T15:51:06.842354+0000","last_undegraded":"2023-08-11T15:51:06.842354+0000","last_fullsized":"2023-08-11T15:51:06.842354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:02.960249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028896+0000","last_change":"2023-08-11T15:51:01.746812+0000","last_active":"2023-08-11T15:51:08.028896+0000","last_peered":"2023-08-11T15:51:08.028896+0000","last_clean":"2023-08-11T15:51:08.028896+0000","last_became_active":"2023-08-11T15:51:01.745729+0000","last_became_peered":"2023-08-11T15:51:01.745729+0000","last_unstale":"2023-08-11T15:51:08.028896+0000","last_undegraded":"2023-08-11T15:51:08.028896+0000","last_fullsized":"2023-08-11T15:51:08.028896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:46.849896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028532+0000","last_change":"2023-08-11T15:51:05.846118+0000","last_active":"2023-08-11T15:51:08.028532+0000","last_peered":"2023-08-11T15:51:08.028532+0000","last_clean":"2023-08-11T15:51:08.028532+0000","last_became_active":"2023-08-11T15:51:05.845703+0000","last_became_peered":"2023-08-11T15:51:05.845703+0000","last_unstale":"2023-08-11T15:51:08.028532+0000","last_undegraded":"2023-08-11T15:51:08.028532+0000","last_fullsized":"2023-08-11T15:51:08.028532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:15.472354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278289+0000","last_change":"2023-08-11T15:51:00.732689+0000","last_active":"2023-08-11T15:51:07.278289+0000","last_peered":"2023-08-11T15:51:07.278289+0000","last_clean":"2023-08-11T15:51:07.278289+0000","last_became_active":"2023-08-11T15:51:00.732185+0000","last_became_peered":"2023-08-11T15:51:00.732185+0000","last_unstale":"2023-08-11T15:51:07.278289+0000","last_undegraded":"2023-08-11T15:51:07.278289+0000","last_fullsized":"2023-08-11T15:51:07.278289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:19.559486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277505+0000","last_change":"2023-08-11T15:51:01.740441+0000","last_active":"2023-08-11T15:51:07.277505+0000","last_peered":"2023-08-11T15:51:07.277505+0000","last_clean":"2023-08-11T15:51:07.277505+0000","last_became_active":"2023-08-11T15:51:01.740104+0000","last_became_peered":"2023-08-11T15:51:01.740104+0000","last_unstale":"2023-08-11T15:51:07.277505+0000","last_undegraded":"2023-08-11T15:51:07.277505+0000","last_fullsized":"2023-08-11T15:51:07.277505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842536+0000","last_change":"2023-08-11T15:51:05.850642+0000","last_active":"2023-08-11T15:51:06.842536+0000","last_peered":"2023-08-11T15:51:06.842536+0000","last_clean":"2023-08-11T15:51:06.842536+0000","last_became_active":"2023-08-11T15:51:05.850276+0000","last_became_peered":"2023-08-11T15:51:05.850276+0000","last_unstale":"2023-08-11T15:51:06.842536+0000","last_undegraded":"2023-08-11T15:51:06.842536+0000","last_fullsized":"2023-08-11T15:51:06.842536+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:06:02.062687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845184+0000","last_change":"2023-08-11T15:51:00.753395+0000","last_active":"2023-08-11T15:51:06.845184+0000","last_peered":"2023-08-11T15:51:06.845184+0000","last_clean":"2023-08-11T15:51:06.845184+0000","last_became_active":"2023-08-11T15:51:00.753094+0000","last_became_peered":"2023-08-11T15:51:00.753094+0000","last_unstale":"2023-08-11T15:51:06.845184+0000","last_undegraded":"2023-08-11T15:51:06.845184+0000","last_fullsized":"2023-08-11T15:51:06.845184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:32.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029427+0000","last_change":"2023-08-11T15:51:01.748176+0000","last_active":"2023-08-11T15:51:08.029427+0000","last_peered":"2023-08-11T15:51:08.029427+0000","last_clean":"2023-08-11T15:51:08.029427+0000","last_became_active":"2023-08-11T15:51:01.747880+0000","last_became_peered":"2023-08-11T15:51:01.747880+0000","last_unstale":"2023-08-11T15:51:08.029427+0000","last_undegraded":"2023-08-11T15:51:08.029427+0000","last_fullsized":"2023-08-11T15:51:08.029427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:40:28.656635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842819+0000","last_change":"2023-08-11T15:51:05.854549+0000","last_active":"2023-08-11T15:51:06.842819+0000","last_peered":"2023-08-11T15:51:06.842819+0000","last_clean":"2023-08-11T15:51:06.842819+0000","last_became_active":"2023-08-11T15:51:05.853942+0000","last_became_peered":"2023-08-11T15:51:05.853942+0000","last_unstale":"2023-08-11T15:51:06.842819+0000","last_undegraded":"2023-08-11T15:51:06.842819+0000","last_fullsized":"2023-08-11T15:51:06.842819+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:12.616756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844270+0000","last_change":"2023-08-11T15:51:00.754010+0000","last_active":"2023-08-11T15:51:06.844270+0000","last_peered":"2023-08-11T15:51:06.844270+0000","last_clean":"2023-08-11T15:51:06.844270+0000","last_became_active":"2023-08-11T15:51:00.753640+0000","last_became_peered":"2023-08-11T15:51:00.753640+0000","last_unstale":"2023-08-11T15:51:06.844270+0000","last_undegraded":"2023-08-11T15:51:06.844270+0000","last_fullsized":"2023-08-11T15:51:06.844270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:59.887617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029380+0000","last_change":"2023-08-11T15:51:01.748240+0000","last_active":"2023-08-11T15:51:08.029380+0000","last_peered":"2023-08-11T15:51:08.029380+0000","last_clean":"2023-08-11T15:51:08.029380+0000","last_became_active":"2023-08-11T15:51:01.747946+0000","last_became_peered":"2023-08-11T15:51:01.747946+0000","last_unstale":"2023-08-11T15:51:08.029380+0000","last_undegraded":"2023-08-11T15:51:08.029380+0000","last_fullsized":"2023-08-11T15:51:08.029380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:08:40.170218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450286+0000","last_change":"2023-08-11T15:51:05.843794+0000","last_active":"2023-08-11T15:51:07.450286+0000","last_peered":"2023-08-11T15:51:07.450286+0000","last_clean":"2023-08-11T15:51:07.450286+0000","last_became_active":"2023-08-11T15:51:05.843006+0000","last_became_peered":"2023-08-11T15:51:05.843006+0000","last_unstale":"2023-08-11T15:51:07.450286+0000","last_undegraded":"2023-08-11T15:51:07.450286+0000","last_fullsized":"2023-08-11T15:51:07.450286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:53:03.920014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841384+0000","last_change":"2023-08-11T15:51:00.753341+0000","last_active":"2023-08-11T15:51:06.841384+0000","last_peered":"2023-08-11T15:51:06.841384+0000","last_clean":"2023-08-11T15:51:06.841384+0000","last_became_active":"2023-08-11T15:51:00.753070+0000","last_became_peered":"2023-08-11T15:51:00.753070+0000","last_unstale":"2023-08-11T15:51:06.841384+0000","last_undegraded":"2023-08-11T15:51:06.841384+0000","last_fullsized":"2023-08-11T15:51:06.841384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:07.508785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575377+0000","last_change":"2023-08-11T15:51:01.745198+0000","last_active":"2023-08-11T15:51:08.575377+0000","last_peered":"2023-08-11T15:51:08.575377+0000","last_clean":"2023-08-11T15:51:08.575377+0000","last_became_active":"2023-08-11T15:51:01.744845+0000","last_became_peered":"2023-08-11T15:51:01.744845+0000","last_unstale":"2023-08-11T15:51:08.575377+0000","last_undegraded":"2023-08-11T15:51:08.575377+0000","last_fullsized":"2023-08-11T15:51:08.575377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:07:54.433272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029435+0000","last_change":"2023-08-11T15:51:05.846476+0000","last_active":"2023-08-11T15:51:08.029435+0000","last_peered":"2023-08-11T15:51:08.029435+0000","last_clean":"2023-08-11T15:51:08.029435+0000","last_became_active":"2023-08-11T15:51:05.846010+0000","last_became_peered":"2023-08-11T15:51:05.846010+0000","last_unstale":"2023-08-11T15:51:08.029435+0000","last_undegraded":"2023-08-11T15:51:08.029435+0000","last_fullsized":"2023-08-11T15:51:08.029435+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:55:13.061478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029848+0000","last_change":"2023-08-11T15:51:00.748391+0000","last_active":"2023-08-11T15:51:08.029848+0000","last_peered":"2023-08-11T15:51:08.029848+0000","last_clean":"2023-08-11T15:51:08.029848+0000","last_became_active":"2023-08-11T15:51:00.744340+0000","last_became_peered":"2023-08-11T15:51:00.744340+0000","last_unstale":"2023-08-11T15:51:08.029848+0000","last_undegraded":"2023-08-11T15:51:08.029848+0000","last_fullsized":"2023-08-11T15:51:08.029848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:56:37.817338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278162+0000","last_change":"2023-08-11T15:51:01.749456+0000","last_active":"2023-08-11T15:51:07.278162+0000","last_peered":"2023-08-11T15:51:07.278162+0000","last_clean":"2023-08-11T15:51:07.278162+0000","last_became_active":"2023-08-11T15:51:01.749042+0000","last_became_peered":"2023-08-11T15:51:01.749042+0000","last_unstale":"2023-08-11T15:51:07.278162+0000","last_undegraded":"2023-08-11T15:51:07.278162+0000","last_fullsized":"2023-08-11T15:51:07.278162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:03.472522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644834+0000","last_change":"2023-08-11T15:51:05.849122+0000","last_active":"2023-08-11T15:51:08.644834+0000","last_peered":"2023-08-11T15:51:08.644834+0000","last_clean":"2023-08-11T15:51:08.644834+0000","last_became_active":"2023-08-11T15:51:05.848698+0000","last_became_peered":"2023-08-11T15:51:05.848698+0000","last_unstale":"2023-08-11T15:51:08.644834+0000","last_undegraded":"2023-08-11T15:51:08.644834+0000","last_fullsized":"2023-08-11T15:51:08.644834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:46:03.919402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449963+0000","last_change":"2023-08-11T15:51:00.743735+0000","last_active":"2023-08-11T15:51:07.449963+0000","last_peered":"2023-08-11T15:51:07.449963+0000","last_clean":"2023-08-11T15:51:07.449963+0000","last_became_active":"2023-08-11T15:51:00.743412+0000","last_became_peered":"2023-08-11T15:51:00.743412+0000","last_unstale":"2023-08-11T15:51:07.449963+0000","last_undegraded":"2023-08-11T15:51:07.449963+0000","last_fullsized":"2023-08-11T15:51:07.449963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:07.487770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841592+0000","last_change":"2023-08-11T15:51:01.740683+0000","last_active":"2023-08-11T15:51:06.841592+0000","last_peered":"2023-08-11T15:51:06.841592+0000","last_clean":"2023-08-11T15:51:06.841592+0000","last_became_active":"2023-08-11T15:51:01.740230+0000","last_became_peered":"2023-08-11T15:51:01.740230+0000","last_unstale":"2023-08-11T15:51:06.841592+0000","last_undegraded":"2023-08-11T15:51:06.841592+0000","last_fullsized":"2023-08-11T15:51:06.841592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:57.966913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841129+0000","last_change":"2023-08-11T15:51:05.843204+0000","last_active":"2023-08-11T15:51:06.841129+0000","last_peered":"2023-08-11T15:51:06.841129+0000","last_clean":"2023-08-11T15:51:06.841129+0000","last_became_active":"2023-08-11T15:51:05.842830+0000","last_became_peered":"2023-08-11T15:51:05.842830+0000","last_unstale":"2023-08-11T15:51:06.841129+0000","last_undegraded":"2023-08-11T15:51:06.841129+0000","last_fullsized":"2023-08-11T15:51:06.841129+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:13.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277138+0000","last_change":"2023-08-11T15:51:00.735703+0000","last_active":"2023-08-11T15:51:07.277138+0000","last_peered":"2023-08-11T15:51:07.277138+0000","last_clean":"2023-08-11T15:51:07.277138+0000","last_became_active":"2023-08-11T15:51:00.735384+0000","last_became_peered":"2023-08-11T15:51:00.735384+0000","last_unstale":"2023-08-11T15:51:07.277138+0000","last_undegraded":"2023-08-11T15:51:07.277138+0000","last_fullsized":"2023-08-11T15:51:07.277138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:59.887617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644911+0000","last_change":"2023-08-11T15:51:01.742945+0000","last_active":"2023-08-11T15:51:08.644911+0000","last_peered":"2023-08-11T15:51:08.644911+0000","last_clean":"2023-08-11T15:51:08.644911+0000","last_became_active":"2023-08-11T15:51:01.742477+0000","last_became_peered":"2023-08-11T15:51:01.742477+0000","last_unstale":"2023-08-11T15:51:08.644911+0000","last_undegraded":"2023-08-11T15:51:08.644911+0000","last_fullsized":"2023-08-11T15:51:08.644911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:17:49.108624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276720+0000","last_change":"2023-08-11T15:51:01.755598+0000","last_active":"2023-08-11T15:51:07.276720+0000","last_peered":"2023-08-11T15:51:07.276720+0000","last_clean":"2023-08-11T15:51:07.276720+0000","last_became_active":"2023-08-11T15:51:01.755329+0000","last_became_peered":"2023-08-11T15:51:01.755329+0000","last_unstale":"2023-08-11T15:51:07.276720+0000","last_undegraded":"2023-08-11T15:51:07.276720+0000","last_fullsized":"2023-08-11T15:51:07.276720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:09.551106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842608+0000","last_change":"2023-08-11T15:51:00.747156+0000","last_active":"2023-08-11T15:51:06.842608+0000","last_peered":"2023-08-11T15:51:06.842608+0000","last_clean":"2023-08-11T15:51:06.842608+0000","last_became_active":"2023-08-11T15:51:00.745449+0000","last_became_peered":"2023-08-11T15:51:00.745449+0000","last_unstale":"2023-08-11T15:51:06.842608+0000","last_undegraded":"2023-08-11T15:51:06.842608+0000","last_fullsized":"2023-08-11T15:51:06.842608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643560+0000","last_change":"2023-08-11T15:51:05.839469+0000","last_active":"2023-08-11T15:51:08.643560+0000","last_peered":"2023-08-11T15:51:08.643560+0000","last_clean":"2023-08-11T15:51:08.643560+0000","last_became_active":"2023-08-11T15:51:05.839035+0000","last_became_peered":"2023-08-11T15:51:05.839035+0000","last_unstale":"2023-08-11T15:51:08.643560+0000","last_undegraded":"2023-08-11T15:51:08.643560+0000","last_fullsized":"2023-08-11T15:51:08.643560+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:11.122033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276959+0000","last_change":"2023-08-11T15:51:01.748158+0000","last_active":"2023-08-11T15:51:07.276959+0000","last_peered":"2023-08-11T15:51:07.276959+0000","last_clean":"2023-08-11T15:51:07.276959+0000","last_became_active":"2023-08-11T15:51:01.747870+0000","last_became_peered":"2023-08-11T15:51:01.747870+0000","last_unstale":"2023-08-11T15:51:07.276959+0000","last_undegraded":"2023-08-11T15:51:07.276959+0000","last_fullsized":"2023-08-11T15:51:07.276959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:57.966913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845778+0000","last_change":"2023-08-11T15:51:00.739033+0000","last_active":"2023-08-11T15:51:06.845778+0000","last_peered":"2023-08-11T15:51:06.845778+0000","last_clean":"2023-08-11T15:51:06.845778+0000","last_became_active":"2023-08-11T15:51:00.738746+0000","last_became_peered":"2023-08-11T15:51:00.738746+0000","last_unstale":"2023-08-11T15:51:06.845778+0000","last_undegraded":"2023-08-11T15:51:06.845778+0000","last_fullsized":"2023-08-11T15:51:06.845778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841377+0000","last_change":"2023-08-11T15:51:05.853925+0000","last_active":"2023-08-11T15:51:06.841377+0000","last_peered":"2023-08-11T15:51:06.841377+0000","last_clean":"2023-08-11T15:51:06.841377+0000","last_became_active":"2023-08-11T15:51:05.853531+0000","last_became_peered":"2023-08-11T15:51:05.853531+0000","last_unstale":"2023-08-11T15:51:06.841377+0000","last_undegraded":"2023-08-11T15:51:06.841377+0000","last_fullsized":"2023-08-11T15:51:06.841377+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:13.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843271+0000","last_change":"2023-08-11T15:51:00.733126+0000","last_active":"2023-08-11T15:51:06.843271+0000","last_peered":"2023-08-11T15:51:06.843271+0000","last_clean":"2023-08-11T15:51:06.843271+0000","last_became_active":"2023-08-11T15:51:00.731695+0000","last_became_peered":"2023-08-11T15:51:00.731695+0000","last_unstale":"2023-08-11T15:51:06.843271+0000","last_undegraded":"2023-08-11T15:51:06.843271+0000","last_fullsized":"2023-08-11T15:51:06.843271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:07.914582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449817+0000","last_change":"2023-08-11T15:51:01.750498+0000","last_active":"2023-08-11T15:51:07.449817+0000","last_peered":"2023-08-11T15:51:07.449817+0000","last_clean":"2023-08-11T15:51:07.449817+0000","last_became_active":"2023-08-11T15:51:01.750204+0000","last_became_peered":"2023-08-11T15:51:01.750204+0000","last_unstale":"2023-08-11T15:51:07.449817+0000","last_undegraded":"2023-08-11T15:51:07.449817+0000","last_fullsized":"2023-08-11T15:51:07.449817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:30:40.844864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277300+0000","last_change":"2023-08-11T15:51:05.849252+0000","last_active":"2023-08-11T15:51:07.277300+0000","last_peered":"2023-08-11T15:51:07.277300+0000","last_clean":"2023-08-11T15:51:07.277300+0000","last_became_active":"2023-08-11T15:51:05.848107+0000","last_became_peered":"2023-08-11T15:51:05.848107+0000","last_unstale":"2023-08-11T15:51:07.277300+0000","last_undegraded":"2023-08-11T15:51:07.277300+0000","last_fullsized":"2023-08-11T15:51:07.277300+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:13.022552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:13.714518+0000","last_change":"2023-08-11T15:51:00.743771+0000","last_active":"2023-08-11T15:51:13.714518+0000","last_peered":"2023-08-11T15:51:13.714518+0000","last_clean":"2023-08-11T15:51:13.714518+0000","last_became_active":"2023-08-11T15:51:00.741410+0000","last_became_peered":"2023-08-11T15:51:00.741410+0000","last_unstale":"2023-08-11T15:51:13.714518+0000","last_undegraded":"2023-08-11T15:51:13.714518+0000","last_fullsized":"2023-08-11T15:51:13.714518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:46:01.024956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029670+0000","last_change":"2023-08-11T15:51:01.742960+0000","last_active":"2023-08-11T15:51:08.029670+0000","last_peered":"2023-08-11T15:51:08.029670+0000","last_clean":"2023-08-11T15:51:08.029670+0000","last_became_active":"2023-08-11T15:51:01.742674+0000","last_became_peered":"2023-08-11T15:51:01.742674+0000","last_unstale":"2023-08-11T15:51:08.029670+0000","last_undegraded":"2023-08-11T15:51:08.029670+0000","last_fullsized":"2023-08-11T15:51:08.029670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:16:42.978408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450187+0000","last_change":"2023-08-11T15:51:05.850946+0000","last_active":"2023-08-11T15:51:07.450187+0000","last_peered":"2023-08-11T15:51:07.450187+0000","last_clean":"2023-08-11T15:51:07.450187+0000","last_became_active":"2023-08-11T15:51:05.850520+0000","last_became_peered":"2023-08-11T15:51:05.850520+0000","last_unstale":"2023-08-11T15:51:07.450187+0000","last_undegraded":"2023-08-11T15:51:07.450187+0000","last_fullsized":"2023-08-11T15:51:07.450187+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:19:49.259478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029612+0000","last_change":"2023-08-11T15:51:01.733689+0000","last_active":"2023-08-11T15:51:08.029612+0000","last_peered":"2023-08-11T15:51:08.029612+0000","last_clean":"2023-08-11T15:51:08.029612+0000","last_became_active":"2023-08-11T15:51:01.732609+0000","last_became_peered":"2023-08-11T15:51:01.732609+0000","last_unstale":"2023-08-11T15:51:08.029612+0000","last_undegraded":"2023-08-11T15:51:08.029612+0000","last_fullsized":"2023-08-11T15:51:08.029612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:37:47.670825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.645014+0000","last_change":"2023-08-11T15:50:54.667601+0000","last_active":"2023-08-11T15:51:08.645014+0000","last_peered":"2023-08-11T15:51:08.645014+0000","last_clean":"2023-08-11T15:51:08.645014+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:51:08.645014+0000","last_undegraded":"2023-08-11T15:51:08.645014+0000","last_fullsized":"2023-08-11T15:51:08.645014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:26.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842067+0000","last_change":"2023-08-11T15:51:00.744472+0000","last_active":"2023-08-11T15:51:06.842067+0000","last_peered":"2023-08-11T15:51:06.842067+0000","last_clean":"2023-08-11T15:51:06.842067+0000","last_became_active":"2023-08-11T15:51:00.744162+0000","last_became_peered":"2023-08-11T15:51:00.744162+0000","last_unstale":"2023-08-11T15:51:06.842067+0000","last_undegraded":"2023-08-11T15:51:06.842067+0000","last_fullsized":"2023-08-11T15:51:06.842067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:44.445917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276689+0000","last_change":"2023-08-11T15:51:05.840924+0000","last_active":"2023-08-11T15:51:07.276689+0000","last_peered":"2023-08-11T15:51:07.276689+0000","last_clean":"2023-08-11T15:51:07.276689+0000","last_became_active":"2023-08-11T15:51:05.840447+0000","last_became_peered":"2023-08-11T15:51:05.840447+0000","last_unstale":"2023-08-11T15:51:07.276689+0000","last_undegraded":"2023-08-11T15:51:07.276689+0000","last_fullsized":"2023-08-11T15:51:07.276689+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:15.105060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277414+0000","last_change":"2023-08-11T15:51:01.747032+0000","last_active":"2023-08-11T15:51:07.277414+0000","last_peered":"2023-08-11T15:51:07.277414+0000","last_clean":"2023-08-11T15:51:07.277414+0000","last_became_active":"2023-08-11T15:51:01.746677+0000","last_became_peered":"2023-08-11T15:51:01.746677+0000","last_unstale":"2023-08-11T15:51:07.277414+0000","last_undegraded":"2023-08-11T15:51:07.277414+0000","last_fullsized":"2023-08-11T15:51:07.277414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:45.458114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450746+0000","last_change":"2023-08-11T15:50:54.662968+0000","last_active":"2023-08-11T15:51:07.450746+0000","last_peered":"2023-08-11T15:51:07.450746+0000","last_clean":"2023-08-11T15:51:07.450746+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:51:07.450746+0000","last_undegraded":"2023-08-11T15:51:07.450746+0000","last_fullsized":"2023-08-11T15:51:07.450746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:21.192999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278963+0000","last_change":"2023-08-11T15:51:00.728408+0000","last_active":"2023-08-11T15:51:07.278963+0000","last_peered":"2023-08-11T15:51:07.278963+0000","last_clean":"2023-08-11T15:51:07.278963+0000","last_became_active":"2023-08-11T15:51:00.721838+0000","last_became_peered":"2023-08-11T15:51:00.721838+0000","last_unstale":"2023-08-11T15:51:07.278963+0000","last_undegraded":"2023-08-11T15:51:07.278963+0000","last_fullsized":"2023-08-11T15:51:07.278963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:10.364383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028778+0000","last_change":"2023-08-11T15:51:05.850931+0000","last_active":"2023-08-11T15:51:08.028778+0000","last_peered":"2023-08-11T15:51:08.028778+0000","last_clean":"2023-08-11T15:51:08.028778+0000","last_became_active":"2023-08-11T15:51:05.850227+0000","last_became_peered":"2023-08-11T15:51:05.850227+0000","last_unstale":"2023-08-11T15:51:08.028778+0000","last_undegraded":"2023-08-11T15:51:08.028778+0000","last_fullsized":"2023-08-11T15:51:08.028778+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.029041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840867+0000","last_change":"2023-08-11T15:51:01.739504+0000","last_active":"2023-08-11T15:51:06.840867+0000","last_peered":"2023-08-11T15:51:06.840867+0000","last_clean":"2023-08-11T15:51:06.840867+0000","last_became_active":"2023-08-11T15:51:01.739172+0000","last_became_peered":"2023-08-11T15:51:01.739172+0000","last_unstale":"2023-08-11T15:51:06.840867+0000","last_undegraded":"2023-08-11T15:51:06.840867+0000","last_fullsized":"2023-08-11T15:51:06.840867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:12:34.705521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844166+0000","last_change":"2023-08-11T15:50:55.272783+0000","last_active":"2023-08-11T15:51:06.844166+0000","last_peered":"2023-08-11T15:51:06.844166+0000","last_clean":"2023-08-11T15:51:06.844166+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:51:06.844166+0000","last_undegraded":"2023-08-11T15:51:06.844166+0000","last_fullsized":"2023-08-11T15:51:06.844166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:26.593873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450158+0000","last_change":"2023-08-11T15:51:00.737083+0000","last_active":"2023-08-11T15:51:07.450158+0000","last_peered":"2023-08-11T15:51:07.450158+0000","last_clean":"2023-08-11T15:51:07.450158+0000","last_became_active":"2023-08-11T15:51:00.736799+0000","last_became_peered":"2023-08-11T15:51:00.736799+0000","last_unstale":"2023-08-11T15:51:07.450158+0000","last_undegraded":"2023-08-11T15:51:07.450158+0000","last_fullsized":"2023-08-11T15:51:07.450158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:03:14.195345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643834+0000","last_change":"2023-08-11T15:51:05.842861+0000","last_active":"2023-08-11T15:51:08.643834+0000","last_peered":"2023-08-11T15:51:08.643834+0000","last_clean":"2023-08-11T15:51:08.643834+0000","last_became_active":"2023-08-11T15:51:05.842445+0000","last_became_peered":"2023-08-11T15:51:05.842445+0000","last_unstale":"2023-08-11T15:51:08.643834+0000","last_undegraded":"2023-08-11T15:51:08.643834+0000","last_fullsized":"2023-08-11T15:51:08.643834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:41.996042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449997+0000","last_change":"2023-08-11T15:51:01.746944+0000","last_active":"2023-08-11T15:51:07.449997+0000","last_peered":"2023-08-11T15:51:07.449997+0000","last_clean":"2023-08-11T15:51:07.449997+0000","last_became_active":"2023-08-11T15:51:01.745106+0000","last_became_peered":"2023-08-11T15:51:01.745106+0000","last_unstale":"2023-08-11T15:51:07.449997+0000","last_undegraded":"2023-08-11T15:51:07.449997+0000","last_fullsized":"2023-08-11T15:51:07.449997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:38:45.900704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450317+0000","last_change":"2023-08-11T15:50:55.270974+0000","last_active":"2023-08-11T15:51:07.450317+0000","last_peered":"2023-08-11T15:51:07.450317+0000","last_clean":"2023-08-11T15:51:07.450317+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:51:07.450317+0000","last_undegraded":"2023-08-11T15:51:07.450317+0000","last_fullsized":"2023-08-11T15:51:07.450317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:49:35.912143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845445+0000","last_change":"2023-08-11T15:51:00.736540+0000","last_active":"2023-08-11T15:51:06.845445+0000","last_peered":"2023-08-11T15:51:06.845445+0000","last_clean":"2023-08-11T15:51:06.845445+0000","last_became_active":"2023-08-11T15:51:00.736211+0000","last_became_peered":"2023-08-11T15:51:00.736211+0000","last_unstale":"2023-08-11T15:51:06.845445+0000","last_undegraded":"2023-08-11T15:51:06.845445+0000","last_fullsized":"2023-08-11T15:51:06.845445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.749276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840401+0000","last_change":"2023-08-11T15:51:05.834845+0000","last_active":"2023-08-11T15:51:06.840401+0000","last_peered":"2023-08-11T15:51:06.840401+0000","last_clean":"2023-08-11T15:51:06.840401+0000","last_became_active":"2023-08-11T15:51:05.834329+0000","last_became_peered":"2023-08-11T15:51:05.834329+0000","last_unstale":"2023-08-11T15:51:06.840401+0000","last_undegraded":"2023-08-11T15:51:06.840401+0000","last_fullsized":"2023-08-11T15:51:06.840401+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:27:54.750554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450003+0000","last_change":"2023-08-11T15:51:01.744716+0000","last_active":"2023-08-11T15:51:07.450003+0000","last_peered":"2023-08-11T15:51:07.450003+0000","last_clean":"2023-08-11T15:51:07.450003+0000","last_became_active":"2023-08-11T15:51:01.744413+0000","last_became_peered":"2023-08-11T15:51:01.744413+0000","last_unstale":"2023-08-11T15:51:07.450003+0000","last_undegraded":"2023-08-11T15:51:07.450003+0000","last_fullsized":"2023-08-11T15:51:07.450003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:43:31.654955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030156+0000","last_change":"2023-08-11T15:51:00.744357+0000","last_active":"2023-08-11T15:51:08.030156+0000","last_peered":"2023-08-11T15:51:08.030156+0000","last_clean":"2023-08-11T15:51:08.030156+0000","last_became_active":"2023-08-11T15:51:00.744005+0000","last_became_peered":"2023-08-11T15:51:00.744005+0000","last_unstale":"2023-08-11T15:51:08.030156+0000","last_undegraded":"2023-08-11T15:51:08.030156+0000","last_fullsized":"2023-08-11T15:51:08.030156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:40:52.791014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842501+0000","last_change":"2023-08-11T15:50:54.660862+0000","last_active":"2023-08-11T15:51:06.842501+0000","last_peered":"2023-08-11T15:51:06.842501+0000","last_clean":"2023-08-11T15:51:06.842501+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:51:06.842501+0000","last_undegraded":"2023-08-11T15:51:06.842501+0000","last_fullsized":"2023-08-11T15:51:06.842501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:01.459444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840964+0000","last_change":"2023-08-11T15:51:05.834675+0000","last_active":"2023-08-11T15:51:06.840964+0000","last_peered":"2023-08-11T15:51:06.840964+0000","last_clean":"2023-08-11T15:51:06.840964+0000","last_became_active":"2023-08-11T15:51:05.833754+0000","last_became_peered":"2023-08-11T15:51:05.833754+0000","last_unstale":"2023-08-11T15:51:06.840964+0000","last_undegraded":"2023-08-11T15:51:06.840964+0000","last_fullsized":"2023-08-11T15:51:06.840964+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:02:13.485965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450824+0000","last_change":"2023-08-11T15:51:01.747929+0000","last_active":"2023-08-11T15:51:07.450824+0000","last_peered":"2023-08-11T15:51:07.450824+0000","last_clean":"2023-08-11T15:51:07.450824+0000","last_became_active":"2023-08-11T15:51:01.747489+0000","last_became_peered":"2023-08-11T15:51:01.747489+0000","last_unstale":"2023-08-11T15:51:07.450824+0000","last_undegraded":"2023-08-11T15:51:07.450824+0000","last_fullsized":"2023-08-11T15:51:07.450824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:16:47.148363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449531+0000","last_change":"2023-08-11T15:51:00.735611+0000","last_active":"2023-08-11T15:51:07.449531+0000","last_peered":"2023-08-11T15:51:07.449531+0000","last_clean":"2023-08-11T15:51:07.449531+0000","last_became_active":"2023-08-11T15:51:00.735334+0000","last_became_peered":"2023-08-11T15:51:00.735334+0000","last_unstale":"2023-08-11T15:51:07.449531+0000","last_undegraded":"2023-08-11T15:51:07.449531+0000","last_fullsized":"2023-08-11T15:51:07.449531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:21:47.785823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574361+0000","last_change":"2023-08-11T15:50:54.660575+0000","last_active":"2023-08-11T15:51:08.574361+0000","last_peered":"2023-08-11T15:51:08.574361+0000","last_clean":"2023-08-11T15:51:08.574361+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:51:08.574361+0000","last_undegraded":"2023-08-11T15:51:08.574361+0000","last_fullsized":"2023-08-11T15:51:08.574361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:44:19.275242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574984+0000","last_change":"2023-08-11T15:51:05.851204+0000","last_active":"2023-08-11T15:51:08.574984+0000","last_peered":"2023-08-11T15:51:08.574984+0000","last_clean":"2023-08-11T15:51:08.574984+0000","last_became_active":"2023-08-11T15:51:05.850707+0000","last_became_peered":"2023-08-11T15:51:05.850707+0000","last_unstale":"2023-08-11T15:51:08.574984+0000","last_undegraded":"2023-08-11T15:51:08.574984+0000","last_fullsized":"2023-08-11T15:51:08.574984+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:46:46.279070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841686+0000","last_change":"2023-08-11T15:51:01.754726+0000","last_active":"2023-08-11T15:51:06.841686+0000","last_peered":"2023-08-11T15:51:06.841686+0000","last_clean":"2023-08-11T15:51:06.841686+0000","last_became_active":"2023-08-11T15:51:01.754139+0000","last_became_peered":"2023-08-11T15:51:01.754139+0000","last_unstale":"2023-08-11T15:51:06.841686+0000","last_undegraded":"2023-08-11T15:51:06.841686+0000","last_fullsized":"2023-08-11T15:51:06.841686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839697+0000","last_change":"2023-08-11T15:51:00.753588+0000","last_active":"2023-08-11T15:51:06.839697+0000","last_peered":"2023-08-11T15:51:06.839697+0000","last_clean":"2023-08-11T15:51:06.839697+0000","last_became_active":"2023-08-11T15:51:00.753328+0000","last_became_peered":"2023-08-11T15:51:00.753328+0000","last_unstale":"2023-08-11T15:51:06.839697+0000","last_undegraded":"2023-08-11T15:51:06.839697+0000","last_fullsized":"2023-08-11T15:51:06.839697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:32.917550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842875+0000","last_change":"2023-08-11T15:50:54.669675+0000","last_active":"2023-08-11T15:51:06.842875+0000","last_peered":"2023-08-11T15:51:06.842875+0000","last_clean":"2023-08-11T15:51:06.842875+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:51:06.842875+0000","last_undegraded":"2023-08-11T15:51:06.842875+0000","last_fullsized":"2023-08-11T15:51:06.842875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:50.905375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845192+0000","last_change":"2023-08-11T15:51:05.839222+0000","last_active":"2023-08-11T15:51:06.845192+0000","last_peered":"2023-08-11T15:51:06.845192+0000","last_clean":"2023-08-11T15:51:06.845192+0000","last_became_active":"2023-08-11T15:51:05.838710+0000","last_became_peered":"2023-08-11T15:51:05.838710+0000","last_unstale":"2023-08-11T15:51:06.845192+0000","last_undegraded":"2023-08-11T15:51:06.845192+0000","last_fullsized":"2023-08-11T15:51:06.845192+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:07.568296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644528+0000","last_change":"2023-08-11T15:51:01.744538+0000","last_active":"2023-08-11T15:51:08.644528+0000","last_peered":"2023-08-11T15:51:08.644528+0000","last_clean":"2023-08-11T15:51:08.644528+0000","last_became_active":"2023-08-11T15:51:01.744177+0000","last_became_peered":"2023-08-11T15:51:01.744177+0000","last_unstale":"2023-08-11T15:51:08.644528+0000","last_undegraded":"2023-08-11T15:51:08.644528+0000","last_fullsized":"2023-08-11T15:51:08.644528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:36.996581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277663+0000","last_change":"2023-08-11T15:51:00.736542+0000","last_active":"2023-08-11T15:51:07.277663+0000","last_peered":"2023-08-11T15:51:07.277663+0000","last_clean":"2023-08-11T15:51:07.277663+0000","last_became_active":"2023-08-11T15:51:00.736156+0000","last_became_peered":"2023-08-11T15:51:00.736156+0000","last_unstale":"2023-08-11T15:51:07.277663+0000","last_undegraded":"2023-08-11T15:51:07.277663+0000","last_fullsized":"2023-08-11T15:51:07.277663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843800+0000","last_change":"2023-08-11T15:50:54.663730+0000","last_active":"2023-08-11T15:51:06.843800+0000","last_peered":"2023-08-11T15:51:06.843800+0000","last_clean":"2023-08-11T15:51:06.843800+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:51:06.843800+0000","last_undegraded":"2023-08-11T15:51:06.843800+0000","last_fullsized":"2023-08-11T15:51:06.843800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:36.826532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843222+0000","last_change":"2023-08-11T15:50:48.426393+0000","last_active":"2023-08-11T15:51:06.843222+0000","last_peered":"2023-08-11T15:51:06.843222+0000","last_clean":"2023-08-11T15:51:06.843222+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:51:06.843222+0000","last_undegraded":"2023-08-11T15:51:06.843222+0000","last_fullsized":"2023-08-11T15:51:06.843222+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_clean_scrub_stamp":"2023-08-11T15:50:47.133560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:47.340557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843724+0000","last_change":"2023-08-11T15:51:05.850670+0000","last_active":"2023-08-11T15:51:06.843724+0000","last_peered":"2023-08-11T15:51:06.843724+0000","last_clean":"2023-08-11T15:51:06.843724+0000","last_became_active":"2023-08-11T15:51:05.850026+0000","last_became_peered":"2023-08-11T15:51:05.850026+0000","last_unstale":"2023-08-11T15:51:06.843724+0000","last_undegraded":"2023-08-11T15:51:06.843724+0000","last_fullsized":"2023-08-11T15:51:06.843724+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:08.068219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450801+0000","last_change":"2023-08-11T15:51:01.748517+0000","last_active":"2023-08-11T15:51:07.450801+0000","last_peered":"2023-08-11T15:51:07.450801+0000","last_clean":"2023-08-11T15:51:07.450801+0000","last_became_active":"2023-08-11T15:51:01.748222+0000","last_became_peered":"2023-08-11T15:51:01.748222+0000","last_unstale":"2023-08-11T15:51:07.450801+0000","last_undegraded":"2023-08-11T15:51:07.450801+0000","last_fullsized":"2023-08-11T15:51:07.450801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:24:23.784093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277502+0000","last_change":"2023-08-11T15:51:00.734516+0000","last_active":"2023-08-11T15:51:07.277502+0000","last_peered":"2023-08-11T15:51:07.277502+0000","last_clean":"2023-08-11T15:51:07.277502+0000","last_became_active":"2023-08-11T15:51:00.734143+0000","last_became_peered":"2023-08-11T15:51:00.734143+0000","last_unstale":"2023-08-11T15:51:07.277502+0000","last_undegraded":"2023-08-11T15:51:07.277502+0000","last_fullsized":"2023-08-11T15:51:07.277502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:02.960249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030585+0000","last_change":"2023-08-11T15:51:05.848396+0000","last_active":"2023-08-11T15:51:08.030585+0000","last_peered":"2023-08-11T15:51:08.030585+0000","last_clean":"2023-08-11T15:51:08.030585+0000","last_became_active":"2023-08-11T15:51:05.846255+0000","last_became_peered":"2023-08-11T15:51:05.846255+0000","last_unstale":"2023-08-11T15:51:08.030585+0000","last_undegraded":"2023-08-11T15:51:08.030585+0000","last_fullsized":"2023-08-11T15:51:08.030585+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:11:39.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.031055+0000","last_change":"2023-08-11T15:51:01.745387+0000","last_active":"2023-08-11T15:51:08.031055+0000","last_peered":"2023-08-11T15:51:08.031055+0000","last_clean":"2023-08-11T15:51:08.031055+0000","last_became_active":"2023-08-11T15:51:01.745076+0000","last_became_peered":"2023-08-11T15:51:01.745076+0000","last_unstale":"2023-08-11T15:51:08.031055+0000","last_undegraded":"2023-08-11T15:51:08.031055+0000","last_fullsized":"2023-08-11T15:51:08.031055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:46:26.462256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030159+0000","last_change":"2023-08-11T15:51:00.749075+0000","last_active":"2023-08-11T15:51:08.030159+0000","last_peered":"2023-08-11T15:51:08.030159+0000","last_clean":"2023-08-11T15:51:08.030159+0000","last_became_active":"2023-08-11T15:51:00.748709+0000","last_became_peered":"2023-08-11T15:51:00.748709+0000","last_unstale":"2023-08-11T15:51:08.030159+0000","last_undegraded":"2023-08-11T15:51:08.030159+0000","last_fullsized":"2023-08-11T15:51:08.030159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:18:35.477914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844801+0000","last_change":"2023-08-11T15:51:05.843837+0000","last_active":"2023-08-11T15:51:06.844801+0000","last_peered":"2023-08-11T15:51:06.844801+0000","last_clean":"2023-08-11T15:51:06.844801+0000","last_became_active":"2023-08-11T15:51:05.843261+0000","last_became_peered":"2023-08-11T15:51:05.843261+0000","last_unstale":"2023-08-11T15:51:06.844801+0000","last_undegraded":"2023-08-11T15:51:06.844801+0000","last_fullsized":"2023-08-11T15:51:06.844801+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:08.068219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030584+0000","last_change":"2023-08-11T15:51:01.735881+0000","last_active":"2023-08-11T15:51:08.030584+0000","last_peered":"2023-08-11T15:51:08.030584+0000","last_clean":"2023-08-11T15:51:08.030584+0000","last_became_active":"2023-08-11T15:51:01.735297+0000","last_became_peered":"2023-08-11T15:51:01.735297+0000","last_unstale":"2023-08-11T15:51:08.030584+0000","last_undegraded":"2023-08-11T15:51:08.030584+0000","last_fullsized":"2023-08-11T15:51:08.030584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:06:05.316781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644461+0000","last_change":"2023-08-11T15:51:00.754786+0000","last_active":"2023-08-11T15:51:08.644461+0000","last_peered":"2023-08-11T15:51:08.644461+0000","last_clean":"2023-08-11T15:51:08.644461+0000","last_became_active":"2023-08-11T15:51:00.754437+0000","last_became_peered":"2023-08-11T15:51:00.754437+0000","last_unstale":"2023-08-11T15:51:08.644461+0000","last_undegraded":"2023-08-11T15:51:08.644461+0000","last_fullsized":"2023-08-11T15:51:08.644461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:41:49.524739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449772+0000","last_change":"2023-08-11T15:51:05.833713+0000","last_active":"2023-08-11T15:51:07.449772+0000","last_peered":"2023-08-11T15:51:07.449772+0000","last_clean":"2023-08-11T15:51:07.449772+0000","last_became_active":"2023-08-11T15:51:05.833089+0000","last_became_peered":"2023-08-11T15:51:05.833089+0000","last_unstale":"2023-08-11T15:51:07.449772+0000","last_undegraded":"2023-08-11T15:51:07.449772+0000","last_fullsized":"2023-08-11T15:51:07.449772+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:17:01.770576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030037+0000","last_change":"2023-08-11T15:51:01.748944+0000","last_active":"2023-08-11T15:51:08.030037+0000","last_peered":"2023-08-11T15:51:08.030037+0000","last_clean":"2023-08-11T15:51:08.030037+0000","last_became_active":"2023-08-11T15:51:01.748661+0000","last_became_peered":"2023-08-11T15:51:01.748661+0000","last_unstale":"2023-08-11T15:51:08.030037+0000","last_undegraded":"2023-08-11T15:51:08.030037+0000","last_fullsized":"2023-08-11T15:51:08.030037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:02:46.657720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278376+0000","last_change":"2023-08-11T15:51:00.743809+0000","last_active":"2023-08-11T15:51:07.278376+0000","last_peered":"2023-08-11T15:51:07.278376+0000","last_clean":"2023-08-11T15:51:07.278376+0000","last_became_active":"2023-08-11T15:51:00.743475+0000","last_became_peered":"2023-08-11T15:51:00.743475+0000","last_unstale":"2023-08-11T15:51:07.278376+0000","last_undegraded":"2023-08-11T15:51:07.278376+0000","last_fullsized":"2023-08-11T15:51:07.278376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574614+0000","last_change":"2023-08-11T15:51:05.845552+0000","last_active":"2023-08-11T15:51:08.574614+0000","last_peered":"2023-08-11T15:51:08.574614+0000","last_clean":"2023-08-11T15:51:08.574614+0000","last_became_active":"2023-08-11T15:51:05.845089+0000","last_became_peered":"2023-08-11T15:51:05.845089+0000","last_unstale":"2023-08-11T15:51:08.574614+0000","last_undegraded":"2023-08-11T15:51:08.574614+0000","last_fullsized":"2023-08-11T15:51:08.574614+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:14:44.850189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840418+0000","last_change":"2023-08-11T15:51:01.754448+0000","last_active":"2023-08-11T15:51:06.840418+0000","last_peered":"2023-08-11T15:51:06.840418+0000","last_clean":"2023-08-11T15:51:06.840418+0000","last_became_active":"2023-08-11T15:51:01.754103+0000","last_became_peered":"2023-08-11T15:51:01.754103+0000","last_unstale":"2023-08-11T15:51:06.840418+0000","last_undegraded":"2023-08-11T15:51:06.840418+0000","last_fullsized":"2023-08-11T15:51:06.840418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:48:19.707007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844315+0000","last_change":"2023-08-11T15:51:00.740399+0000","last_active":"2023-08-11T15:51:06.844315+0000","last_peered":"2023-08-11T15:51:06.844315+0000","last_clean":"2023-08-11T15:51:06.844315+0000","last_became_active":"2023-08-11T15:51:00.740100+0000","last_became_peered":"2023-08-11T15:51:00.740100+0000","last_unstale":"2023-08-11T15:51:06.844315+0000","last_undegraded":"2023-08-11T15:51:06.844315+0000","last_fullsized":"2023-08-11T15:51:06.844315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:07.914582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574234+0000","last_change":"2023-08-11T15:51:05.857518+0000","last_active":"2023-08-11T15:51:08.574234+0000","last_peered":"2023-08-11T15:51:08.574234+0000","last_clean":"2023-08-11T15:51:08.574234+0000","last_became_active":"2023-08-11T15:51:05.857113+0000","last_became_peered":"2023-08-11T15:51:05.857113+0000","last_unstale":"2023-08-11T15:51:08.574234+0000","last_undegraded":"2023-08-11T15:51:08.574234+0000","last_fullsized":"2023-08-11T15:51:08.574234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:27:22.211685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643846+0000","last_change":"2023-08-11T15:51:01.754343+0000","last_active":"2023-08-11T15:51:08.643846+0000","last_peered":"2023-08-11T15:51:08.643846+0000","last_clean":"2023-08-11T15:51:08.643846+0000","last_became_active":"2023-08-11T15:51:01.754068+0000","last_became_peered":"2023-08-11T15:51:01.754068+0000","last_unstale":"2023-08-11T15:51:08.643846+0000","last_undegraded":"2023-08-11T15:51:08.643846+0000","last_fullsized":"2023-08-11T15:51:08.643846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:17.417849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843859+0000","last_change":"2023-08-11T15:51:00.739404+0000","last_active":"2023-08-11T15:51:06.843859+0000","last_peered":"2023-08-11T15:51:06.843859+0000","last_clean":"2023-08-11T15:51:06.843859+0000","last_became_active":"2023-08-11T15:51:00.738824+0000","last_became_peered":"2023-08-11T15:51:00.738824+0000","last_unstale":"2023-08-11T15:51:06.843859+0000","last_undegraded":"2023-08-11T15:51:06.843859+0000","last_fullsized":"2023-08-11T15:51:06.843859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574756+0000","last_change":"2023-08-11T15:51:05.852495+0000","last_active":"2023-08-11T15:51:08.574756+0000","last_peered":"2023-08-11T15:51:08.574756+0000","last_clean":"2023-08-11T15:51:08.574756+0000","last_became_active":"2023-08-11T15:51:05.852028+0000","last_became_peered":"2023-08-11T15:51:05.852028+0000","last_unstale":"2023-08-11T15:51:08.574756+0000","last_undegraded":"2023-08-11T15:51:08.574756+0000","last_fullsized":"2023-08-11T15:51:08.574756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:00:50.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030008+0000","last_change":"2023-08-11T15:51:01.746332+0000","last_active":"2023-08-11T15:51:08.030008+0000","last_peered":"2023-08-11T15:51:08.030008+0000","last_clean":"2023-08-11T15:51:08.030008+0000","last_became_active":"2023-08-11T15:51:01.746037+0000","last_became_peered":"2023-08-11T15:51:01.746037+0000","last_unstale":"2023-08-11T15:51:08.030008+0000","last_undegraded":"2023-08-11T15:51:08.030008+0000","last_fullsized":"2023-08-11T15:51:08.030008+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:01:53.294858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846006+0000","last_change":"2023-08-11T15:51:00.750241+0000","last_active":"2023-08-11T15:51:06.846006+0000","last_peered":"2023-08-11T15:51:06.846006+0000","last_clean":"2023-08-11T15:51:06.846006+0000","last_became_active":"2023-08-11T15:51:00.749526+0000","last_became_peered":"2023-08-11T15:51:00.749526+0000","last_unstale":"2023-08-11T15:51:06.846006+0000","last_undegraded":"2023-08-11T15:51:06.846006+0000","last_fullsized":"2023-08-11T15:51:06.846006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.749276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574297+0000","last_change":"2023-08-11T15:51:05.852589+0000","last_active":"2023-08-11T15:51:08.574297+0000","last_peered":"2023-08-11T15:51:08.574297+0000","last_clean":"2023-08-11T15:51:08.574297+0000","last_became_active":"2023-08-11T15:51:05.852123+0000","last_became_peered":"2023-08-11T15:51:05.852123+0000","last_unstale":"2023-08-11T15:51:08.574297+0000","last_undegraded":"2023-08-11T15:51:08.574297+0000","last_fullsized":"2023-08-11T15:51:08.574297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:52.516197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841591+0000","last_change":"2023-08-11T15:51:01.740461+0000","last_active":"2023-08-11T15:51:06.841591+0000","last_peered":"2023-08-11T15:51:06.841591+0000","last_clean":"2023-08-11T15:51:06.841591+0000","last_became_active":"2023-08-11T15:51:01.740152+0000","last_became_peered":"2023-08-11T15:51:01.740152+0000","last_unstale":"2023-08-11T15:51:06.841591+0000","last_undegraded":"2023-08-11T15:51:06.841591+0000","last_fullsized":"2023-08-11T15:51:06.841591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:09.551106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846676+0000","last_change":"2023-08-11T15:51:00.749804+0000","last_active":"2023-08-11T15:51:06.846676+0000","last_peered":"2023-08-11T15:51:06.846676+0000","last_clean":"2023-08-11T15:51:06.846676+0000","last_became_active":"2023-08-11T15:51:00.749397+0000","last_became_peered":"2023-08-11T15:51:00.749397+0000","last_unstale":"2023-08-11T15:51:06.846676+0000","last_undegraded":"2023-08-11T15:51:06.846676+0000","last_fullsized":"2023-08-11T15:51:06.846676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843683+0000","last_change":"2023-08-11T15:51:05.857250+0000","last_active":"2023-08-11T15:51:06.843683+0000","last_peered":"2023-08-11T15:51:06.843683+0000","last_clean":"2023-08-11T15:51:06.843683+0000","last_became_active":"2023-08-11T15:51:05.856843+0000","last_became_peered":"2023-08-11T15:51:05.856843+0000","last_unstale":"2023-08-11T15:51:06.843683+0000","last_undegraded":"2023-08-11T15:51:06.843683+0000","last_fullsized":"2023-08-11T15:51:06.843683+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:30:03.847116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846267+0000","last_change":"2023-08-11T15:51:05.849584+0000","last_active":"2023-08-11T15:51:06.846267+0000","last_peered":"2023-08-11T15:51:06.846267+0000","last_clean":"2023-08-11T15:51:06.846267+0000","last_became_active":"2023-08-11T15:51:05.849144+0000","last_became_peered":"2023-08-11T15:51:05.849144+0000","last_unstale":"2023-08-11T15:51:06.846267+0000","last_undegraded":"2023-08-11T15:51:06.846267+0000","last_fullsized":"2023-08-11T15:51:06.846267+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:19:51.734914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643988+0000","last_change":"2023-08-11T15:51:00.743098+0000","last_active":"2023-08-11T15:51:08.643988+0000","last_peered":"2023-08-11T15:51:08.643988+0000","last_clean":"2023-08-11T15:51:08.643988+0000","last_became_active":"2023-08-11T15:51:00.742606+0000","last_became_peered":"2023-08-11T15:51:00.742606+0000","last_unstale":"2023-08-11T15:51:08.643988+0000","last_undegraded":"2023-08-11T15:51:08.643988+0000","last_fullsized":"2023-08-11T15:51:08.643988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:38:17.546002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278685+0000","last_change":"2023-08-11T15:51:01.746170+0000","last_active":"2023-08-11T15:51:07.278685+0000","last_peered":"2023-08-11T15:51:07.278685+0000","last_clean":"2023-08-11T15:51:07.278685+0000","last_became_active":"2023-08-11T15:51:01.745672+0000","last_became_peered":"2023-08-11T15:51:01.745672+0000","last_unstale":"2023-08-11T15:51:07.278685+0000","last_undegraded":"2023-08-11T15:51:07.278685+0000","last_fullsized":"2023-08-11T15:51:07.278685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:20:27.436792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278398+0000","last_change":"2023-08-11T15:51:05.844332+0000","last_active":"2023-08-11T15:51:07.278398+0000","last_peered":"2023-08-11T15:51:07.278398+0000","last_clean":"2023-08-11T15:51:07.278398+0000","last_became_active":"2023-08-11T15:51:05.841336+0000","last_became_peered":"2023-08-11T15:51:05.841336+0000","last_unstale":"2023-08-11T15:51:07.278398+0000","last_undegraded":"2023-08-11T15:51:07.278398+0000","last_fullsized":"2023-08-11T15:51:07.278398+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:19:51.734914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844582+0000","last_change":"2023-08-11T15:51:00.750609+0000","last_active":"2023-08-11T15:51:06.844582+0000","last_peered":"2023-08-11T15:51:06.844582+0000","last_clean":"2023-08-11T15:51:06.844582+0000","last_became_active":"2023-08-11T15:51:00.750277+0000","last_became_peered":"2023-08-11T15:51:00.750277+0000","last_unstale":"2023-08-11T15:51:06.844582+0000","last_undegraded":"2023-08-11T15:51:06.844582+0000","last_fullsized":"2023-08-11T15:51:06.844582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:19.559486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841973+0000","last_change":"2023-08-11T15:51:01.750332+0000","last_active":"2023-08-11T15:51:06.841973+0000","last_peered":"2023-08-11T15:51:06.841973+0000","last_clean":"2023-08-11T15:51:06.841973+0000","last_became_active":"2023-08-11T15:51:01.749234+0000","last_became_peered":"2023-08-11T15:51:01.749234+0000","last_unstale":"2023-08-11T15:51:06.841973+0000","last_undegraded":"2023-08-11T15:51:06.841973+0000","last_fullsized":"2023-08-11T15:51:06.841973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:08.926778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839995+0000","last_change":"2023-08-11T15:51:00.745865+0000","last_active":"2023-08-11T15:51:06.839995+0000","last_peered":"2023-08-11T15:51:06.839995+0000","last_clean":"2023-08-11T15:51:06.839995+0000","last_became_active":"2023-08-11T15:51:00.745469+0000","last_became_peered":"2023-08-11T15:51:00.745469+0000","last_unstale":"2023-08-11T15:51:06.839995+0000","last_undegraded":"2023-08-11T15:51:06.839995+0000","last_fullsized":"2023-08-11T15:51:06.839995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:25:15.827962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845118+0000","last_change":"2023-08-11T15:51:01.740388+0000","last_active":"2023-08-11T15:51:06.845118+0000","last_peered":"2023-08-11T15:51:06.845118+0000","last_clean":"2023-08-11T15:51:06.845118+0000","last_became_active":"2023-08-11T15:51:01.739972+0000","last_became_peered":"2023-08-11T15:51:01.739972+0000","last_unstale":"2023-08-11T15:51:06.845118+0000","last_undegraded":"2023-08-11T15:51:06.845118+0000","last_fullsized":"2023-08-11T15:51:06.845118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:41.231851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277756+0000","last_change":"2023-08-11T15:51:05.847377+0000","last_active":"2023-08-11T15:51:07.277756+0000","last_peered":"2023-08-11T15:51:07.277756+0000","last_clean":"2023-08-11T15:51:07.277756+0000","last_became_active":"2023-08-11T15:51:05.843902+0000","last_became_peered":"2023-08-11T15:51:05.843902+0000","last_unstale":"2023-08-11T15:51:07.277756+0000","last_undegraded":"2023-08-11T15:51:07.277756+0000","last_fullsized":"2023-08-11T15:51:07.277756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:45.327626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843318+0000","last_change":"2023-08-11T15:51:00.753432+0000","last_active":"2023-08-11T15:51:06.843318+0000","last_peered":"2023-08-11T15:51:06.843318+0000","last_clean":"2023-08-11T15:51:06.843318+0000","last_became_active":"2023-08-11T15:51:00.753099+0000","last_became_peered":"2023-08-11T15:51:00.753099+0000","last_unstale":"2023-08-11T15:51:06.843318+0000","last_undegraded":"2023-08-11T15:51:06.843318+0000","last_fullsized":"2023-08-11T15:51:06.843318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:44.445917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278374+0000","last_change":"2023-08-11T15:51:01.754975+0000","last_active":"2023-08-11T15:51:07.278374+0000","last_peered":"2023-08-11T15:51:07.278374+0000","last_clean":"2023-08-11T15:51:07.278374+0000","last_became_active":"2023-08-11T15:51:01.754617+0000","last_became_peered":"2023-08-11T15:51:01.754617+0000","last_unstale":"2023-08-11T15:51:07.278374+0000","last_undegraded":"2023-08-11T15:51:07.278374+0000","last_fullsized":"2023-08-11T15:51:07.278374+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:20.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844645+0000","last_change":"2023-08-11T15:51:05.841963+0000","last_active":"2023-08-11T15:51:06.844645+0000","last_peered":"2023-08-11T15:51:06.844645+0000","last_clean":"2023-08-11T15:51:06.844645+0000","last_became_active":"2023-08-11T15:51:05.841100+0000","last_became_peered":"2023-08-11T15:51:05.841100+0000","last_unstale":"2023-08-11T15:51:06.844645+0000","last_undegraded":"2023-08-11T15:51:06.844645+0000","last_fullsized":"2023-08-11T15:51:06.844645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:49.553888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643596+0000","last_change":"2023-08-11T15:51:01.740691+0000","last_active":"2023-08-11T15:51:08.643596+0000","last_peered":"2023-08-11T15:51:08.643596+0000","last_clean":"2023-08-11T15:51:08.643596+0000","last_became_active":"2023-08-11T15:51:01.740411+0000","last_became_peered":"2023-08-11T15:51:01.740411+0000","last_unstale":"2023-08-11T15:51:08.643596+0000","last_undegraded":"2023-08-11T15:51:08.643596+0000","last_fullsized":"2023-08-11T15:51:08.643596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:14:24.513369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276978+0000","last_change":"2023-08-11T15:51:01.754967+0000","last_active":"2023-08-11T15:51:07.276978+0000","last_peered":"2023-08-11T15:51:07.276978+0000","last_clean":"2023-08-11T15:51:07.276978+0000","last_became_active":"2023-08-11T15:51:01.754670+0000","last_became_peered":"2023-08-11T15:51:01.754670+0000","last_unstale":"2023-08-11T15:51:07.276978+0000","last_undegraded":"2023-08-11T15:51:07.276978+0000","last_fullsized":"2023-08-11T15:51:07.276978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029518+0000","last_change":"2023-08-11T15:51:01.743911+0000","last_active":"2023-08-11T15:51:08.029518+0000","last_peered":"2023-08-11T15:51:08.029518+0000","last_clean":"2023-08-11T15:51:08.029518+0000","last_became_active":"2023-08-11T15:51:01.743608+0000","last_became_peered":"2023-08-11T15:51:01.743608+0000","last_unstale":"2023-08-11T15:51:08.029518+0000","last_undegraded":"2023-08-11T15:51:08.029518+0000","last_fullsized":"2023-08-11T15:51:08.029518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:45:30.132181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277768+0000","last_change":"2023-08-11T15:51:01.748054+0000","last_active":"2023-08-11T15:51:07.277768+0000","last_peered":"2023-08-11T15:51:07.277768+0000","last_clean":"2023-08-11T15:51:07.277768+0000","last_became_active":"2023-08-11T15:51:01.747751+0000","last_became_peered":"2023-08-11T15:51:01.747751+0000","last_unstale":"2023-08-11T15:51:07.277768+0000","last_undegraded":"2023-08-11T15:51:07.277768+0000","last_fullsized":"2023-08-11T15:51:07.277768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:33.655411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450215+0000","last_change":"2023-08-11T15:51:01.745398+0000","last_active":"2023-08-11T15:51:07.450215+0000","last_peered":"2023-08-11T15:51:07.450215+0000","last_clean":"2023-08-11T15:51:07.450215+0000","last_became_active":"2023-08-11T15:51:01.745013+0000","last_became_peered":"2023-08-11T15:51:01.745013+0000","last_unstale":"2023-08-11T15:51:07.450215+0000","last_undegraded":"2023-08-11T15:51:07.450215+0000","last_fullsized":"2023-08-11T15:51:07.450215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:37:45.341896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844793+0000","last_change":"2023-08-11T15:51:01.747243+0000","last_active":"2023-08-11T15:51:06.844793+0000","last_peered":"2023-08-11T15:51:06.844793+0000","last_clean":"2023-08-11T15:51:06.844793+0000","last_became_active":"2023-08-11T15:51:01.746905+0000","last_became_peered":"2023-08-11T15:51:01.746905+0000","last_unstale":"2023-08-11T15:51:06.844793+0000","last_undegraded":"2023-08-11T15:51:06.844793+0000","last_fullsized":"2023-08-11T15:51:06.844793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450600+0000","last_change":"2023-08-11T15:51:01.743655+0000","last_active":"2023-08-11T15:51:07.450600+0000","last_peered":"2023-08-11T15:51:07.450600+0000","last_clean":"2023-08-11T15:51:07.450600+0000","last_became_active":"2023-08-11T15:51:01.743308+0000","last_became_peered":"2023-08-11T15:51:01.743308+0000","last_unstale":"2023-08-11T15:51:07.450600+0000","last_undegraded":"2023-08-11T15:51:07.450600+0000","last_fullsized":"2023-08-11T15:51:07.450600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:08:59.509573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450686+0000","last_change":"2023-08-11T15:51:01.747270+0000","last_active":"2023-08-11T15:51:07.450686+0000","last_peered":"2023-08-11T15:51:07.450686+0000","last_clean":"2023-08-11T15:51:07.450686+0000","last_became_active":"2023-08-11T15:51:01.746924+0000","last_became_peered":"2023-08-11T15:51:01.746924+0000","last_unstale":"2023-08-11T15:51:07.450686+0000","last_undegraded":"2023-08-11T15:51:07.450686+0000","last_fullsized":"2023-08-11T15:51:07.450686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:07:01.100072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842527+0000","last_change":"2023-08-11T15:51:01.750707+0000","last_active":"2023-08-11T15:51:06.842527+0000","last_peered":"2023-08-11T15:51:06.842527+0000","last_clean":"2023-08-11T15:51:06.842527+0000","last_became_active":"2023-08-11T15:51:01.750243+0000","last_became_peered":"2023-08-11T15:51:01.750243+0000","last_unstale":"2023-08-11T15:51:06.842527+0000","last_undegraded":"2023-08-11T15:51:06.842527+0000","last_fullsized":"2023-08-11T15:51:06.842527+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276812+0000","last_change":"2023-08-11T15:51:01.744825+0000","last_active":"2023-08-11T15:51:07.276812+0000","last_peered":"2023-08-11T15:51:07.276812+0000","last_clean":"2023-08-11T15:51:07.276812+0000","last_became_active":"2023-08-11T15:51:01.744446+0000","last_became_peered":"2023-08-11T15:51:01.744446+0000","last_unstale":"2023-08-11T15:51:07.276812+0000","last_undegraded":"2023-08-11T15:51:07.276812+0000","last_fullsized":"2023-08-11T15:51:07.276812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:08.520981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839890+0000","last_change":"2023-08-11T15:51:01.754226+0000","last_active":"2023-08-11T15:51:06.839890+0000","last_peered":"2023-08-11T15:51:06.839890+0000","last_clean":"2023-08-11T15:51:06.839890+0000","last_became_active":"2023-08-11T15:51:01.753864+0000","last_became_peered":"2023-08-11T15:51:01.753864+0000","last_unstale":"2023-08-11T15:51:06.839890+0000","last_undegraded":"2023-08-11T15:51:06.839890+0000","last_fullsized":"2023-08-11T15:51:06.839890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:42:42.499555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843198+0000","last_change":"2023-08-11T15:51:01.751698+0000","last_active":"2023-08-11T15:51:06.843198+0000","last_peered":"2023-08-11T15:51:06.843198+0000","last_clean":"2023-08-11T15:51:06.843198+0000","last_became_active":"2023-08-11T15:51:01.751183+0000","last_became_peered":"2023-08-11T15:51:01.751183+0000","last_unstale":"2023-08-11T15:51:06.843198+0000","last_undegraded":"2023-08-11T15:51:06.843198+0000","last_fullsized":"2023-08-11T15:51:06.843198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029934+0000","last_change":"2023-08-11T15:51:01.743064+0000","last_active":"2023-08-11T15:51:08.029934+0000","last_peered":"2023-08-11T15:51:08.029934+0000","last_clean":"2023-08-11T15:51:08.029934+0000","last_became_active":"2023-08-11T15:51:01.742783+0000","last_became_peered":"2023-08-11T15:51:01.742783+0000","last_unstale":"2023-08-11T15:51:08.029934+0000","last_undegraded":"2023-08-11T15:51:08.029934+0000","last_fullsized":"2023-08-11T15:51:08.029934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:25:56.529094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574852+0000","last_change":"2023-08-11T15:51:01.746080+0000","last_active":"2023-08-11T15:51:08.574852+0000","last_peered":"2023-08-11T15:51:08.574852+0000","last_clean":"2023-08-11T15:51:08.574852+0000","last_became_active":"2023-08-11T15:51:01.745366+0000","last_became_peered":"2023-08-11T15:51:01.745366+0000","last_unstale":"2023-08-11T15:51:08.574852+0000","last_undegraded":"2023-08-11T15:51:08.574852+0000","last_fullsized":"2023-08-11T15:51:08.574852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:58:33.514033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030489+0000","last_change":"2023-08-11T15:51:01.735183+0000","last_active":"2023-08-11T15:51:08.030489+0000","last_peered":"2023-08-11T15:51:08.030489+0000","last_clean":"2023-08-11T15:51:08.030489+0000","last_became_active":"2023-08-11T15:51:01.734771+0000","last_became_peered":"2023-08-11T15:51:01.734771+0000","last_unstale":"2023-08-11T15:51:08.030489+0000","last_undegraded":"2023-08-11T15:51:08.030489+0000","last_fullsized":"2023-08-11T15:51:08.030489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:28:21.255388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842288+0000","last_change":"2023-08-11T15:51:01.753111+0000","last_active":"2023-08-11T15:51:06.842288+0000","last_peered":"2023-08-11T15:51:06.842288+0000","last_clean":"2023-08-11T15:51:06.842288+0000","last_became_active":"2023-08-11T15:51:01.752792+0000","last_became_peered":"2023-08-11T15:51:01.752792+0000","last_unstale":"2023-08-11T15:51:06.842288+0000","last_undegraded":"2023-08-11T15:51:06.842288+0000","last_fullsized":"2023-08-11T15:51:06.842288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028692+0000","last_change":"2023-08-11T15:51:01.737005+0000","last_active":"2023-08-11T15:51:08.028692+0000","last_peered":"2023-08-11T15:51:08.028692+0000","last_clean":"2023-08-11T15:51:08.028692+0000","last_became_active":"2023-08-11T15:51:01.736669+0000","last_became_peered":"2023-08-11T15:51:01.736669+0000","last_unstale":"2023-08-11T15:51:08.028692+0000","last_undegraded":"2023-08-11T15:51:08.028692+0000","last_fullsized":"2023-08-11T15:51:08.028692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:10:00.491572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844298+0000","last_change":"2023-08-11T15:51:01.755893+0000","last_active":"2023-08-11T15:51:06.844298+0000","last_peered":"2023-08-11T15:51:06.844298+0000","last_clean":"2023-08-11T15:51:06.844298+0000","last_became_active":"2023-08-11T15:51:01.755182+0000","last_became_peered":"2023-08-11T15:51:01.755182+0000","last_unstale":"2023-08-11T15:51:06.844298+0000","last_undegraded":"2023-08-11T15:51:06.844298+0000","last_fullsized":"2023-08-11T15:51:06.844298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:03.472522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842189+0000","last_change":"2023-08-11T15:51:01.739785+0000","last_active":"2023-08-11T15:51:06.842189+0000","last_peered":"2023-08-11T15:51:06.842189+0000","last_clean":"2023-08-11T15:51:06.842189+0000","last_became_active":"2023-08-11T15:51:01.739500+0000","last_became_peered":"2023-08-11T15:51:01.739500+0000","last_unstale":"2023-08-11T15:51:06.842189+0000","last_undegraded":"2023-08-11T15:51:06.842189+0000","last_fullsized":"2023-08-11T15:51:06.842189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574594+0000","last_change":"2023-08-11T15:51:01.732565+0000","last_active":"2023-08-11T15:51:08.574594+0000","last_peered":"2023-08-11T15:51:08.574594+0000","last_clean":"2023-08-11T15:51:08.574594+0000","last_became_active":"2023-08-11T15:51:01.732076+0000","last_became_peered":"2023-08-11T15:51:01.732076+0000","last_unstale":"2023-08-11T15:51:08.574594+0000","last_undegraded":"2023-08-11T15:51:08.574594+0000","last_fullsized":"2023-08-11T15:51:08.574594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:20.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575381+0000","last_change":"2023-08-11T15:51:01.736603+0000","last_active":"2023-08-11T15:51:08.575381+0000","last_peered":"2023-08-11T15:51:08.575381+0000","last_clean":"2023-08-11T15:51:08.575381+0000","last_became_active":"2023-08-11T15:51:01.735835+0000","last_became_peered":"2023-08-11T15:51:01.735835+0000","last_unstale":"2023-08-11T15:51:08.575381+0000","last_undegraded":"2023-08-11T15:51:08.575381+0000","last_fullsized":"2023-08-11T15:51:08.575381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:07.026259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030580+0000","last_change":"2023-08-11T15:51:01.752520+0000","last_active":"2023-08-11T15:51:08.030580+0000","last_peered":"2023-08-11T15:51:08.030580+0000","last_clean":"2023-08-11T15:51:08.030580+0000","last_became_active":"2023-08-11T15:51:01.752203+0000","last_became_peered":"2023-08-11T15:51:01.752203+0000","last_unstale":"2023-08-11T15:51:08.030580+0000","last_undegraded":"2023-08-11T15:51:08.030580+0000","last_fullsized":"2023-08-11T15:51:08.030580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:29:45.923242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845432+0000","last_change":"2023-08-11T15:51:01.737002+0000","last_active":"2023-08-11T15:51:06.845432+0000","last_peered":"2023-08-11T15:51:06.845432+0000","last_clean":"2023-08-11T15:51:06.845432+0000","last_became_active":"2023-08-11T15:51:01.736720+0000","last_became_peered":"2023-08-11T15:51:01.736720+0000","last_unstale":"2023-08-11T15:51:06.845432+0000","last_undegraded":"2023-08-11T15:51:06.845432+0000","last_fullsized":"2023-08-11T15:51:06.845432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:20.571682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841857+0000","last_change":"2023-08-11T15:51:01.752888+0000","last_active":"2023-08-11T15:51:06.841857+0000","last_peered":"2023-08-11T15:51:06.841857+0000","last_clean":"2023-08-11T15:51:06.841857+0000","last_became_active":"2023-08-11T15:51:01.752593+0000","last_became_peered":"2023-08-11T15:51:01.752593+0000","last_unstale":"2023-08-11T15:51:06.841857+0000","last_undegraded":"2023-08-11T15:51:06.841857+0000","last_fullsized":"2023-08-11T15:51:06.841857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:39:00.899813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841369+0000","last_change":"2023-08-11T15:51:01.741200+0000","last_active":"2023-08-11T15:51:06.841369+0000","last_peered":"2023-08-11T15:51:06.841369+0000","last_clean":"2023-08-11T15:51:06.841369+0000","last_became_active":"2023-08-11T15:51:01.738593+0000","last_became_peered":"2023-08-11T15:51:01.738593+0000","last_unstale":"2023-08-11T15:51:06.841369+0000","last_undegraded":"2023-08-11T15:51:06.841369+0000","last_fullsized":"2023-08-11T15:51:06.841369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:02:09.390191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844003+0000","last_change":"2023-08-11T15:51:01.756072+0000","last_active":"2023-08-11T15:51:06.844003+0000","last_peered":"2023-08-11T15:51:06.844003+0000","last_clean":"2023-08-11T15:51:06.844003+0000","last_became_active":"2023-08-11T15:51:01.755522+0000","last_became_peered":"2023-08-11T15:51:01.755522+0000","last_unstale":"2023-08-11T15:51:06.844003+0000","last_undegraded":"2023-08-11T15:51:06.844003+0000","last_fullsized":"2023-08-11T15:51:06.844003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:41.231851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029178+0000","last_change":"2023-08-11T15:51:01.737005+0000","last_active":"2023-08-11T15:51:08.029178+0000","last_peered":"2023-08-11T15:51:08.029178+0000","last_clean":"2023-08-11T15:51:08.029178+0000","last_became_active":"2023-08-11T15:51:01.736657+0000","last_became_peered":"2023-08-11T15:51:01.736657+0000","last_unstale":"2023-08-11T15:51:08.029178+0000","last_undegraded":"2023-08-11T15:51:08.029178+0000","last_fullsized":"2023-08-11T15:51:08.029178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:20:52.818361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841216+0000","last_change":"2023-08-11T15:51:01.749800+0000","last_active":"2023-08-11T15:51:06.841216+0000","last_peered":"2023-08-11T15:51:06.841216+0000","last_clean":"2023-08-11T15:51:06.841216+0000","last_became_active":"2023-08-11T15:51:01.749436+0000","last_became_peered":"2023-08-11T15:51:01.749436+0000","last_unstale":"2023-08-11T15:51:06.841216+0000","last_undegraded":"2023-08-11T15:51:06.841216+0000","last_fullsized":"2023-08-11T15:51:06.841216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644342+0000","last_change":"2023-08-11T15:51:01.739883+0000","last_active":"2023-08-11T15:51:08.644342+0000","last_peered":"2023-08-11T15:51:08.644342+0000","last_clean":"2023-08-11T15:51:08.644342+0000","last_became_active":"2023-08-11T15:51:01.739490+0000","last_became_peered":"2023-08-11T15:51:01.739490+0000","last_unstale":"2023-08-11T15:51:08.644342+0000","last_undegraded":"2023-08-11T15:51:08.644342+0000","last_fullsized":"2023-08-11T15:51:08.644342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:19.428228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030296+0000","last_change":"2023-08-11T15:51:01.737093+0000","last_active":"2023-08-11T15:51:08.030296+0000","last_peered":"2023-08-11T15:51:08.030296+0000","last_clean":"2023-08-11T15:51:08.030296+0000","last_became_active":"2023-08-11T15:51:01.736803+0000","last_became_peered":"2023-08-11T15:51:01.736803+0000","last_unstale":"2023-08-11T15:51:08.030296+0000","last_undegraded":"2023-08-11T15:51:08.030296+0000","last_fullsized":"2023-08-11T15:51:08.030296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:50:46.864696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844939+0000","last_change":"2023-08-11T15:51:01.753163+0000","last_active":"2023-08-11T15:51:06.844939+0000","last_peered":"2023-08-11T15:51:06.844939+0000","last_clean":"2023-08-11T15:51:06.844939+0000","last_became_active":"2023-08-11T15:51:01.752836+0000","last_became_peered":"2023-08-11T15:51:01.752836+0000","last_unstale":"2023-08-11T15:51:06.844939+0000","last_undegraded":"2023-08-11T15:51:06.844939+0000","last_fullsized":"2023-08-11T15:51:06.844939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:15.195226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575171+0000","last_change":"2023-08-11T15:51:01.739059+0000","last_active":"2023-08-11T15:51:08.575171+0000","last_peered":"2023-08-11T15:51:08.575171+0000","last_clean":"2023-08-11T15:51:08.575171+0000","last_became_active":"2023-08-11T15:51:01.738739+0000","last_became_peered":"2023-08-11T15:51:01.738739+0000","last_unstale":"2023-08-11T15:51:08.575171+0000","last_undegraded":"2023-08-11T15:51:08.575171+0000","last_fullsized":"2023-08-11T15:51:08.575171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:11:40.167154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":921600,"data_stored":717480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]}]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":123848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.827}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.917}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1219999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1629999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":319488,"data_stored":126952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":123286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5469999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7070000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.605}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5369999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":897024,"data_stored":714146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:51:27.911 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-11T15:51:27.912 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-11T15:51:27.912 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-08-11T15:51:27.912 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-08-11T15:51:27.913 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-08-11T15:51:28.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.179+0000 7ff4da6fa640 1 Processor -- start 2023-08-11T15:51:28.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.179+0000 7ff4da6fa640 1 -- start start 2023-08-11T15:51:28.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.181+0000 7ff4da6fa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d415a140 0x7ff4d406b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.181+0000 7ff4da6fa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff4d406f5d0 0x7ff4d406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.181+0000 7ff4da6fa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d40c8a30 con 0x7ff4d40cb430 2023-08-11T15:51:28.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.181+0000 7ff4da6fa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d406ffa0 con 0x7ff4d406f5d0 2023-08-11T15:51:28.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.181+0000 7ff4da6fa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40b1580 con 0x7ff4d415a140 2023-08-11T15:51:28.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4cbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff4d40cb430 0x7ff4d40b0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48992/0 (socket says 172.21.15.27:48992) 2023-08-11T15:51:28.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4cbfff640 1 -- 172.21.15.27:0/3872203161 learned_addr learned my addr 172.21.15.27:0/3872203161 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:28.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4d8c97640 1 --2- 172.21.15.27:0/3872203161 >> v2:172.21.15.27:3300/0 conn(0x7ff4d406f5d0 0x7ff4d406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4d3fff640 1 --2- 172.21.15.27:0/3872203161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d415a140 0x7ff4d406b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4d37fe640 1 -- 172.21.15.27:0/3872203161 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857598941 0 0) 0x7ff4d40c8a30 con 0x7ff4d40cb430 2023-08-11T15:51:28.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4d37fe640 1 -- 172.21.15.27:0/3872203161 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4c0003520 con 0x7ff4d40cb430 2023-08-11T15:51:28.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4d8c97640 1 -- 172.21.15.27:0/3872203161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d415a140 msgr2=0x7ff4d406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:28.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4d8c97640 1 --2- 172.21.15.27:0/3872203161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d415a140 0x7ff4d406b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4d8c97640 1 -- 172.21.15.27:0/3872203161 >> v1:172.21.15.27:6789/0 conn(0x7ff4d40cb430 legacy=0x7ff4d40b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4d8c97640 1 -- 172.21.15.27:0/3872203161 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d406c0c0 con 0x7ff4d406f5d0 2023-08-11T15:51:28.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.182+0000 7ff4d3fff640 1 --2- 172.21.15.27:0/3872203161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d415a140 0x7ff4d406b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:28.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.184+0000 7ff4d8c97640 1 --2- 172.21.15.27:0/3872203161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d406f5d0 0x7ff4d406fa60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009dc0 tx=0x7ff4cc02be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb58601300ded39 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.184+0000 7ff4d37fe640 1 -- 172.21.15.27:0/3872203161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc02cc30 con 0x7ff4d406f5d0 2023-08-11T15:51:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.184+0000 7ff4d37fe640 1 -- 172.21.15.27:0/3872203161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc02cd90 con 0x7ff4d406f5d0 2023-08-11T15:51:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.184+0000 7ff4d37fe640 1 -- 172.21.15.27:0/3872203161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc034640 con 0x7ff4d406f5d0 2023-08-11T15:51:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.185+0000 7ff4da6fa640 1 -- 172.21.15.27:0/3872203161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d406f5d0 msgr2=0x7ff4d406fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.185+0000 7ff4da6fa640 1 --2- 172.21.15.27:0/3872203161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d406f5d0 0x7ff4d406fa60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009dc0 tx=0x7ff4cc02be80 comp rx=0 tx=0).stop 2023-08-11T15:51:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.185+0000 7ff4da6fa640 1 -- 172.21.15.27:0/3872203161 shutdown_connections 2023-08-11T15:51:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.185+0000 7ff4da6fa640 1 --2- 172.21.15.27:0/3872203161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d406f5d0 0x7ff4d406fa60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.185+0000 7ff4da6fa640 1 --2- 172.21.15.27:0/3872203161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d415a140 0x7ff4d406b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.185+0000 7ff4da6fa640 1 -- 172.21.15.27:0/3872203161 >> 172.21.15.27:0/3872203161 conn(0x7ff4d40afb70 msgr2=0x7ff4d40aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.185+0000 7ff4da6fa640 1 -- 172.21.15.27:0/3872203161 shutdown_connections 2023-08-11T15:51:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.185+0000 7ff4da6fa640 1 -- 172.21.15.27:0/3872203161 wait complete. 2023-08-11T15:51:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.187+0000 7ff4da6fa640 1 Processor -- start 2023-08-11T15:51:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.187+0000 7ff4da6fa640 1 -- start start 2023-08-11T15:51:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.188+0000 7ff4da6fa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d406f5d0 0x7ff4d40c1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.188+0000 7ff4da6fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d40cb430 0x7ff4d4149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.188+0000 7ff4da6fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d415a140 0x7ff4d4156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.188+0000 7ff4da6fa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40ac990 con 0x7ff4d40cb430 2023-08-11T15:51:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.188+0000 7ff4da6fa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40c1a60 con 0x7ff4d415a140 2023-08-11T15:51:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.188+0000 7ff4da6fa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40cbb70 con 0x7ff4d406f5d0 2023-08-11T15:51:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.188+0000 7ff4cbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d40cb430 0x7ff4d4149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.189+0000 7ff4cbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d40cb430 0x7ff4d4149380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46342/0 (socket says 172.21.15.27:46342) 2023-08-11T15:51:28.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.189+0000 7ff4cbfff640 1 -- 172.21.15.27:0/2205870609 learned_addr learned my addr 172.21.15.27:0/2205870609 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:28.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.189+0000 7ff4d3fff640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d406f5d0 0x7ff4d40c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.189+0000 7ff4d8c97640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d415a140 0x7ff4d4156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.189+0000 7ff4cbfff640 1 -- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d406f5d0 msgr2=0x7ff4d40c1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.189+0000 7ff4cbfff640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d406f5d0 0x7ff4d40c1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.189+0000 7ff4cbfff640 1 -- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d415a140 msgr2=0x7ff4d4156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.189+0000 7ff4cbfff640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d415a140 0x7ff4d4156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.189+0000 7ff4cbfff640 1 -- 172.21.15.27:0/2205870609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc009a70 con 0x7ff4d40cb430 2023-08-11T15:51:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.189+0000 7ff4cbfff640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d40cb430 0x7ff4d4149380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c430 tx=0x7ff4bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.190+0000 7ff4d17fa640 1 -- 172.21.15.27:0/2205870609 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc005ce0 con 0x7ff4d40cb430 2023-08-11T15:51:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.190+0000 7ff4d17fa640 1 -- 172.21.15.27:0/2205870609 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc005e40 con 0x7ff4d40cb430 2023-08-11T15:51:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.190+0000 7ff4d17fa640 1 -- 172.21.15.27:0/2205870609 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0072f0 con 0x7ff4d40cb430 2023-08-11T15:51:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.190+0000 7ff4da6fa640 1 -- 172.21.15.27:0/2205870609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d40ace60 con 0x7ff4d40cb430 2023-08-11T15:51:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.190+0000 7ff4da6fa640 1 -- 172.21.15.27:0/2205870609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d40ac6b0 con 0x7ff4d40cb430 2023-08-11T15:51:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.191+0000 7ff4da6fa640 1 -- 172.21.15.27:0/2205870609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d4073990 con 0x7ff4d40cb430 2023-08-11T15:51:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.192+0000 7ff4d17fa640 1 -- 172.21.15.27:0/2205870609 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff4bc007450 con 0x7ff4d40cb430 2023-08-11T15:51:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.193+0000 7ff4d17fa640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b406a080 0x7ff4b406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.193+0000 7ff4d17fa640 1 -- 172.21.15.27:0/2205870609 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff4bc087fb0 con 0x7ff4d40cb430 2023-08-11T15:51:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.194+0000 7ff4d3fff640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b406a080 0x7ff4b406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.195+0000 7ff4d3fff640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b406a080 0x7ff4b406c540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400b910 tx=0x7ff4c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:28.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.197+0000 7ff4d17fa640 1 -- 172.21.15.27:0/2205870609 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ff4bc053310 con 0x7ff4d40cb430 2023-08-11T15:51:28.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.443+0000 7ff4da6fa640 1 -- 172.21.15.27:0/2205870609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff4d40cd520 con 0x7ff4d40cb430 2023-08-11T15:51:28.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.447+0000 7ff4d17fa640 1 -- 172.21.15.27:0/2205870609 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169493 (secure 0 0 0) 0x7ff4bc017020 con 0x7ff4d40cb430 2023-08-11T15:51:28.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 -- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b406a080 msgr2=0x7ff4b406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b406a080 0x7ff4b406c540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400b910 tx=0x7ff4c4008040 comp rx=0 tx=0).stop 2023-08-11T15:51:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 -- 172.21.15.27:0/2205870609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d40cb430 msgr2=0x7ff4d4149380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d40cb430 0x7ff4d4149380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c430 tx=0x7ff4bc00c8f0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 -- 172.21.15.27:0/2205870609 shutdown_connections 2023-08-11T15:51:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff4b406a080 0x7ff4b406c540 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d415a140 0x7ff4d4156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4d40cb430 0x7ff4d4149380 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 --2- 172.21.15.27:0/2205870609 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4d406f5d0 0x7ff4d40c1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 -- 172.21.15.27:0/2205870609 >> 172.21.15.27:0/2205870609 conn(0x7ff4d40afb70 msgr2=0x7ff4d4158720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:28.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.452+0000 7ff4da6fa640 1 -- 172.21.15.27:0/2205870609 shutdown_connections 2023-08-11T15:51:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.453+0000 7ff4da6fa640 1 -- 172.21.15.27:0/2205870609 wait complete. 2023-08-11T15:51:28.474 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":792514541},{"type":"v1","addr":"172.21.15.119:6833","nonce":792514541}]},"active_addr":"172.21.15.119:6833/792514541","active_change":"2023-08-11T15:50:38.680136+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1328954781}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1506123883}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":85848703}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":210191354}]}]} 2023-08-11T15:51:28.479 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-08-11T15:51:28.479 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-08-11T15:51:28.479 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:28.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.715+0000 7f5615671640 1 Processor -- start 2023-08-11T15:51:28.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.715+0000 7f5615671640 1 -- start start 2023-08-11T15:51:28.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.716+0000 7f5615671640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f561015c590 0x7f56100b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.716+0000 7f5615671640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100d1480 0x7f56100d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f5615671640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56100d1e20 con 0x7f56100cd0e0 2023-08-11T15:51:28.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f5615671640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56100d0ed0 con 0x7f561015c590 2023-08-11T15:51:28.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f5615671640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561015c970 con 0x7f56100d1480 2023-08-11T15:51:28.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f560ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f561015c590 0x7f56100b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f560ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f561015c590 0x7f56100b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46346/0 (socket says 172.21.15.27:46346) 2023-08-11T15:51:28.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f5614e70640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f56100cd0e0 0x7f56100cd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48996/0 (socket says 172.21.15.27:48996) 2023-08-11T15:51:28.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f560ffff640 1 -- 172.21.15.27:0/806734523 learned_addr learned my addr 172.21.15.27:0/806734523 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:28.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f560f7fe640 1 --2- 172.21.15.27:0/806734523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100d1480 0x7f56100d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f560ffff640 1 -- 172.21.15.27:0/806734523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100d1480 msgr2=0x7f56100d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:28.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f560ffff640 1 --2- 172.21.15.27:0/806734523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100d1480 0x7f56100d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f560effd640 1 -- 172.21.15.27:0/806734523 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962689755 0 0) 0x7f56100d1e20 con 0x7f56100cd0e0 2023-08-11T15:51:28.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f560ffff640 1 -- 172.21.15.27:0/806734523 >> v1:172.21.15.27:6789/0 conn(0x7f56100cd0e0 legacy=0x7f56100cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f560ffff640 1 -- 172.21.15.27:0/806734523 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56100d0250 con 0x7f561015c590 2023-08-11T15:51:28.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.717+0000 7f560f7fe640 1 --2- 172.21.15.27:0/806734523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100d1480 0x7f56100d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:28.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.719+0000 7f560ffff640 1 --2- 172.21.15.27:0/806734523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 0x7f56100b5440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5600009e50 tx=0x7f560002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf76bf3b1c318b49 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:28.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.719+0000 7f560effd640 1 -- 172.21.15.27:0/806734523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560002c9f0 con 0x7f561015c590 2023-08-11T15:51:28.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.719+0000 7f560effd640 1 -- 172.21.15.27:0/806734523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560002cb50 con 0x7f561015c590 2023-08-11T15:51:28.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.719+0000 7f560effd640 1 -- 172.21.15.27:0/806734523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56000343f0 con 0x7f561015c590 2023-08-11T15:51:28.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.719+0000 7f5615671640 1 -- 172.21.15.27:0/806734523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 msgr2=0x7f56100b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.719+0000 7f5615671640 1 --2- 172.21.15.27:0/806734523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 0x7f56100b5440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5600009e50 tx=0x7f560002be50 comp rx=0 tx=0).stop 2023-08-11T15:51:28.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.720+0000 7f5615671640 1 -- 172.21.15.27:0/806734523 shutdown_connections 2023-08-11T15:51:28.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.720+0000 7f5615671640 1 --2- 172.21.15.27:0/806734523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100d1480 0x7f56100d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.720+0000 7f5615671640 1 --2- 172.21.15.27:0/806734523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 0x7f56100b5440 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.720+0000 7f5615671640 1 -- 172.21.15.27:0/806734523 >> 172.21.15.27:0/806734523 conn(0x7f561006a250 msgr2=0x7f561006a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:28.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.720+0000 7f5615671640 1 -- 172.21.15.27:0/806734523 shutdown_connections 2023-08-11T15:51:28.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.720+0000 7f5615671640 1 -- 172.21.15.27:0/806734523 wait complete. 2023-08-11T15:51:28.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.721+0000 7f5615671640 1 Processor -- start 2023-08-11T15:51:28.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.722+0000 7f5615671640 1 -- start start 2023-08-11T15:51:28.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5615671640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100cd0e0 0x7f56100b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5615671640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100d1480 0x7f5610150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5615671640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 0x7f561015d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5615671640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56100b39a0 con 0x7f561015c590 2023-08-11T15:51:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5615671640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56100b2d20 con 0x7f56100d1480 2023-08-11T15:51:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5615671640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56100b3000 con 0x7f56100cd0e0 2023-08-11T15:51:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5614e70640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 0x7f561015d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5614e70640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 0x7f561015d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46360/0 (socket says 172.21.15.27:46360) 2023-08-11T15:51:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5614e70640 1 -- 172.21.15.27:0/1382205324 learned_addr learned my addr 172.21.15.27:0/1382205324 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f560ffff640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100cd0e0 0x7f56100b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5614e70640 1 -- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100cd0e0 msgr2=0x7f56100b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5614e70640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100cd0e0 0x7f56100b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f560f7fe640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100d1480 0x7f5610150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5614e70640 1 -- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100d1480 msgr2=0x7f5610150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5614e70640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100d1480 0x7f5610150360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.723+0000 7f5614e70640 1 -- 172.21.15.27:0/1382205324 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5600009b30 con 0x7f561015c590 2023-08-11T15:51:28.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.724+0000 7f5614e70640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 0x7f561015d450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f560800c450 tx=0x7f560800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:28.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.724+0000 7f560cff9640 1 -- 172.21.15.27:0/1382205324 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56080074e0 con 0x7f561015c590 2023-08-11T15:51:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.724+0000 7f560cff9640 1 -- 172.21.15.27:0/1382205324 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5608007660 con 0x7f561015c590 2023-08-11T15:51:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.724+0000 7f560cff9640 1 -- 172.21.15.27:0/1382205324 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608004a20 con 0x7f561015c590 2023-08-11T15:51:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.724+0000 7f5615671640 1 -- 172.21.15.27:0/1382205324 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56100b32e0 con 0x7f561015c590 2023-08-11T15:51:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.724+0000 7f5615671640 1 -- 172.21.15.27:0/1382205324 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56100b4300 con 0x7f561015c590 2023-08-11T15:51:28.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.726+0000 7f5615671640 1 -- 172.21.15.27:0/1382205324 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56100b3ae0 con 0x7f561015c590 2023-08-11T15:51:28.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.727+0000 7f560cff9640 1 -- 172.21.15.27:0/1382205324 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5608004b80 con 0x7f561015c590 2023-08-11T15:51:28.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.727+0000 7f560cff9640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f55f006a050 0x7f55f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:28.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.728+0000 7f560cff9640 1 -- 172.21.15.27:0/1382205324 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5608087900 con 0x7f561015c590 2023-08-11T15:51:28.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.728+0000 7f560ffff640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f55f006a050 0x7f55f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:28.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.729+0000 7f560ffff640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f55f006a050 0x7f55f006c510 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5600035000 tx=0x7f5600030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.732+0000 7f560cff9640 1 -- 172.21.15.27:0/1382205324 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f56100b3ae0 con 0x7f561015c590 2023-08-11T15:51:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.934+0000 7f5615671640 1 -- 172.21.15.27:0/1382205324 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56100b3ae0 con 0x7f561015c590 2023-08-11T15:51:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.937+0000 7f560cff9640 1 -- 172.21.15.27:0/1382205324 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7f56100b3ae0 con 0x7f561015c590 2023-08-11T15:51:28.939 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:28.940 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:28.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.940+0000 7f5615671640 1 -- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f55f006a050 msgr2=0x7f55f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.940+0000 7f5615671640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f55f006a050 0x7f55f006c510 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5600035000 tx=0x7f5600030000 comp rx=0 tx=0).stop 2023-08-11T15:51:28.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.940+0000 7f5615671640 1 -- 172.21.15.27:0/1382205324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 msgr2=0x7f561015d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:28.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.940+0000 7f5615671640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 0x7f561015d450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f560800c450 tx=0x7f560800c910 comp rx=0 tx=0).stop 2023-08-11T15:51:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.941+0000 7f5615671640 1 -- 172.21.15.27:0/1382205324 shutdown_connections 2023-08-11T15:51:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.941+0000 7f5615671640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f55f006a050 0x7f55f006c510 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.941+0000 7f5615671640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f561015c590 0x7f561015d450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.941+0000 7f5615671640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100d1480 0x7f5610150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.941+0000 7f5615671640 1 --2- 172.21.15.27:0/1382205324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56100cd0e0 0x7f56100b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.941+0000 7f5615671640 1 -- 172.21.15.27:0/1382205324 >> 172.21.15.27:0/1382205324 conn(0x7f561006a250 msgr2=0x7f561006a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.941+0000 7f5615671640 1 -- 172.21.15.27:0/1382205324 shutdown_connections 2023-08-11T15:51:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:28.941+0000 7f5615671640 1 -- 172.21.15.27:0/1382205324 wait complete. 2023-08-11T15:51:28.962 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-08-11T15:51:28.962 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:29.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.198+0000 7f0fb0c6a640 1 Processor -- start 2023-08-11T15:51:29.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.199+0000 7f0fb0c6a640 1 -- start start 2023-08-11T15:51:29.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.200+0000 7f0fb0c6a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0fac15a020 0x7f0fac15e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.200+0000 7f0fb0c6a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0fac0c9600 0x7f0fac071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.200+0000 7f0fb0c6a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fac0c85f0 con 0x7f0fac06f5d0 2023-08-11T15:51:29.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.200+0000 7f0fb0c6a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac15eae0 con 0x7f0fac15a020 2023-08-11T15:51:29.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.200+0000 7f0fb0c6a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac071980 con 0x7f0fac0c9600 2023-08-11T15:51:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.201+0000 7f0fab7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0fac15a020 0x7f0fac15e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.201+0000 7f0fab7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0fac15a020 0x7f0fac15e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46202/0 (socket says 172.21.15.27:46202) 2023-08-11T15:51:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.201+0000 7f0fab7fe640 1 -- 172.21.15.27:0/471331524 learned_addr learned my addr 172.21.15.27:0/471331524 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.201+0000 7f0fa3fff640 1 --2- 172.21.15.27:0/471331524 >> v2:172.21.15.119:3300/0 conn(0x7f0fac0c9600 0x7f0fac071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.201+0000 7f0fab7fe640 1 -- 172.21.15.27:0/471331524 >> v2:172.21.15.119:3300/0 conn(0x7f0fac0c9600 msgr2=0x7f0fac071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.201+0000 7f0fab7fe640 1 --2- 172.21.15.27:0/471331524 >> v2:172.21.15.119:3300/0 conn(0x7f0fac0c9600 0x7f0fac071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.201+0000 7f0fab7fe640 1 -- 172.21.15.27:0/471331524 >> v1:172.21.15.119:6789/0 conn(0x7f0fac06f5d0 legacy=0x7f0fac06b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.201+0000 7f0fab7fe640 1 -- 172.21.15.27:0/471331524 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac0c8840 con 0x7f0fac15a020 2023-08-11T15:51:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.201+0000 7f0fa3fff640 1 --2- 172.21.15.27:0/471331524 >> v2:172.21.15.119:3300/0 conn(0x7f0fac0c9600 0x7f0fac071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:29.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.203+0000 7f0fab7fe640 1 --2- 172.21.15.27:0/471331524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac15a020 0x7f0fac15e5a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0c8580 tx=0x7f0f9c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b57380ce9260a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.203+0000 7f0faaffd640 1 -- 172.21.15.27:0/471331524 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c031800 con 0x7f0fac15a020 2023-08-11T15:51:29.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.203+0000 7f0faaffd640 1 -- 172.21.15.27:0/471331524 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9c031960 con 0x7f0fac15a020 2023-08-11T15:51:29.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.203+0000 7f0faaffd640 1 -- 172.21.15.27:0/471331524 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c031cf0 con 0x7f0fac15a020 2023-08-11T15:51:29.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.204+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/471331524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac15a020 msgr2=0x7f0fac15e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.204+0000 7f0fb0c6a640 1 --2- 172.21.15.27:0/471331524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac15a020 0x7f0fac15e5a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0c8580 tx=0x7f0f9c02b8a0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.204+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/471331524 shutdown_connections 2023-08-11T15:51:29.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.204+0000 7f0fb0c6a640 1 --2- 172.21.15.27:0/471331524 >> v2:172.21.15.119:3300/0 conn(0x7f0fac0c9600 0x7f0fac071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.204+0000 7f0fb0c6a640 1 --2- 172.21.15.27:0/471331524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac15a020 0x7f0fac15e5a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.204+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/471331524 >> 172.21.15.27:0/471331524 conn(0x7f0fac0afb70 msgr2=0x7f0fac0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.204+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/471331524 shutdown_connections 2023-08-11T15:51:29.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.204+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/471331524 wait complete. 2023-08-11T15:51:29.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.206+0000 7f0fb0c6a640 1 Processor -- start 2023-08-11T15:51:29.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.206+0000 7f0fb0c6a640 1 -- start start 2023-08-11T15:51:29.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.207+0000 7f0fb0c6a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac06f5d0 0x7f0fac15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.207+0000 7f0fb0c6a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0fac0c9600 0x7f0fac156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.207+0000 7f0fb0c6a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac15a020 0x7f0fac0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.207+0000 7f0fb0c6a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0c35f0 con 0x7f0fac06f5d0 2023-08-11T15:51:29.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.207+0000 7f0fb0c6a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0cce60 con 0x7f0fac15a020 2023-08-11T15:51:29.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.207+0000 7f0fb0c6a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac15ae90 con 0x7f0fac0c9600 2023-08-11T15:51:29.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.207+0000 7f0fab7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac06f5d0 0x7f0fac15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.207+0000 7f0fab7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac06f5d0 0x7f0fac15a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46208/0 (socket says 172.21.15.27:46208) 2023-08-11T15:51:29.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.207+0000 7f0fab7fe640 1 -- 172.21.15.27:0/2243666963 learned_addr learned my addr 172.21.15.27:0/2243666963 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.208+0000 7f0fabfff640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac15a020 0x7f0fac0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.208+0000 7f0fa3fff640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0fac0c9600 0x7f0fac156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.208+0000 7f0fab7fe640 1 -- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0fac0c9600 msgr2=0x7f0fac156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.208+0000 7f0fab7fe640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0fac0c9600 0x7f0fac156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.208+0000 7f0fab7fe640 1 -- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac15a020 msgr2=0x7f0fac0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.208+0000 7f0fab7fe640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac15a020 0x7f0fac0cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.208+0000 7f0fab7fe640 1 -- 172.21.15.27:0/2243666963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c0094d0 con 0x7f0fac06f5d0 2023-08-11T15:51:29.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.208+0000 7f0fab7fe640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac06f5d0 0x7f0fac15a950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c000c00 tx=0x7f0f9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.208+0000 7f0fa8ff9640 1 -- 172.21.15.27:0/2243666963 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c03ac60 con 0x7f0fac06f5d0 2023-08-11T15:51:29.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.209+0000 7f0fa8ff9640 1 -- 172.21.15.27:0/2243666963 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9c03adc0 con 0x7f0fac06f5d0 2023-08-11T15:51:29.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.209+0000 7f0fa8ff9640 1 -- 172.21.15.27:0/2243666963 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c044610 con 0x7f0fac06f5d0 2023-08-11T15:51:29.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.209+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/2243666963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fac0c2cc0 con 0x7f0fac06f5d0 2023-08-11T15:51:29.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.209+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/2243666963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac0c3280 con 0x7f0fac06f5d0 2023-08-11T15:51:29.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.210+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/2243666963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fac04e420 con 0x7f0fac06f5d0 2023-08-11T15:51:29.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.211+0000 7f0fa8ff9640 1 -- 172.21.15.27:0/2243666963 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0f9c04d450 con 0x7f0fac06f5d0 2023-08-11T15:51:29.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.212+0000 7f0fa8ff9640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0f8c06a080 0x7f0f8c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.212+0000 7f0fa8ff9640 1 -- 172.21.15.27:0/2243666963 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0f9c02e070 con 0x7f0fac06f5d0 2023-08-11T15:51:29.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.213+0000 7f0fa3fff640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0f8c06a080 0x7f0f8c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.214+0000 7f0fa3fff640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0f8c06a080 0x7f0f8c06c540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0cdd50 tx=0x7f0f94008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.216+0000 7f0fa8ff9640 1 -- 172.21.15.27:0/2243666963 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0f9c075f50 con 0x7f0fac06f5d0 2023-08-11T15:51:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.410+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/2243666963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fac0723d0 con 0x7f0fac06f5d0 2023-08-11T15:51:29.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.413+0000 7f0fa8ff9640 1 -- 172.21.15.27:0/2243666963 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7f0f9c079570 con 0x7f0fac06f5d0 2023-08-11T15:51:29.415 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:29.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.415+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0f8c06a080 msgr2=0x7f0f8c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.415+0000 7f0fb0c6a640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0f8c06a080 0x7f0f8c06c540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0cdd50 tx=0x7f0f94008040 comp rx=0 tx=0).stop 2023-08-11T15:51:29.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.415+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/2243666963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac06f5d0 msgr2=0x7f0fac15a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.415+0000 7f0fb0c6a640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac06f5d0 0x7f0fac15a950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c000c00 tx=0x7f0f9c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.415+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/2243666963 shutdown_connections 2023-08-11T15:51:29.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.415+0000 7f0fb0c6a640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0f8c06a080 0x7f0f8c06c540 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.415+0000 7f0fb0c6a640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0fac0c9600 0x7f0fac156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.415+0000 7f0fb0c6a640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac15a020 0x7f0fac0cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.415+0000 7f0fb0c6a640 1 --2- 172.21.15.27:0/2243666963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fac06f5d0 0x7f0fac15a950 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.415+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/2243666963 >> 172.21.15.27:0/2243666963 conn(0x7f0fac0afb70 msgr2=0x7f0fac0aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.416+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/2243666963 shutdown_connections 2023-08-11T15:51:29.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.416+0000 7f0fb0c6a640 1 -- 172.21.15.27:0/2243666963 wait complete. 2023-08-11T15:51:29.439 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-08-11T15:51:29.440 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-08-11T15:51:29.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-08-11T15:51:29.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-08-11T15:51:29.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-08-11T15:51:29.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-08-11T15:51:29.443 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-08-11T15:51:29.443 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-08-11T15:51:29.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.770+0000 7f48b8a3f640 1 Processor -- start 2023-08-11T15:51:29.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.770+0000 7f48b8a3f640 1 -- start start 2023-08-11T15:51:29.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.772+0000 7f48b8a3f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f48b415c290 0x7f48b41607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.772+0000 7f48b8a3f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48b40c9f70 0x7f48b40ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.772+0000 7f48b8a3f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b4160d30 con 0x7f48b40abe20 2023-08-11T15:51:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.772+0000 7f48b8a3f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b40c9b50 con 0x7f48b415c290 2023-08-11T15:51:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.772+0000 7f48b8a3f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b415d680 con 0x7f48b40c9f70 2023-08-11T15:51:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.773+0000 7f48b2d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f48b40abe20 0x7f48b40cbfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48392/0 (socket says 172.21.15.27:48392) 2023-08-11T15:51:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.773+0000 7f48b2d9d640 1 -- 172.21.15.27:0/1005925324 learned_addr learned my addr 172.21.15.27:0/1005925324 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.773+0000 7f48b259c640 1 --2- 172.21.15.27:0/1005925324 >> v2:172.21.15.27:3300/0 conn(0x7f48b415c290 0x7f48b41607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.773+0000 7f48b1d9b640 1 --2- 172.21.15.27:0/1005925324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48b40c9f70 0x7f48b40ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.774+0000 7f48b159a640 1 -- 172.21.15.27:0/1005925324 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658301749 0 0) 0x7f48b4160d30 con 0x7f48b40abe20 2023-08-11T15:51:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.774+0000 7f48b159a640 1 -- 172.21.15.27:0/1005925324 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f489c003520 con 0x7f48b40abe20 2023-08-11T15:51:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.774+0000 7f48b259c640 1 -- 172.21.15.27:0/1005925324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48b40c9f70 msgr2=0x7f48b40ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:29.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.774+0000 7f48b259c640 1 --2- 172.21.15.27:0/1005925324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48b40c9f70 0x7f48b40ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.774+0000 7f48b259c640 1 -- 172.21.15.27:0/1005925324 >> v1:172.21.15.27:6789/0 conn(0x7f48b40abe20 legacy=0x7f48b40cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.774+0000 7f48b259c640 1 -- 172.21.15.27:0/1005925324 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b40c8ed0 con 0x7f48b415c290 2023-08-11T15:51:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.776+0000 7f48b259c640 1 --2- 172.21.15.27:0/1005925324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48b415c290 0x7f48b41607f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48a80097a0 tx=0x7f48a802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dd2f29eb12cb4e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.777+0000 7f48b159a640 1 -- 172.21.15.27:0/1005925324 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a80316f0 con 0x7f48b415c290 2023-08-11T15:51:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.777+0000 7f48b159a640 1 -- 172.21.15.27:0/1005925324 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a8031850 con 0x7f48b415c290 2023-08-11T15:51:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.777+0000 7f48b159a640 1 -- 172.21.15.27:0/1005925324 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8031be0 con 0x7f48b415c290 2023-08-11T15:51:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.778+0000 7f48b8a3f640 1 -- 172.21.15.27:0/1005925324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48b415c290 msgr2=0x7f48b41607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.778+0000 7f48b8a3f640 1 --2- 172.21.15.27:0/1005925324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48b415c290 0x7f48b41607f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48a80097a0 tx=0x7f48a802b790 comp rx=0 tx=0).stop 2023-08-11T15:51:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.778+0000 7f48b8a3f640 1 -- 172.21.15.27:0/1005925324 shutdown_connections 2023-08-11T15:51:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.778+0000 7f48b8a3f640 1 --2- 172.21.15.27:0/1005925324 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48b40c9f70 0x7f48b40ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.778+0000 7f48b8a3f640 1 --2- 172.21.15.27:0/1005925324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48b415c290 0x7f48b41607f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.778+0000 7f48b8a3f640 1 -- 172.21.15.27:0/1005925324 >> 172.21.15.27:0/1005925324 conn(0x7f48b40cdfd0 msgr2=0x7f48b40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.779+0000 7f48b8a3f640 1 -- 172.21.15.27:0/1005925324 shutdown_connections 2023-08-11T15:51:29.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.779+0000 7f48b8a3f640 1 -- 172.21.15.27:0/1005925324 wait complete. 2023-08-11T15:51:29.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.782+0000 7f48b8a3f640 1 Processor -- start 2023-08-11T15:51:29.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.782+0000 7f48b8a3f640 1 -- start start 2023-08-11T15:51:29.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.784+0000 7f48b8a3f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b40abe20 0x7f48b41fe530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.784+0000 7f48b8a3f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48b40c9f70 0x7f48b40aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.784+0000 7f48b8a3f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48b415c290 0x7f48b41591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.785+0000 7f48b8a3f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41fea70 con 0x7f48b415c290 2023-08-11T15:51:29.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.785+0000 7f48b259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b40abe20 0x7f48b41fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.785+0000 7f48b259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b40abe20 0x7f48b41fe530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40642/0 (socket says 172.21.15.27:40642) 2023-08-11T15:51:29.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.785+0000 7f48b259c640 1 -- 172.21.15.27:0/453298059 learned_addr learned my addr 172.21.15.27:0/453298059 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.785+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40c1f90 con 0x7f48b40abe20 2023-08-11T15:51:29.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.785+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40c2270 con 0x7f48b40c9f70 2023-08-11T15:51:29.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.785+0000 7f48b1d9b640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48b40c9f70 0x7f48b40aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.785+0000 7f48b2d9d640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48b415c290 0x7f48b41591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.786+0000 7f48b2d9d640 1 -- 172.21.15.27:0/453298059 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48b40c9f70 msgr2=0x7f48b40aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.786+0000 7f48b2d9d640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48b40c9f70 0x7f48b40aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.786+0000 7f48b2d9d640 1 -- 172.21.15.27:0/453298059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b40abe20 msgr2=0x7f48b41fe530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.786+0000 7f48b2d9d640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b40abe20 0x7f48b41fe530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.786+0000 7f48b2d9d640 1 -- 172.21.15.27:0/453298059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a8009400 con 0x7f48b415c290 2023-08-11T15:51:29.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.787+0000 7fb951f86640 1 Processor -- start 2023-08-11T15:51:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.787+0000 7f48b2d9d640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48b415c290 0x7f48b41591c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f48ac000f80 tx=0x7f48ac00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.788+0000 7f48a2ffd640 1 -- 172.21.15.27:0/453298059 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac004d60 con 0x7f48b415c290 2023-08-11T15:51:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.787+0000 7fb951f86640 1 -- start start 2023-08-11T15:51:29.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.789+0000 7fb951f86640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb94403a3c0 0x7fb94404f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.789+0000 7fb951f86640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb9440393f0 0x7fb94404fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.789+0000 7fb951f86640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9440077c0 con 0x7fb944050230 2023-08-11T15:51:29.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.789+0000 7fb951f86640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9440068d0 con 0x7fb94403a3c0 2023-08-11T15:51:29.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.789+0000 7fb951f86640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb944006bb0 con 0x7fb9440393f0 2023-08-11T15:51:29.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.788+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b40c2550 con 0x7f48b415c290 2023-08-11T15:51:29.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.788+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b40c2b10 con 0x7f48b415c290 2023-08-11T15:51:29.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.789+0000 7f48a2ffd640 1 -- 172.21.15.27:0/453298059 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac00a640 con 0x7f48b415c290 2023-08-11T15:51:29.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.789+0000 7f48a2ffd640 1 -- 172.21.15.27:0/453298059 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac00aa30 con 0x7f48b415c290 2023-08-11T15:51:29.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.790+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4898000fc0 con 0x7f48b415c290 2023-08-11T15:51:29.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.790+0000 7fb94b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb94403a3c0 0x7fb94404f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.790+0000 7fb94b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb94403a3c0 0x7fb94404f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46230/0 (socket says 172.21.15.27:46230) 2023-08-11T15:51:29.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.790+0000 7fb94b7fe640 1 -- 172.21.15.27:0/1551702024 learned_addr learned my addr 172.21.15.27:0/1551702024 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.790+0000 7fb94b7fe640 1 -- 172.21.15.27:0/1551702024 >> v2:172.21.15.119:3300/0 conn(0x7fb9440393f0 msgr2=0x7fb94404fb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:51:29.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.790+0000 7fb94b7fe640 1 --2- 172.21.15.27:0/1551702024 >> v2:172.21.15.119:3300/0 conn(0x7fb9440393f0 0x7fb94404fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.790+0000 7fb94b7fe640 1 -- 172.21.15.27:0/1551702024 >> v1:172.21.15.27:6789/0 conn(0x7fb944050230 legacy=0x7fb9440de740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:51:29.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.790+0000 7fb94b7fe640 1 -- 172.21.15.27:0/1551702024 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb944006e90 con 0x7fb94403a3c0 2023-08-11T15:51:29.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.791+0000 7f85c776a640 1 Processor -- start 2023-08-11T15:51:29.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.792+0000 7f85c776a640 1 -- start start 2023-08-11T15:51:29.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.792+0000 7fb94b7fe640 1 --2- 172.21.15.27:0/1551702024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb94403a3c0 0x7fb94404f5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb94c078c80 tx=0x7fb94c0677b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ae770d395bfde server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.792+0000 7fb94a7fc640 1 -- 172.21.15.27:0/1551702024 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0b1400 con 0x7fb94403a3c0 2023-08-11T15:51:29.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.793+0000 7f205a206640 1 Processor -- start 2023-08-11T15:51:29.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.793+0000 7f205a206640 1 -- start start 2023-08-11T15:51:29.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.793+0000 7fb94a7fc640 1 -- 172.21.15.27:0/1551702024 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c0ab070 con 0x7fb94403a3c0 2023-08-11T15:51:29.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.793+0000 7fb94a7fc640 1 -- 172.21.15.27:0/1551702024 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0b1400 con 0x7fb94403a3c0 2023-08-11T15:51:29.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.794+0000 7fb951f86640 1 -- 172.21.15.27:0/1551702024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb94403a3c0 msgr2=0x7fb94404f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.794+0000 7fb951f86640 1 --2- 172.21.15.27:0/1551702024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb94403a3c0 0x7fb94404f5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb94c078c80 tx=0x7fb94c0677b0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.794+0000 7f85c776a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85c00cd050 0x7f85c015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f85c5506640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f85c00ca8e0 0x7f85c00cacc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48402/0 (socket says 172.21.15.27:48402) 2023-08-11T15:51:29.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.793+0000 7f48a2ffd640 1 -- 172.21.15.27:0/453298059 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f48ac00ab90 con 0x7f48b415c290 2023-08-11T15:51:29.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f48a2ffd640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f489406a0f0 0x7f489406c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f48a2ffd640 1 -- 172.21.15.27:0/453298059 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f48ac088720 con 0x7f48b415c290 2023-08-11T15:51:29.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f48a2ffd640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] conn(0x7f4894073dd0 0x7f4894076210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f48a2ffd640 1 -- 172.21.15.27:0/453298059 --> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48940768e0 con 0x7f4894073dd0 2023-08-11T15:51:29.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f48a2ffd640 1 -- 172.21.15.27:0/453298059 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ac088ad0 con 0x7f48b415c290 2023-08-11T15:51:29.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.793+0000 7f52f31fd640 1 Processor -- start 2023-08-11T15:51:29.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.794+0000 7f52f31fd640 1 -- start start 2023-08-11T15:51:29.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f48b259c640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f489406a0f0 0x7f489406c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f48b1d9b640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] conn(0x7f4894073dd0 0x7f4894076210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.799+0000 7f48b1d9b640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] conn(0x7f4894073dd0 0x7f4894076210 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7fb951f86640 1 -- 172.21.15.27:0/1551702024 shutdown_connections 2023-08-11T15:51:29.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7fb951f86640 1 --2- 172.21.15.27:0/1551702024 >> v2:172.21.15.119:3300/0 conn(0x7fb9440393f0 0x7fb94404fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7fb951f86640 1 --2- 172.21.15.27:0/1551702024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb94403a3c0 0x7fb94404f5e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7fb951f86640 1 -- 172.21.15.27:0/1551702024 >> 172.21.15.27:0/1551702024 conn(0x7fb944009fe0 msgr2=0x7fb94400a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7fb951f86640 1 -- 172.21.15.27:0/1551702024 shutdown_connections 2023-08-11T15:51:29.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7fb951f86640 1 -- 172.21.15.27:0/1551702024 wait complete. 2023-08-11T15:51:29.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.799+0000 7fb951f86640 1 Processor -- start 2023-08-11T15:51:29.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f85c776a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85c00cd4b0 0x7f85c015c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f85c776a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c00ccbc0 con 0x7f85c00ca8e0 2023-08-11T15:51:29.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f85c776a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c015cc70 con 0x7f85c00cd050 2023-08-11T15:51:29.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f85c776a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00cc590 con 0x7f85c00cd4b0 2023-08-11T15:51:29.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f85c4d05640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85c00cd050 0x7f85c015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f85c4d05640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85c00cd050 0x7f85c015c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46232/0 (socket says 172.21.15.27:46232) 2023-08-11T15:51:29.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f85c4d05640 1 -- 172.21.15.27:0/840552255 learned_addr learned my addr 172.21.15.27:0/840552255 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.797+0000 7f85c5d07640 1 --2- 172.21.15.27:0/840552255 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85c00cd4b0 0x7f85c015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.799+0000 7f85c4d05640 1 -- 172.21.15.27:0/840552255 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85c00cd4b0 msgr2=0x7f85c015c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:29.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.799+0000 7f85c4d05640 1 --2- 172.21.15.27:0/840552255 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85c00cd4b0 0x7f85c015c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.799+0000 7f85c4d05640 1 -- 172.21.15.27:0/840552255 >> v1:172.21.15.27:6789/0 conn(0x7f85c00ca8e0 legacy=0x7f85c00cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.799+0000 7f85c4d05640 1 -- 172.21.15.27:0/840552255 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c015cf20 con 0x7f85c00cd050 2023-08-11T15:51:29.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f205a206640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f20540ceac0 0x7f2054159460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f205a206640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f20540c9030 0x7f205406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f205a206640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 0x7f205415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f205a206640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2054070990 con 0x7f20540ceac0 2023-08-11T15:51:29.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f205a206640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205406b9c0 con 0x7f20540c9030 2023-08-11T15:51:29.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.795+0000 7f205a206640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205415f220 con 0x7f2054070520 2023-08-11T15:51:29.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f2053fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 0x7f205415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f2053fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 0x7f205415beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53414/0 (socket says 172.21.15.27:53414) 2023-08-11T15:51:29.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f2053fff640 1 -- 172.21.15.27:0/3399418169 learned_addr learned my addr 172.21.15.27:0/3399418169 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.797+0000 7f20537fe640 1 --2- 172.21.15.27:0/3399418169 >> v2:172.21.15.27:3300/0 conn(0x7f20540ceac0 0x7f2054159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.797+0000 7f2052ffd640 1 --2- 172.21.15.27:0/3399418169 >> v2:172.21.15.119:3300/0 conn(0x7f20540c9030 0x7f205406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.798+0000 7f20537fe640 1 -- 172.21.15.27:0/3399418169 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 msgr2=0x7f205415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:29.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.798+0000 7f20537fe640 1 --2- 172.21.15.27:0/3399418169 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 0x7f205415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.798+0000 7f20537fe640 1 -- 172.21.15.27:0/3399418169 >> v2:172.21.15.119:3300/0 conn(0x7f20540c9030 msgr2=0x7f205406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:29.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.798+0000 7f20537fe640 1 --2- 172.21.15.27:0/3399418169 >> v2:172.21.15.119:3300/0 conn(0x7f20540c9030 0x7f205406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.798+0000 7f20537fe640 1 -- 172.21.15.27:0/3399418169 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20540c9a80 con 0x7f20540ceac0 2023-08-11T15:51:29.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f52f31fd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec152a70 0x7f52ec15bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f52f31fd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f52ec15c4e0 0x7f52ec0ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f52f31fd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ec1589b0 con 0x7f52ec06d6c0 2023-08-11T15:51:29.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f52f31fd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ec152ef0 con 0x7f52ec15c4e0 2023-08-11T15:51:29.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.796+0000 7f52f31fd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec15c9f0 con 0x7f52ec152a70 2023-08-11T15:51:29.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.797+0000 7f52ebfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec152a70 0x7f52ec15bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.797+0000 7f52ebfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec152a70 0x7f52ec15bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53420/0 (socket says 172.21.15.27:53420) 2023-08-11T15:51:29.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.797+0000 7f52ebfff640 1 -- 172.21.15.27:0/1442667187 learned_addr learned my addr 172.21.15.27:0/1442667187 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.797+0000 7f52f179a640 1 --2- 172.21.15.27:0/1442667187 >> v2:172.21.15.119:3300/0 conn(0x7f52ec15c4e0 0x7f52ec0ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.798+0000 7f52ebfff640 1 -- 172.21.15.27:0/1442667187 >> v2:172.21.15.119:3300/0 conn(0x7f52ec15c4e0 msgr2=0x7f52ec0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:29.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.798+0000 7f52ebfff640 1 --2- 172.21.15.27:0/1442667187 >> v2:172.21.15.119:3300/0 conn(0x7f52ec15c4e0 0x7f52ec0ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.798+0000 7f52ebfff640 1 -- 172.21.15.27:0/1442667187 >> v1:172.21.15.119:6789/0 conn(0x7f52ec06d6c0 legacy=0x7f52ec159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.798+0000 7f52ebfff640 1 -- 172.21.15.27:0/1442667187 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec15ccd0 con 0x7f52ec152a70 2023-08-11T15:51:29.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.798+0000 7f52f179a640 1 --2- 172.21.15.27:0/1442667187 >> v2:172.21.15.119:3300/0 conn(0x7f52ec15c4e0 0x7f52ec0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:29.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.799+0000 7f52ebfff640 1 --2- 172.21.15.27:0/1442667187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec152a70 0x7f52ec15bfa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009e50 tx=0x7f52dc02bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e3653d2089b602e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.799+0000 7f20537fe640 1 --2- 172.21.15.27:0/3399418169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20540ceac0 0x7f2054159460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f20540caa20 tx=0x7f204402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4948be26e8563e0b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.800+0000 7f52eb7fe640 1 -- 172.21.15.27:0/1442667187 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc02cb90 con 0x7f52ec152a70 2023-08-11T15:51:29.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.800+0000 7f52eb7fe640 1 -- 172.21.15.27:0/1442667187 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc02ccf0 con 0x7f52ec152a70 2023-08-11T15:51:29.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.800+0000 7f52eb7fe640 1 -- 172.21.15.27:0/1442667187 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc034610 con 0x7f52ec152a70 2023-08-11T15:51:29.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.800+0000 7f48a2ffd640 1 -- 172.21.15.27:0/453298059 <== osd.0 v2:172.21.15.27:6800/3363308208 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f48940768e0 con 0x7f4894073dd0 2023-08-11T15:51:29.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.800+0000 7fb951f86640 1 -- start start 2023-08-11T15:51:29.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7f52f31fd640 1 -- 172.21.15.27:0/1442667187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec152a70 msgr2=0x7f52ec15bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7f52f31fd640 1 --2- 172.21.15.27:0/1442667187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec152a70 0x7f52ec15bfa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009e50 tx=0x7f52dc02bff0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7f85c4d05640 1 --2- 172.21.15.27:0/840552255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c00cd050 0x7f85c015c1f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85bc008870 tx=0x7f85bc02df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5189286ec1836f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7f52f31fd640 1 -- 172.21.15.27:0/1442667187 shutdown_connections 2023-08-11T15:51:29.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7f52f31fd640 1 --2- 172.21.15.27:0/1442667187 >> v2:172.21.15.119:3300/0 conn(0x7f52ec15c4e0 0x7f52ec0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7f52f31fd640 1 --2- 172.21.15.27:0/1442667187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec152a70 0x7f52ec15bfa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7f52f31fd640 1 -- 172.21.15.27:0/1442667187 >> 172.21.15.27:0/1442667187 conn(0x7f52ec0afb00 msgr2=0x7f52ec0afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7f48b259c640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f489406a0f0 0x7f489406c5b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48a802b7d0 tx=0x7f48a8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7fb951f86640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb9440393f0 0x7fb94416c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7fb951f86640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94403a3c0 0x7fb94404d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7fb951f86640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb94416a8f0 0x7fb9441676e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7fb951f86640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94416d700 con 0x7fb94416a8f0 2023-08-11T15:51:29.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f85b7fff640 1 -- 172.21.15.27:0/840552255 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc02ed40 con 0x7f85c00cd050 2023-08-11T15:51:29.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f85b7fff640 1 -- 172.21.15.27:0/840552255 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc032070 con 0x7f85c00cd050 2023-08-11T15:51:29.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f85b7fff640 1 -- 172.21.15.27:0/840552255 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc02eea0 con 0x7f85c00cd050 2023-08-11T15:51:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb94416a8f0 0x7fb9441676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb94416a8f0 0x7fb9441676e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46262/0 (socket says 172.21.15.27:46262) 2023-08-11T15:51:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7fb94bfff640 1 -- 172.21.15.27:0/990259152 learned_addr learned my addr 172.21.15.27:0/990259152 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7fb94b7fe640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb9440393f0 0x7fb94416c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb944189140 con 0x7fb94403a3c0 2023-08-11T15:51:29.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f52f31fd640 1 -- 172.21.15.27:0/1442667187 shutdown_connections 2023-08-11T15:51:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f52f31fd640 1 -- 172.21.15.27:0/1442667187 wait complete. 2023-08-11T15:51:29.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7fb94affd640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94403a3c0 0x7fb94404d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94416ad80 con 0x7fb9440393f0 2023-08-11T15:51:29.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.803+0000 7fb94bfff640 1 -- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb9440393f0 msgr2=0x7fb94416c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.803+0000 7fb94bfff640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb9440393f0 0x7fb94416c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.803+0000 7fb94bfff640 1 -- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94403a3c0 msgr2=0x7fb94404d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.803+0000 7fb94bfff640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94403a3c0 0x7fb94404d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.803+0000 7fb94bfff640 1 -- 172.21.15.27:0/990259152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c073490 con 0x7fb94416a8f0 2023-08-11T15:51:29.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.803+0000 7fb94b7fe640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb9440393f0 0x7fb94416c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:29.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.803+0000 7fb94affd640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94403a3c0 0x7fb94404d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:29.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.804+0000 7fb94bfff640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb94416a8f0 0x7fb9441676e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb94000b3f0 tx=0x7fb94000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.804+0000 7fb937fff640 1 -- 172.21.15.27:0/990259152 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940011830 con 0x7fb94416a8f0 2023-08-11T15:51:29.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.804+0000 7fb937fff640 1 -- 172.21.15.27:0/990259152 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb940011990 con 0x7fb94416a8f0 2023-08-11T15:51:29.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.804+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94416b060 con 0x7fb94416a8f0 2023-08-11T15:51:29.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.805+0000 7f52f31fd640 1 Processor -- start 2023-08-11T15:51:29.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.806+0000 7f52f31fd640 1 -- start start 2023-08-11T15:51:29.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.807+0000 7f85c776a640 1 -- 172.21.15.27:0/840552255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c00cd050 msgr2=0x7f85c015c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.807+0000 7f85c776a640 1 --2- 172.21.15.27:0/840552255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c00cd050 0x7f85c015c1f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85bc008870 tx=0x7f85bc02df90 comp rx=0 tx=0).stop 2023-08-11T15:51:29.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.806+0000 7fb937fff640 1 -- 172.21.15.27:0/990259152 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940011d40 con 0x7fb94416a8f0 2023-08-11T15:51:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.806+0000 7fb937fff640 1 -- 172.21.15.27:0/990259152 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb94001a400 con 0x7fb94416a8f0 2023-08-11T15:51:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.807+0000 7f52f31fd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec06d6c0 0x7f52ec15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f52f31fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec152a70 0x7f52ec0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7f20527fc640 1 -- 172.21.15.27:0/3399418169 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204402d040 con 0x7f20540ceac0 2023-08-11T15:51:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.801+0000 7f20527fc640 1 -- 172.21.15.27:0/3399418169 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2044031810 con 0x7f20540ceac0 2023-08-11T15:51:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f205a206640 1 -- 172.21.15.27:0/3399418169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20540ceac0 msgr2=0x7f2054159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f205a206640 1 --2- 172.21.15.27:0/3399418169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20540ceac0 0x7f2054159460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f20540caa20 tx=0x7f204402b8b0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f52f31fd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec15c4e0 0x7f52ec0c3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f205a206640 1 -- 172.21.15.27:0/3399418169 shutdown_connections 2023-08-11T15:51:29.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f205a206640 1 --2- 172.21.15.27:0/3399418169 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 0x7f205415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f205a206640 1 --2- 172.21.15.27:0/3399418169 >> v2:172.21.15.119:3300/0 conn(0x7f20540c9030 0x7f205406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f205a206640 1 --2- 172.21.15.27:0/3399418169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20540ceac0 0x7f2054159460 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f205a206640 1 -- 172.21.15.27:0/3399418169 >> 172.21.15.27:0/3399418169 conn(0x7f20540cdfd0 msgr2=0x7f20540ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f205a206640 1 -- 172.21.15.27:0/3399418169 shutdown_connections 2023-08-11T15:51:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.802+0000 7f205a206640 1 -- 172.21.15.27:0/3399418169 wait complete. 2023-08-11T15:51:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.805+0000 7f205a206640 1 Processor -- start 2023-08-11T15:51:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.806+0000 7f205a206640 1 -- start start 2023-08-11T15:51:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.807+0000 7f205a206640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 0x7f20540c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.807+0000 7f205a206640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20540c9030 0x7f20540c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7fb937fff640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb93006a120 0x7fb93006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f85c776a640 1 -- 172.21.15.27:0/840552255 shutdown_connections 2023-08-11T15:51:29.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f85c776a640 1 --2- 172.21.15.27:0/840552255 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85c00cd4b0 0x7f85c015c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f85c776a640 1 --2- 172.21.15.27:0/840552255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c00cd050 0x7f85c015c1f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f85c776a640 1 -- 172.21.15.27:0/840552255 >> 172.21.15.27:0/840552255 conn(0x7f85c0069e90 msgr2=0x7f85c00afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7fb94b7fe640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb93006a120 0x7fb93006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f52ebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec152a70 0x7f52ec0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f52f0f99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec06d6c0 0x7f52ec15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f52f0f99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec06d6c0 0x7f52ec15b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40672/0 (socket says 172.21.15.27:40672) 2023-08-11T15:51:29.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f52ebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec152a70 0x7f52ec0ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46276/0 (socket says 172.21.15.27:46276) 2023-08-11T15:51:29.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f20537fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 0x7f20540c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94416b620 con 0x7fb94416a8f0 2023-08-11T15:51:29.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f85c776a640 1 -- 172.21.15.27:0/840552255 shutdown_connections 2023-08-11T15:51:29.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f2052ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20540c9030 0x7f20540c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f52f0f99640 1 -- 172.21.15.27:0/685600090 learned_addr learned my addr 172.21.15.27:0/685600090 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f20537fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 0x7f20540c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53450/0 (socket says 172.21.15.27:53450) 2023-08-11T15:51:29.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f2052ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20540c9030 0x7f20540c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40670/0 (socket says 172.21.15.27:40670) 2023-08-11T15:51:29.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.808+0000 7f205a206640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2054150fa0 0x7f205414ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec1423d0 con 0x7f52ec152a70 2023-08-11T15:51:29.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f2053fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2054150fa0 0x7f205414ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f2053fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2054150fa0 0x7f205414ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46278/0 (socket says 172.21.15.27:46278) 2023-08-11T15:51:29.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f20537fe640 1 -- 172.21.15.27:0/3856088315 learned_addr learned my addr 172.21.15.27:0/3856088315 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f205a206640 1 -- 172.21.15.27:0/3856088315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20540c2b50 con 0x7f2054150fa0 2023-08-11T15:51:29.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f205a206640 1 -- 172.21.15.27:0/3856088315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20541514a0 con 0x7f20540c9030 2023-08-11T15:51:29.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f205a206640 1 -- 172.21.15.27:0/3856088315 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2054151780 con 0x7f2054070520 2023-08-11T15:51:29.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.810+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec15bae0 con 0x7f52ec06d6c0 2023-08-11T15:51:29.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.810+0000 7f52f179a640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec15c4e0 0x7f52ec0c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7fb937fff640 1 -- 172.21.15.27:0/990259152 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb94005aae0 con 0x7fb94416a8f0 2023-08-11T15:51:29.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.809+0000 7f85c776a640 1 -- 172.21.15.27:0/840552255 wait complete. 2023-08-11T15:51:29.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.810+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec141d80 con 0x7f52ec15c4e0 2023-08-11T15:51:29.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.810+0000 7fb94b7fe640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb93006a120 0x7fb93006c5e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb944008570 tx=0x7fb94c0ad000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.810+0000 7fb951f86640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] conn(0x7fb944050230 0x7fb9441895d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.810+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 --> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb944007170 con 0x7fb944050230 2023-08-11T15:51:29.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f2052ffd640 1 -- 172.21.15.27:0/3856088315 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 msgr2=0x7f20540c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f2052ffd640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2054070520 0x7f20540c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f2052ffd640 1 -- 172.21.15.27:0/3856088315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2054150fa0 msgr2=0x7f205414ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f2052ffd640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2054150fa0 0x7f205414ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f2052ffd640 1 -- 172.21.15.27:0/3856088315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2044009400 con 0x7f20540c9030 2023-08-11T15:51:29.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7fb94affd640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] conn(0x7fb944050230 0x7fb9441895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f2052ffd640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20540c9030 0x7f20540c6ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f204800d5f0 tx=0x7f204800dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f52ebfff640 1 -- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec15c4e0 msgr2=0x7f52ec0c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f52ebfff640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec15c4e0 0x7f52ec0c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f52ebfff640 1 -- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec06d6c0 msgr2=0x7f52ec15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f52ebfff640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec06d6c0 0x7f52ec15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f52ebfff640 1 -- 172.21.15.27:0/685600090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4009400 con 0x7f52ec152a70 2023-08-11T15:51:29.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f52f179a640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52ec15c4e0 0x7f52ec0c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:29.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.811+0000 7f52ebfff640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec152a70 0x7f52ec0ae900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52dc032000 tx=0x7f52dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.812+0000 7f85c776a640 1 Processor -- start 2023-08-11T15:51:29.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.812+0000 7f52e97fa640 1 -- 172.21.15.27:0/685600090 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc03e450 con 0x7f52ec152a70 2023-08-11T15:51:29.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.812+0000 7fb94affd640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] conn(0x7fb944050230 0x7fb9441895d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.812+0000 7f52e97fa640 1 -- 172.21.15.27:0/685600090 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc03e5b0 con 0x7f52ec152a70 2023-08-11T15:51:29.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.812+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc009b30 con 0x7f52ec152a70 2023-08-11T15:51:29.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.812+0000 7f52e97fa640 1 -- 172.21.15.27:0/685600090 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc03e940 con 0x7f52ec152a70 2023-08-11T15:51:29.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.812+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec142cf0 con 0x7f52ec152a70 2023-08-11T15:51:29.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.812+0000 7f85c776a640 1 -- start start 2023-08-11T15:51:29.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.813+0000 7ff70e3c4640 1 Processor -- start 2023-08-11T15:51:29.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f203bfff640 1 -- 172.21.15.27:0/3856088315 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048014620 con 0x7f20540c9030 2023-08-11T15:51:29.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f203bfff640 1 -- 172.21.15.27:0/3856088315 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2048014780 con 0x7f20540c9030 2023-08-11T15:51:29.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f203bfff640 1 -- 172.21.15.27:0/3856088315 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048014b10 con 0x7f20540c9030 2023-08-11T15:51:29.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7fb937fff640 1 -- 172.21.15.27:0/990259152 <== osd.1 v2:172.21.15.27:6808/889585576 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb938009950 con 0x7fb944050230 2023-08-11T15:51:29.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f85c776a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85c00ca8e0 0x7f85c00c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52cc000fc0 con 0x7f52ec152a70 2023-08-11T15:51:29.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f85c776a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85c00cd050 0x7f85c00cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f85c776a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c00ce530 0x7f85c00b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f85c776a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00ce9e0 con 0x7f85c00ce530 2023-08-11T15:51:29.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f85c776a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00b3790 con 0x7f85c00cd050 2023-08-11T15:51:29.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f85c776a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00c1be0 con 0x7f85c00ca8e0 2023-08-11T15:51:29.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7ff70e3c4640 1 -- start start 2023-08-11T15:51:29.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f205a206640 1 -- 172.21.15.27:0/3856088315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2054151a60 con 0x7f20540c9030 2023-08-11T15:51:29.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.814+0000 7f205a206640 1 -- 172.21.15.27:0/3856088315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20540c55e0 con 0x7f20540c9030 2023-08-11T15:51:29.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.815+0000 7f85c5506640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85c00ca8e0 0x7f85c00c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.815+0000 7f85c5d07640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c00ce530 0x7f85c00b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.815+0000 7f85c5d07640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c00ce530 0x7f85c00b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46296/0 (socket says 172.21.15.27:46296) 2023-08-11T15:51:29.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.815+0000 7f85c5d07640 1 -- 172.21.15.27:0/4090920940 learned_addr learned my addr 172.21.15.27:0/4090920940 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.815+0000 7f85c4d05640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85c00cd050 0x7f85c00cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.815+0000 7f205a206640 1 -- 172.21.15.27:0/3856088315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f20540ca040 con 0x7f20540c9030 2023-08-11T15:51:29.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.815+0000 7ff70e3c4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff70815c3d0 0x7ff7080d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7f85c5d07640 1 -- 172.21.15.27:0/4090920940 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85c00ca8e0 msgr2=0x7f85c00c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7f85c5d07640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85c00ca8e0 0x7f85c00c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7f85c5d07640 1 -- 172.21.15.27:0/4090920940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85c00cd050 msgr2=0x7f85c00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7f85c5d07640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85c00cd050 0x7f85c00cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7f85c5d07640 1 -- 172.21.15.27:0/4090920940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc008550 con 0x7f85c00ce530 2023-08-11T15:51:29.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7ff70e3c4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7080d0430 con 0x7ff7080d1cf0 2023-08-11T15:51:29.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7ff70e3c4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7080d0710 con 0x7ff7080cf240 2023-08-11T15:51:29.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7ff70e3c4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7080d09f0 con 0x7ff70815c3d0 2023-08-11T15:51:29.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7ff70cbc1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff7080d1cf0 0x7ff7080ceb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48412/0 (socket says 172.21.15.27:48412) 2023-08-11T15:51:29.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7ff70cbc1640 1 -- 172.21.15.27:0/3000610110 learned_addr learned my addr 172.21.15.27:0/3000610110 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:29.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7ff70d3c2640 1 --2- 172.21.15.27:0/3000610110 >> v2:172.21.15.119:3300/0 conn(0x7ff70815c3d0 0x7ff7080d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.816+0000 7f52e97fa640 1 -- 172.21.15.27:0/685600090 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f52dc03eaa0 con 0x7f52ec152a70 2023-08-11T15:51:29.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.817+0000 7ff70d3c2640 1 -- 172.21.15.27:0/3000610110 >> v1:172.21.15.119:6789/0 conn(0x7ff7080cf240 legacy=0x7ff7080b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.817+0000 7ff70d3c2640 1 -- 172.21.15.27:0/3000610110 >> v1:172.21.15.27:6789/0 conn(0x7ff7080d1cf0 legacy=0x7ff7080ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.817+0000 7ff70d3c2640 1 -- 172.21.15.27:0/3000610110 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7080d0cd0 con 0x7ff70815c3d0 2023-08-11T15:51:29.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.818+0000 7f85c5d07640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c00ce530 0x7f85c00b2a30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009fd0 tx=0x7f85b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.818+0000 7f52e97fa640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f52d406a200 0x7f52d406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.818+0000 7f52e97fa640 1 -- 172.21.15.27:0/685600090 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f52dc0ab390 con 0x7f52ec152a70 2023-08-11T15:51:29.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f52e97fa640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] conn(0x7f52d4073ee0 0x7f52d4076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f52e97fa640 1 -- 172.21.15.27:0/685600090 --> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52d40769f0 con 0x7f52d4073ee0 2023-08-11T15:51:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f52e97fa640 1 -- 172.21.15.27:0/685600090 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52dc0ad970 con 0x7f52ec152a70 2023-08-11T15:51:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f94e055d640 1 Processor -- start 2023-08-11T15:51:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7ff70d3c2640 1 --2- 172.21.15.27:0/3000610110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff70815c3d0 0x7ff7080d17b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80097a0 tx=0x7ff6f802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa3f95e4e044295 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f85b5ffb640 1 -- 172.21.15.27:0/4090920940 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8014420 con 0x7f85c00ce530 2023-08-11T15:51:29.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f85b5ffb640 1 -- 172.21.15.27:0/4090920940 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b8014580 con 0x7f85c00ce530 2023-08-11T15:51:29.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f85c776a640 1 -- 172.21.15.27:0/4090920940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c00c1ec0 con 0x7f85c00ce530 2023-08-11T15:51:29.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.820+0000 7f85c776a640 1 -- 172.21.15.27:0/4090920940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c00b3a70 con 0x7f85c00ce530 2023-08-11T15:51:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f203bfff640 1 -- 172.21.15.27:0/3856088315 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2048014c70 con 0x7f20540c9030 2023-08-11T15:51:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f203bfff640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f204006a120 0x7f204006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f203bfff640 1 -- 172.21.15.27:0/3856088315 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2048088f50 con 0x7f20540c9030 2023-08-11T15:51:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f52f179a640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] conn(0x7f52d4073ee0 0x7f52d4076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.820+0000 7f52f0f99640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f52d406a200 0x7f52d406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7ff6fffff640 1 -- 172.21.15.27:0/3000610110 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8030610 con 0x7ff70815c3d0 2023-08-11T15:51:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.820+0000 7ff6fffff640 1 -- 172.21.15.27:0/3000610110 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f8030770 con 0x7ff70815c3d0 2023-08-11T15:51:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.820+0000 7ff6fffff640 1 -- 172.21.15.27:0/3000610110 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8030b00 con 0x7ff70815c3d0 2023-08-11T15:51:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7ff70e3c4640 1 -- 172.21.15.27:0/3000610110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff70815c3d0 msgr2=0x7ff7080d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7ff70e3c4640 1 --2- 172.21.15.27:0/3000610110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff70815c3d0 0x7ff7080d17b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80097a0 tx=0x7ff6f802b960 comp rx=0 tx=0).stop 2023-08-11T15:51:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f94e055d640 1 -- start start 2023-08-11T15:51:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f6193153640 1 Processor -- start 2023-08-11T15:51:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f203bfff640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] conn(0x7f2040073e00 0x7f2040076240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f203bfff640 1 -- 172.21.15.27:0/3856088315 --> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2040076910 con 0x7f2040073e00 2023-08-11T15:51:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f203bfff640 1 -- 172.21.15.27:0/3856088315 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2048089300 con 0x7f20540c9030 2023-08-11T15:51:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f20537fe640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f204006a120 0x7f204006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f85b5ffb640 1 -- 172.21.15.27:0/4090920940 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8014930 con 0x7f85c00ce530 2023-08-11T15:51:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f2053fff640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] conn(0x7f2040073e00 0x7f2040076240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f94e055d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d815aec0 0x7f94d80ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f94e055d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d8161e90 con 0x7f94d8161800 2023-08-11T15:51:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f94e055d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d81653d0 con 0x7f94d80cadf0 2023-08-11T15:51:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f94e055d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8161c70 con 0x7f94d815aec0 2023-08-11T15:51:29.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.822+0000 7f52f179a640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] conn(0x7f52d4073ee0 0x7f52d4076320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.819+0000 7f6193153640 1 -- start start 2023-08-11T15:51:29.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f6193153640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c9e90 0x7f618c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f6193153640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f618c06f940 0x7f618c162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f6193153640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c06c460 con 0x7f618c0c8520 2023-08-11T15:51:29.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f6193153640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c158980 con 0x7f618c06f940 2023-08-11T15:51:29.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.821+0000 7f6193153640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0cc510 con 0x7f618c0c9e90 2023-08-11T15:51:29.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.822+0000 7f52f0f99640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f52d406a200 0x7f52d406c6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52e00017a0 tx=0x7f52e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.822+0000 7f6192151640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f618c0c8520 0x7f618c06f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48428/0 (socket says 172.21.15.27:48428) 2023-08-11T15:51:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.822+0000 7f6192151640 1 -- 172.21.15.27:0/1398497963 learned_addr learned my addr 172.21.15.27:0/1398497963 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.822+0000 7ff70e3c4640 1 -- 172.21.15.27:0/3000610110 shutdown_connections 2023-08-11T15:51:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.823+0000 7ff70e3c4640 1 --2- 172.21.15.27:0/3000610110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff70815c3d0 0x7ff7080d17b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.823+0000 7ff70e3c4640 1 -- 172.21.15.27:0/3000610110 >> 172.21.15.27:0/3000610110 conn(0x7ff70806a270 msgr2=0x7ff70806a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.822+0000 7f94deafa640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f94d8161800 0x7f94d80c8660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48426/0 (socket says 172.21.15.27:48426) 2023-08-11T15:51:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.823+0000 7f94deafa640 1 -- 172.21.15.27:0/3334447325 learned_addr learned my addr 172.21.15.27:0/3334447325 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.823+0000 7f85c776a640 1 -- 172.21.15.27:0/4090920940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85c00cc850 con 0x7f85c00ce530 2023-08-11T15:51:29.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.823+0000 7f2053fff640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] conn(0x7f2040073e00 0x7f2040076240 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.823+0000 7f20537fe640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f204006a120 0x7f204006c5e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2044034040 tx=0x7f204403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.823+0000 7f94de2f9640 1 --2- 172.21.15.27:0/3334447325 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d815aec0 0x7f94d80ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.823+0000 7f6192952640 1 --2- 172.21.15.27:0/1398497963 >> v2:172.21.15.119:3300/0 conn(0x7f618c06f940 0x7f618c162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.823+0000 7f85b5ffb640 1 -- 172.21.15.27:0/4090920940 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f85b8014a90 con 0x7f85c00ce530 2023-08-11T15:51:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 416960579 0 0) 0x7f94d8161e90 con 0x7f94d8161800 2023-08-11T15:51:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94c4003520 con 0x7f94d8161800 2023-08-11T15:51:29.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f619114f640 1 -- 172.21.15.27:0/1398497963 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426712963 0 0) 0x7f618c06c460 con 0x7f618c0c8520 2023-08-11T15:51:29.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f619114f640 1 -- 172.21.15.27:0/1398497963 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6178005730 con 0x7f618c0c8520 2023-08-11T15:51:29.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f6192952640 1 -- 172.21.15.27:0/1398497963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c9e90 msgr2=0x7f618c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:29.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f619114f640 1 -- 172.21.15.27:0/1398497963 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1977662930 0 0) 0x7f6178005730 con 0x7f618c0c8520 2023-08-11T15:51:29.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 587271898 0 0) 0x7f94c4003520 con 0x7f94d8161800 2023-08-11T15:51:29.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d815aec0 msgr2=0x7f94d80ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:29.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7ff70e3c4640 1 -- 172.21.15.27:0/3000610110 shutdown_connections 2023-08-11T15:51:29.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7ff70e3c4640 1 -- 172.21.15.27:0/3000610110 wait complete. 2023-08-11T15:51:29.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f94dd2f7640 1 --2- 172.21.15.27:0/3334447325 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d815aec0 0x7f94d80ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 >> v1:172.21.15.119:6789/0 conn(0x7f94d80cadf0 legacy=0x7f94d81610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8164610 con 0x7f94d8161800 2023-08-11T15:51:29.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.824+0000 7f85b5ffb640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85ac06a230 0x7f85ac06c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f52e97fa640 1 -- 172.21.15.27:0/685600090 <== osd.4 v2:172.21.15.119:6808/2943646764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f52d40769f0 con 0x7f52d4073ee0 2023-08-11T15:51:29.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f94d00019f0 con 0x7f94d8161800 2023-08-11T15:51:29.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f85b5ffb640 1 -- 172.21.15.27:0/4090920940 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f85b80890f0 con 0x7f85c00ce530 2023-08-11T15:51:29.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f85b5ffb640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] conn(0x7f85ac073f10 0x7f85ac076350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f85b5ffb640 1 -- 172.21.15.27:0/4090920940 --> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85ac076a20 con 0x7f85ac073f10 2023-08-11T15:51:29.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f85b5ffb640 1 -- 172.21.15.27:0/4090920940 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85b808b6d0 con 0x7f85c00ce530 2023-08-11T15:51:29.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f85c5506640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85ac06a230 0x7f85ac06c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 >> v1:172.21.15.27:6789/0 conn(0x7f94d8161800 legacy=0x7f94d80c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f94dd2f7640 1 --2- 172.21.15.27:0/3334447325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c40039a0 0x7f94c4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f94dd2f7640 1 --2- 172.21.15.27:0/3334447325 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d815aec0 0x7f94c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f94dd2f7640 1 --2- 172.21.15.27:0/3334447325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 0x7f94c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8161c70 con 0x7f94c4004d50 2023-08-11T15:51:29.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c400bb60 con 0x7f94c40039a0 2023-08-11T15:51:29.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c400bd20 con 0x7f94d815aec0 2023-08-11T15:51:29.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.826+0000 7f94deafa640 1 --2- 172.21.15.27:0/3334447325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 0x7f94c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.826+0000 7f94deafa640 1 -- 172.21.15.27:0/3334447325 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d815aec0 msgr2=0x7f94c40047c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:51:29.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.826+0000 7f94deafa640 1 --2- 172.21.15.27:0/3334447325 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d815aec0 0x7f94c40047c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.826+0000 7f94deafa640 1 -- 172.21.15.27:0/3334447325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c40039a0 msgr2=0x7f94c4003d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:51:29.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.826+0000 7f94deafa640 1 --2- 172.21.15.27:0/3334447325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c40039a0 0x7f94c4003d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.826+0000 7f94deafa640 1 -- 172.21.15.27:0/3334447325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c4003520 con 0x7f94c4004d50 2023-08-11T15:51:29.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.828+0000 7f94deafa640 1 --2- 172.21.15.27:0/3334447325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 0x7f94c40051d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94d0005930 tx=0x7f94c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5840f998428290b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.826+0000 7f6191950640 1 --2- 172.21.15.27:0/1398497963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c9e90 0x7f618c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.826+0000 7f6192952640 1 --2- 172.21.15.27:0/1398497963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c9e90 0x7f618c06c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.826+0000 7f6192952640 1 -- 172.21.15.27:0/1398497963 >> v1:172.21.15.27:6789/0 conn(0x7f618c0c8520 legacy=0x7f618c06f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.826+0000 7f6192952640 1 -- 172.21.15.27:0/1398497963 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0cc7f0 con 0x7f618c06f940 2023-08-11T15:51:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.828+0000 7f6191950640 1 --2- 172.21.15.27:0/1398497963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c9e90 0x7f618c06c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.828+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d002e6c0 con 0x7f94c4004d50 2023-08-11T15:51:29.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.828+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d002e840 con 0x7f94c4004d50 2023-08-11T15:51:29.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.828+0000 7f94dd2f7640 1 -- 172.21.15.27:0/3334447325 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d002ec30 con 0x7f94c4004d50 2023-08-11T15:51:29.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.828+0000 7f6192952640 1 --2- 172.21.15.27:0/1398497963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f618c06f940 0x7f618c162090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f618400aea0 tx=0x7f618402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa8fac54d2e0830 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.828+0000 7f619114f640 1 -- 172.21.15.27:0/1398497963 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188004e60 con 0x7f618c06f940 2023-08-11T15:51:29.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7f619114f640 1 -- 172.21.15.27:0/1398497963 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618400a2e0 con 0x7f618c06f940 2023-08-11T15:51:29.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7f619114f640 1 -- 172.21.15.27:0/1398497963 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618402c930 con 0x7f618c06f940 2023-08-11T15:51:29.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7f94e055d640 1 -- 172.21.15.27:0/3334447325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 msgr2=0x7f94c40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7f94e055d640 1 --2- 172.21.15.27:0/3334447325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 0x7f94c40051d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94d0005930 tx=0x7f94c40057f0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7f85c4d05640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] conn(0x7f85ac073f10 0x7f85ac076350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.825+0000 7f203bfff640 1 -- 172.21.15.27:0/3856088315 <== osd.3 v2:172.21.15.27:6816/4099583167 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2040076910 con 0x7f2040073e00 2023-08-11T15:51:29.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7f94deafa640 1 -- 172.21.15.27:0/3334447325 reap_dead start 2023-08-11T15:51:29.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7f94e055d640 1 -- 172.21.15.27:0/3334447325 shutdown_connections 2023-08-11T15:51:29.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7f94e055d640 1 -- 172.21.15.27:0/3334447325 >> 172.21.15.27:0/3334447325 conn(0x7f94d8069e90 msgr2=0x7f94d80afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.830+0000 7f94e055d640 1 -- 172.21.15.27:0/3334447325 shutdown_connections 2023-08-11T15:51:29.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.830+0000 7f6193153640 1 -- 172.21.15.27:0/1398497963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f618c06f940 msgr2=0x7f618c162090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.830+0000 7f6193153640 1 --2- 172.21.15.27:0/1398497963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f618c06f940 0x7f618c162090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f618400aea0 tx=0x7f618402be10 comp rx=0 tx=0).stop 2023-08-11T15:51:29.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.827+0000 7ff70e3c4640 1 Processor -- start 2023-08-11T15:51:29.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.827+0000 7ff70e3c4640 1 -- start start 2023-08-11T15:51:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7ff70e3c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7080cf240 0x7ff7080c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7ff70d3c2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7080cf240 0x7ff7080c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7ff70d3c2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7080cf240 0x7ff7080c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46322/0 (socket says 172.21.15.27:46322) 2023-08-11T15:51:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7ff70e3c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7080d1cf0 0x7ff7081550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7ff70e3c4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff70815c3d0 0x7ff70814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7ff70e3c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7080c74a0 con 0x7ff7080cf240 2023-08-11T15:51:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7ff70e3c4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff708089c70 con 0x7ff7080d1cf0 2023-08-11T15:51:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.829+0000 7ff70e3c4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7080c6c50 con 0x7ff70815c3d0 2023-08-11T15:51:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.830+0000 7ff70d3c2640 1 -- 172.21.15.27:0/3466348418 learned_addr learned my addr 172.21.15.27:0/3466348418 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.830+0000 7ff70cbc1640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7080d1cf0 0x7ff7081550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.830+0000 7ff70dbc3640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff70815c3d0 0x7ff70814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.831+0000 7f85c4d05640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] conn(0x7f85ac073f10 0x7f85ac076350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.831+0000 7ff70d3c2640 1 -- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff70815c3d0 msgr2=0x7ff70814c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.831+0000 7ff70d3c2640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff70815c3d0 0x7ff70814c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.831+0000 7ff70d3c2640 1 -- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7080d1cf0 msgr2=0x7ff7081550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.831+0000 7ff70d3c2640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7080d1cf0 0x7ff7081550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.831+0000 7ff70d3c2640 1 -- 172.21.15.27:0/3466348418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f8009400 con 0x7ff7080cf240 2023-08-11T15:51:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.831+0000 7ff70d3c2640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7080cf240 0x7ff7080c6660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff6f802bf70 tx=0x7ff6f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.831+0000 7f85c5506640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85ac06a230 0x7f85ac06c6f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85c00b0be0 tx=0x7f85b0007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.832+0000 7f6193153640 1 -- 172.21.15.27:0/1398497963 shutdown_connections 2023-08-11T15:51:29.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.832+0000 7f6193153640 1 --2- 172.21.15.27:0/1398497963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f618c06f940 0x7f618c162090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.832+0000 7f6193153640 1 --2- 172.21.15.27:0/1398497963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c9e90 0x7f618c06c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.832+0000 7f6193153640 1 -- 172.21.15.27:0/1398497963 >> 172.21.15.27:0/1398497963 conn(0x7f618c069800 msgr2=0x7f618c069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.832+0000 7f6193153640 1 -- 172.21.15.27:0/1398497963 shutdown_connections 2023-08-11T15:51:29.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.832+0000 7f6193153640 1 -- 172.21.15.27:0/1398497963 wait complete. 2023-08-11T15:51:29.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.830+0000 7f94e055d640 1 -- 172.21.15.27:0/3334447325 wait complete. 2023-08-11T15:51:29.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.832+0000 7f94e055d640 1 Processor -- start 2023-08-11T15:51:29.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.832+0000 7f94e055d640 1 -- start start 2023-08-11T15:51:29.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94e055d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 0x7f94d81fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94e055d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d80b25e0 0x7f94d80b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94e055d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80b10a0 0x7f94d815f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94e055d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80b21c0 con 0x7f94c4004d50 2023-08-11T15:51:29.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94e055d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d81fe6f0 con 0x7f94d80b10a0 2023-08-11T15:51:29.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94e055d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80b1540 con 0x7f94d80b25e0 2023-08-11T15:51:29.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94de2f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 0x7f94d81fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94de2f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 0x7f94d81fe1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46326/0 (socket says 172.21.15.27:46326) 2023-08-11T15:51:29.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94de2f9640 1 -- 172.21.15.27:0/1799121063 learned_addr learned my addr 172.21.15.27:0/1799121063 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f6193153640 1 Processor -- start 2023-08-11T15:51:29.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94deafa640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80b10a0 0x7f94d815f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.834+0000 7f94ddaf8640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d80b25e0 0x7f94d80b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.835+0000 7f6193153640 1 -- start start 2023-08-11T15:51:29.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.835+0000 7f94de2f9640 1 -- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d80b25e0 msgr2=0x7f94d80b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.835+0000 7f94de2f9640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94d80b25e0 0x7f94d80b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.835+0000 7f94de2f9640 1 -- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80b10a0 msgr2=0x7f94d815f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.835+0000 7f94de2f9640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80b10a0 0x7f94d815f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.835+0000 7f94de2f9640 1 -- 172.21.15.27:0/1799121063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d80b1820 con 0x7f94c4004d50 2023-08-11T15:51:29.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.836+0000 7f85b5ffb640 1 -- 172.21.15.27:0/4090920940 <== osd.2 v2:172.21.15.27:6821/2059564233 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f85ac076a20 con 0x7f85ac073f10 2023-08-11T15:51:29.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.835+0000 7f94de2f9640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 0x7f94d81fe1b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94d400c4c0 tx=0x7f94d400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.835+0000 7ff6fdffb640 1 -- 172.21.15.27:0/3466348418 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f803e070 con 0x7ff7080cf240 2023-08-11T15:51:29.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.836+0000 7ff70e3c4640 1 -- 172.21.15.27:0/3466348418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7080c6f30 con 0x7ff7080cf240 2023-08-11T15:51:29.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.836+0000 7f94ceffd640 1 -- 172.21.15.27:0/1799121063 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4004220 con 0x7f94c4004d50 2023-08-11T15:51:29.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.836+0000 7f94ceffd640 1 -- 172.21.15.27:0/1799121063 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d4004380 con 0x7f94c4004d50 2023-08-11T15:51:29.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.836+0000 7ff70e3c4640 1 -- 172.21.15.27:0/3466348418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7080c79d0 con 0x7ff7080cf240 2023-08-11T15:51:29.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.836+0000 7f94ceffd640 1 -- 172.21.15.27:0/1799121063 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4003830 con 0x7f94c4004d50 2023-08-11T15:51:29.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.836+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d80b1b00 con 0x7f94c4004d50 2023-08-11T15:51:29.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.836+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d80c3390 con 0x7f94c4004d50 2023-08-11T15:51:29.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.837+0000 7ff6fdffb640 1 -- 172.21.15.27:0/3466348418 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f8039a50 con 0x7ff7080cf240 2023-08-11T15:51:29.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.837+0000 7ff6fdffb640 1 -- 172.21.15.27:0/3466348418 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8039de0 con 0x7ff7080cf240 2023-08-11T15:51:29.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.837+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94d8164bd0 con 0x7f94c4004d50 2023-08-11T15:51:29.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.838+0000 7f6193153640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f618c06f940 0x7f618c1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.838+0000 7f6193153640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c8520 0x7f618c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.839+0000 7f6193153640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f618c0c9e90 0x7f618c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.839+0000 7f6193153640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c1e53b0 con 0x7f618c0c9e90 2023-08-11T15:51:29.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.839+0000 7f6193153640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c15b810 con 0x7f618c06f940 2023-08-11T15:51:29.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.839+0000 7f6193153640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c159030 con 0x7f618c0c8520 2023-08-11T15:51:29.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.839+0000 7f6191950640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c8520 0x7f618c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.839+0000 7f6191950640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c8520 0x7f618c15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53498/0 (socket says 172.21.15.27:53498) 2023-08-11T15:51:29.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.839+0000 7f6191950640 1 -- 172.21.15.27:0/881546369 learned_addr learned my addr 172.21.15.27:0/881546369 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:29.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.840+0000 7f94ceffd640 1 -- 172.21.15.27:0/1799121063 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f94d4003990 con 0x7f94c4004d50 2023-08-11T15:51:29.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.840+0000 7f6192151640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f618c06f940 0x7f618c1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.840+0000 7f6191950640 1 -- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f618c06f940 msgr2=0x7f618c1576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.840+0000 7f6191950640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f618c06f940 0x7f618c1576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.841+0000 7f6191950640 1 -- 172.21.15.27:0/881546369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f618c0c9e90 msgr2=0x7f618c15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.841+0000 7f6191950640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f618c0c9e90 0x7f618c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.841+0000 7f6191950640 1 -- 172.21.15.27:0/881546369 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618400ab80 con 0x7f618c0c8520 2023-08-11T15:51:29.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.841+0000 7f6192151640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f618c06f940 0x7f618c1576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:29.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7f6191950640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c8520 0x7f618c15ad90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f617c00c820 tx=0x7f617c00cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7f94ceffd640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f94c406b9d0 0x7f94c406de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7f94ceffd640 1 -- 172.21.15.27:0/1799121063 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f94d40889b0 con 0x7f94c4004d50 2023-08-11T15:51:29.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7f6182ffd640 1 -- 172.21.15.27:0/881546369 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c007a30 con 0x7f618c0c8520 2023-08-11T15:51:29.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7f6182ffd640 1 -- 172.21.15.27:0/881546369 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c007b90 con 0x7f618c0c8520 2023-08-11T15:51:29.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7f6182ffd640 1 -- 172.21.15.27:0/881546369 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c00ce70 con 0x7f618c0c8520 2023-08-11T15:51:29.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.840+0000 7ff6fdffb640 1 -- 172.21.15.27:0/3466348418 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff6f804b400 con 0x7ff7080cf240 2023-08-11T15:51:29.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7ff6fdffb640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff6ec06a150 0x7ff6ec06c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7ff6fdffb640 1 -- 172.21.15.27:0/3466348418 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff6f80aac80 con 0x7ff7080cf240 2023-08-11T15:51:29.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7f94ceffd640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] conn(0x7f94c40756b0 0x7f94c4077af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7f94ceffd640 1 -- 172.21.15.27:0/1799121063 --> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94c40781c0 con 0x7f94c40756b0 2023-08-11T15:51:29.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.842+0000 7f94ceffd640 1 -- 172.21.15.27:0/1799121063 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94d4088d20 con 0x7f94c4004d50 2023-08-11T15:51:29.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.843+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c159310 con 0x7f618c0c8520 2023-08-11T15:51:29.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.843+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c1598d0 con 0x7f618c0c8520 2023-08-11T15:51:29.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.843+0000 7ff70cbc1640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff6ec06a150 0x7ff6ec06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.843+0000 7f94ddaf8640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f94c406b9d0 0x7f94c406de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.843+0000 7f94deafa640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] conn(0x7f94c40756b0 0x7f94c4077af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.844+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f618c06c0f0 con 0x7f618c0c8520 2023-08-11T15:51:29.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.846+0000 7ff70cbc1640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff6ec06a150 0x7ff6ec06c610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff70815ba30 tx=0x7ff704009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.846+0000 7f94ddaf8640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f94c406b9d0 0x7f94c406de90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f94d80b3a60 tx=0x7f94c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.846+0000 7f94deafa640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] conn(0x7f94c40756b0 0x7f94c4077af0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.849+0000 7f6182ffd640 1 -- 172.21.15.27:0/881546369 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f617c0113f0 con 0x7f618c0c8520 2023-08-11T15:51:29.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.850+0000 7f94ceffd640 1 -- 172.21.15.27:0/1799121063 <== osd.6 v2:172.21.15.119:6800/2672123554 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f94c40781c0 con 0x7f94c40756b0 2023-08-11T15:51:29.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.850+0000 7ff70e3c4640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] conn(0x7ff6e80015e0 0x7ff6e8003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.850+0000 7ff70e3c4640 1 -- 172.21.15.27:0/3466348418 --> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6e8006b40 con 0x7ff6e80015e0 2023-08-11T15:51:29.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.850+0000 7f6182ffd640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f617006a050 0x7f617006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.850+0000 7f6182ffd640 1 -- 172.21.15.27:0/881546369 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f617c088f90 con 0x7f618c0c8520 2023-08-11T15:51:29.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.851+0000 7f6182ffd640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] conn(0x7f6170073cb0 0x7f61700760f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:29.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.851+0000 7f6182ffd640 1 -- 172.21.15.27:0/881546369 --> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61700767c0 con 0x7f6170073cb0 2023-08-11T15:51:29.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.851+0000 7f6182ffd640 1 -- 172.21.15.27:0/881546369 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f617c0893e0 con 0x7f618c0c8520 2023-08-11T15:51:29.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.851+0000 7ff70dbc3640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] conn(0x7ff6e80015e0 0x7ff6e8003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.852+0000 7f6192151640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f617006a050 0x7f617006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.852+0000 7f6192952640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] conn(0x7f6170073cb0 0x7f61700760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:29.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.853+0000 7f6192151640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f617006a050 0x7f617006c510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f61880049d0 tx=0x7f6188004920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.854+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 --> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4898002cb0 con 0x7f4894073dd0 2023-08-11T15:51:29.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.854+0000 7ff70dbc3640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] conn(0x7ff6e80015e0 0x7ff6e8003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.854+0000 7f6192952640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] conn(0x7f6170073cb0 0x7f61700760f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.855+0000 7f48a2ffd640 1 -- 172.21.15.27:0/453298059 <== osd.0 v2:172.21.15.27:6800/3363308208 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4898002cb0 con 0x7f4894073dd0 2023-08-11T15:51:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.856+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 >> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] conn(0x7f4894073dd0 msgr2=0x7f4894076210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.856+0000 7f48b8a3f640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] conn(0x7f4894073dd0 0x7f4894076210 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.857+0000 7f6182ffd640 1 -- 172.21.15.27:0/881546369 <== osd.7 v2:172.21.15.119:6824/1341885104 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f61700767c0 con 0x7f6170073cb0 2023-08-11T15:51:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.857+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f489406a0f0 msgr2=0x7f489406c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.857+0000 7ff6fdffb640 1 -- 172.21.15.27:0/3466348418 <== osd.5 v2:172.21.15.119:6816/1811178752 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff6e8006b40 con 0x7ff6e80015e0 2023-08-11T15:51:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.858+0000 7f48b8a3f640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f489406a0f0 0x7f489406c5b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48a802b7d0 tx=0x7f48a8008030 comp rx=0 tx=0).stop 2023-08-11T15:51:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.858+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48b415c290 msgr2=0x7f48b41591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.858+0000 7f48b8a3f640 1 --2- 172.21.15.27:0/453298059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48b415c290 0x7f48b41591c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f48ac000f80 tx=0x7f48ac00cbe0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.858+0000 7f48b2d9d640 1 -- 172.21.15.27:0/453298059 reap_dead start 2023-08-11T15:51:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.859+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 shutdown_connections 2023-08-11T15:51:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.859+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 >> 172.21.15.27:0/453298059 conn(0x7f48b40cdfd0 msgr2=0x7f48b415f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.860+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 shutdown_connections 2023-08-11T15:51:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.860+0000 7f48b8a3f640 1 -- 172.21.15.27:0/453298059 wait complete. 2023-08-11T15:51:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.861+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 --> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52cc002cb0 con 0x7f52d4073ee0 2023-08-11T15:51:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.861+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 --> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9441892f0 con 0x7fb944050230 2023-08-11T15:51:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.862+0000 7fb937fff640 1 -- 172.21.15.27:0/990259152 <== osd.1 v2:172.21.15.27:6808/889585576 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb9441892f0 con 0x7fb944050230 2023-08-11T15:51:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.862+0000 7f52e97fa640 1 -- 172.21.15.27:0/685600090 <== osd.4 v2:172.21.15.119:6808/2943646764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f52cc002cb0 con 0x7f52d4073ee0 2023-08-11T15:51:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.864+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] conn(0x7f52d4073ee0 msgr2=0x7f52d4076320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.864+0000 7f52f31fd640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] conn(0x7f52d4073ee0 0x7f52d4076320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.864+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f52d406a200 msgr2=0x7f52d406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.864+0000 7f52f31fd640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f52d406a200 0x7f52d406c6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52e00017a0 tx=0x7f52e0009040 comp rx=0 tx=0).stop 2023-08-11T15:51:29.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.864+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec152a70 msgr2=0x7f52ec0ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.864+0000 7f52f31fd640 1 --2- 172.21.15.27:0/685600090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec152a70 0x7f52ec0ae900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52dc032000 tx=0x7f52dc004a40 comp rx=0 tx=0).stop 2023-08-11T15:51:29.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.864+0000 7f52f179a640 1 -- 172.21.15.27:0/685600090 reap_dead start 2023-08-11T15:51:29.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.865+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 >> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] conn(0x7fb944050230 msgr2=0x7fb9441895d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.865+0000 7fb951f86640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] conn(0x7fb944050230 0x7fb9441895d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.865+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 shutdown_connections 2023-08-11T15:51:29.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.865+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 >> 172.21.15.27:0/685600090 conn(0x7f52ec0afb00 msgr2=0x7f52ec147180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.865+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb93006a120 msgr2=0x7fb93006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.865+0000 7fb951f86640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb93006a120 0x7fb93006c5e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb944008570 tx=0x7fb94c0ad000 comp rx=0 tx=0).stop 2023-08-11T15:51:29.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.865+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb94416a8f0 msgr2=0x7fb9441676e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.865+0000 7fb951f86640 1 --2- 172.21.15.27:0/990259152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb94416a8f0 0x7fb9441676e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb94000b3f0 tx=0x7fb94000b8b0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.865+0000 7fb94bfff640 1 -- 172.21.15.27:0/990259152 reap_dead start 2023-08-11T15:51:29.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.866+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 shutdown_connections 2023-08-11T15:51:29.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.866+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 shutdown_connections 2023-08-11T15:51:29.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.866+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 >> 172.21.15.27:0/990259152 conn(0x7fb944009fe0 msgr2=0x7fb944033620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.866+0000 7f52f31fd640 1 -- 172.21.15.27:0/685600090 wait complete. 2023-08-11T15:51:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.866+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 shutdown_connections 2023-08-11T15:51:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.866+0000 7fb951f86640 1 -- 172.21.15.27:0/990259152 wait complete. 2023-08-11T15:51:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.867+0000 7f205a206640 1 -- 172.21.15.27:0/3856088315 --> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f20540c9d60 con 0x7f2040073e00 2023-08-11T15:51:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.868+0000 7f203bfff640 1 -- 172.21.15.27:0/3856088315 <== osd.3 v2:172.21.15.27:6816/4099583167 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f204c009d60 con 0x7f2040073e00 2023-08-11T15:51:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.869+0000 7f2039ffb640 1 -- 172.21.15.27:0/3856088315 >> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] conn(0x7f2040073e00 msgr2=0x7f2040076240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.869+0000 7f2039ffb640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] conn(0x7f2040073e00 0x7f2040076240 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.870+0000 7f2039ffb640 1 -- 172.21.15.27:0/3856088315 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f204006a120 msgr2=0x7f204006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.870+0000 7f2039ffb640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f204006a120 0x7f204006c5e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2044034040 tx=0x7f204403c040 comp rx=0 tx=0).stop 2023-08-11T15:51:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.870+0000 7f2039ffb640 1 -- 172.21.15.27:0/3856088315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20540c9030 msgr2=0x7f20540c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.871+0000 7f2039ffb640 1 --2- 172.21.15.27:0/3856088315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20540c9030 0x7f20540c6ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f204800d5f0 tx=0x7f204800dab0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.871+0000 7f2053fff640 1 -- 172.21.15.27:0/3856088315 reap_dead start 2023-08-11T15:51:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.871+0000 7f2039ffb640 1 -- 172.21.15.27:0/3856088315 shutdown_connections 2023-08-11T15:51:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.871+0000 7f2039ffb640 1 -- 172.21.15.27:0/3856088315 >> 172.21.15.27:0/3856088315 conn(0x7f20540cdfd0 msgr2=0x7f2054158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.872+0000 7f2039ffb640 1 -- 172.21.15.27:0/3856088315 shutdown_connections 2023-08-11T15:51:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.872+0000 7f2039ffb640 1 -- 172.21.15.27:0/3856088315 wait complete. 2023-08-11T15:51:29.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.877+0000 7f85c776a640 1 -- 172.21.15.27:0/4090920940 --> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f85c0073990 con 0x7f85ac073f10 2023-08-11T15:51:29.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.878+0000 7f85b5ffb640 1 -- 172.21.15.27:0/4090920940 <== osd.2 v2:172.21.15.27:6821/2059564233 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f85c0073990 con 0x7f85ac073f10 2023-08-11T15:51:29.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.879+0000 7f85ab7fe640 1 -- 172.21.15.27:0/4090920940 >> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] conn(0x7f85ac073f10 msgr2=0x7f85ac076350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.879+0000 7f85ab7fe640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] conn(0x7f85ac073f10 0x7f85ac076350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.879+0000 7f85ab7fe640 1 -- 172.21.15.27:0/4090920940 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85ac06a230 msgr2=0x7f85ac06c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.879+0000 7f85ab7fe640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85ac06a230 0x7f85ac06c6f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85c00b0be0 tx=0x7f85b0007e80 comp rx=0 tx=0).stop 2023-08-11T15:51:29.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.879+0000 7f85ab7fe640 1 -- 172.21.15.27:0/4090920940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c00ce530 msgr2=0x7f85c00b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.880+0000 7f85ab7fe640 1 --2- 172.21.15.27:0/4090920940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c00ce530 0x7f85c00b2a30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009fd0 tx=0x7f85b800e4b0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.880+0000 7f85c5d07640 1 -- 172.21.15.27:0/4090920940 reap_dead start 2023-08-11T15:51:29.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.880+0000 7f85ab7fe640 1 -- 172.21.15.27:0/4090920940 shutdown_connections 2023-08-11T15:51:29.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.880+0000 7f85ab7fe640 1 -- 172.21.15.27:0/4090920940 >> 172.21.15.27:0/4090920940 conn(0x7f85c0069e90 msgr2=0x7f85c0152f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.880+0000 7f85ab7fe640 1 -- 172.21.15.27:0/4090920940 shutdown_connections 2023-08-11T15:51:29.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.881+0000 7f85ab7fe640 1 -- 172.21.15.27:0/4090920940 wait complete. 2023-08-11T15:51:29.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.882+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 --> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94d81648f0 con 0x7f94c40756b0 2023-08-11T15:51:29.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.884+0000 7f94ceffd640 1 -- 172.21.15.27:0/1799121063 <== osd.6 v2:172.21.15.119:6800/2672123554 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f94d0001e70 con 0x7f94c40756b0 2023-08-11T15:51:29.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.887+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 --> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f618c06be10 con 0x7f6170073cb0 2023-08-11T15:51:29.928 INFO:teuthology.orchestra.run.smithi027.stdout:60129542157 2023-08-11T15:51:29.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-11T15:51:29.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.887+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] conn(0x7f94c40756b0 msgr2=0x7f94c4077af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.887+0000 7f94e055d640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] conn(0x7f94c40756b0 0x7f94c4077af0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f94c406b9d0 msgr2=0x7f94c406de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7f94e055d640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f94c406b9d0 0x7f94c406de90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f94d80b3a60 tx=0x7f94c8009040 comp rx=0 tx=0).stop 2023-08-11T15:51:29.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7ff70e3c4640 1 -- 172.21.15.27:0/3466348418 --> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff6e8005c20 con 0x7ff6e80015e0 2023-08-11T15:51:29.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 msgr2=0x7f94d81fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7f94e055d640 1 --2- 172.21.15.27:0/1799121063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c4004d50 0x7f94d81fe1b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94d400c4c0 tx=0x7f94d400c980 comp rx=0 tx=0).stop 2023-08-11T15:51:29.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7f94deafa640 1 -- 172.21.15.27:0/1799121063 reap_dead start 2023-08-11T15:51:29.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7f6182ffd640 1 -- 172.21.15.27:0/881546369 <== osd.7 v2:172.21.15.119:6824/1341885104 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f618400a560 con 0x7f6170073cb0 2023-08-11T15:51:29.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 shutdown_connections 2023-08-11T15:51:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 >> 172.21.15.27:0/1799121063 conn(0x7f94d8069e90 msgr2=0x7f94d80af280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 shutdown_connections 2023-08-11T15:51:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.888+0000 7f94e055d640 1 -- 172.21.15.27:0/1799121063 wait complete. 2023-08-11T15:51:29.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] conn(0x7f6170073cb0 msgr2=0x7f61700760f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6193153640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] conn(0x7f6170073cb0 0x7f61700760f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f617006a050 msgr2=0x7f617006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6193153640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f617006a050 0x7f617006c510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f61880049d0 tx=0x7f6188004920 comp rx=0 tx=0).stop 2023-08-11T15:51:29.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c8520 msgr2=0x7f618c15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6193153640 1 --2- 172.21.15.27:0/881546369 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f618c0c8520 0x7f618c15ad90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f617c00c820 tx=0x7f617c00cc20 comp rx=0 tx=0).stop 2023-08-11T15:51:29.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6192952640 1 -- 172.21.15.27:0/881546369 reap_dead start 2023-08-11T15:51:29.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 shutdown_connections 2023-08-11T15:51:29.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 >> 172.21.15.27:0/881546369 conn(0x7f618c069800 msgr2=0x7f618c06f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 shutdown_connections 2023-08-11T15:51:29.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.889+0000 7f6193153640 1 -- 172.21.15.27:0/881546369 wait complete. 2023-08-11T15:51:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.890+0000 7ff6fdffb640 1 -- 172.21.15.27:0/3466348418 <== osd.5 v2:172.21.15.119:6816/1811178752 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff6e8005c20 con 0x7ff6e80015e0 2023-08-11T15:51:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.891+0000 7ff6f37fe640 1 -- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] conn(0x7ff6e80015e0 msgr2=0x7ff6e8003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.891+0000 7ff6f37fe640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] conn(0x7ff6e80015e0 0x7ff6e8003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.891+0000 7ff6f37fe640 1 -- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff6ec06a150 msgr2=0x7ff6ec06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.891+0000 7ff6f37fe640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff6ec06a150 0x7ff6ec06c610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff70815ba30 tx=0x7ff704009040 comp rx=0 tx=0).stop 2023-08-11T15:51:29.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.891+0000 7ff6f37fe640 1 -- 172.21.15.27:0/3466348418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7080cf240 msgr2=0x7ff7080c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:29.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.891+0000 7ff6f37fe640 1 --2- 172.21.15.27:0/3466348418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7080cf240 0x7ff7080c6660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff6f802bf70 tx=0x7ff6f80046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:29.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.891+0000 7ff70dbc3640 1 -- 172.21.15.27:0/3466348418 reap_dead start 2023-08-11T15:51:29.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.891+0000 7ff6f37fe640 1 -- 172.21.15.27:0/3466348418 shutdown_connections 2023-08-11T15:51:29.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.892+0000 7ff6f37fe640 1 -- 172.21.15.27:0/3466348418 >> 172.21.15.27:0/3466348418 conn(0x7ff70806a270 msgr2=0x7ff7080b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:29.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.892+0000 7ff6f37fe640 1 -- 172.21.15.27:0/3466348418 shutdown_connections 2023-08-11T15:51:29.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:29.893+0000 7ff6f37fe640 1 -- 172.21.15.27:0/3466348418 wait complete. 2023-08-11T15:51:29.938 INFO:teuthology.orchestra.run.smithi027.stdout:60129542155 2023-08-11T15:51:29.938 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-11T15:51:29.939 INFO:teuthology.orchestra.run.smithi027.stdout:60129542155 2023-08-11T15:51:29.939 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-11T15:51:29.940 INFO:teuthology.orchestra.run.smithi027.stdout:60129542156 2023-08-11T15:51:29.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-11T15:51:29.941 INFO:teuthology.orchestra.run.smithi027.stdout:60129542155 2023-08-11T15:51:29.941 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-11T15:51:29.942 INFO:teuthology.orchestra.run.smithi027.stdout:60129542156 2023-08-11T15:51:29.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-11T15:51:29.943 INFO:teuthology.orchestra.run.smithi027.stdout:60129542156 2023-08-11T15:51:29.944 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-11T15:51:29.945 INFO:teuthology.orchestra.run.smithi027.stdout:60129542157 2023-08-11T15:51:29.945 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-11T15:51:30.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.298+0000 7f625e887640 1 Processor -- start 2023-08-11T15:51:30.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.299+0000 7f625e887640 1 -- start start 2023-08-11T15:51:30.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.301+0000 7f625e887640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 0x7f6258159460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.301+0000 7f625e887640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f62580c9030 0x7f62580703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.301+0000 7f625e887640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625806b9c0 con 0x7f62580ceac0 2023-08-11T15:51:30.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.301+0000 7f625e887640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6258070900 con 0x7f62580c9030 2023-08-11T15:51:30.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.301+0000 7f625e887640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62580c97a0 con 0x7f625806ffe0 2023-08-11T15:51:30.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.301+0000 7f625ce24640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f62580ceac0 0x7f625815c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48444/0 (socket says 172.21.15.27:48444) 2023-08-11T15:51:30.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.301+0000 7f625ce24640 1 -- 172.21.15.27:0/1101356638 learned_addr learned my addr 172.21.15.27:0/1101356638 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:30.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.302+0000 7f6257fff640 1 --2- 172.21.15.27:0/1101356638 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 0x7f6258159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.302+0000 7f62577fe640 1 --2- 172.21.15.27:0/1101356638 >> v2:172.21.15.119:3300/0 conn(0x7f62580c9030 0x7f62580703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.302+0000 7f6256ffd640 1 -- 172.21.15.27:0/1101356638 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772910172 0 0) 0x7f625806b9c0 con 0x7f62580ceac0 2023-08-11T15:51:30.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.302+0000 7f6256ffd640 1 -- 172.21.15.27:0/1101356638 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6240003520 con 0x7f62580ceac0 2023-08-11T15:51:30.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.302+0000 7f6257fff640 1 -- 172.21.15.27:0/1101356638 >> v2:172.21.15.119:3300/0 conn(0x7f62580c9030 msgr2=0x7f62580703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:30.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.302+0000 7f6257fff640 1 --2- 172.21.15.27:0/1101356638 >> v2:172.21.15.119:3300/0 conn(0x7f62580c9030 0x7f62580703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.302+0000 7f6257fff640 1 -- 172.21.15.27:0/1101356638 >> v1:172.21.15.27:6789/0 conn(0x7f62580ceac0 legacy=0x7f625815c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.302+0000 7f6257fff640 1 -- 172.21.15.27:0/1101356638 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62580c9a80 con 0x7f625806ffe0 2023-08-11T15:51:30.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.304+0000 7f6257fff640 1 --2- 172.21.15.27:0/1101356638 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 0x7f6258159460 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f624c0097a0 tx=0x7f624c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8179f7dcd719cb97 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.305+0000 7f6256ffd640 1 -- 172.21.15.27:0/1101356638 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c030800 con 0x7f625806ffe0 2023-08-11T15:51:30.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.305+0000 7f6256ffd640 1 -- 172.21.15.27:0/1101356638 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c030960 con 0x7f625806ffe0 2023-08-11T15:51:30.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.305+0000 7f6256ffd640 1 -- 172.21.15.27:0/1101356638 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c030cf0 con 0x7f625806ffe0 2023-08-11T15:51:30.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.306+0000 7f625e887640 1 -- 172.21.15.27:0/1101356638 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 msgr2=0x7f6258159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.306+0000 7f625e887640 1 --2- 172.21.15.27:0/1101356638 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 0x7f6258159460 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f624c0097a0 tx=0x7f624c02b940 comp rx=0 tx=0).stop 2023-08-11T15:51:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.309+0000 7f625e887640 1 -- 172.21.15.27:0/1101356638 shutdown_connections 2023-08-11T15:51:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.309+0000 7f625e887640 1 --2- 172.21.15.27:0/1101356638 >> v2:172.21.15.119:3300/0 conn(0x7f62580c9030 0x7f62580703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.309+0000 7f625e887640 1 --2- 172.21.15.27:0/1101356638 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 0x7f6258159460 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.309+0000 7f625e887640 1 -- 172.21.15.27:0/1101356638 >> 172.21.15.27:0/1101356638 conn(0x7f62580cdfd0 msgr2=0x7f62580ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.309+0000 7f625e887640 1 -- 172.21.15.27:0/1101356638 shutdown_connections 2023-08-11T15:51:30.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.309+0000 7f625e887640 1 -- 172.21.15.27:0/1101356638 wait complete. 2023-08-11T15:51:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.311+0000 7f625e887640 1 Processor -- start 2023-08-11T15:51:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.312+0000 7fe4e966a640 1 Processor -- start 2023-08-11T15:51:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.313+0000 7fe4e966a640 1 -- start start 2023-08-11T15:51:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.312+0000 7fee4965d640 1 Processor -- start 2023-08-11T15:51:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.313+0000 7fee4965d640 1 -- start start 2023-08-11T15:51:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.312+0000 7f625e887640 1 -- start start 2023-08-11T15:51:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.313+0000 7f625e887640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 0x7f62580b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.314+0000 7f4ec668e640 1 Processor -- start 2023-08-11T15:51:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.314+0000 7f625e887640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62580c9030 0x7f62580b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.314+0000 7f625e887640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62580ceac0 0x7f6258159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fe4e966a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4e40c14c0 0x7fe4e40ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fe4e966a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4e406c150 0x7fe4e406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fe4e966a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e406bcc0 con 0x7fe4e40ccb70 2023-08-11T15:51:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fe4e966a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e40c91e0 con 0x7fe4e40c14c0 2023-08-11T15:51:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fe4e966a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40cb070 con 0x7fe4e406c150 2023-08-11T15:51:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fee4965d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee440c9e90 0x7fee440c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fee4965d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee4406c8f0 0x7fee4406f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fee4965d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4406cd80 con 0x7fee4406a2f0 2023-08-11T15:51:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fee4965d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4406c460 con 0x7fee440c9e90 2023-08-11T15:51:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fee4965d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee440cc7f0 con 0x7fee4406c8f0 2023-08-11T15:51:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7f6257fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 0x7f62580b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7f6257fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 0x7f62580b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53520/0 (socket says 172.21.15.27:53520) 2023-08-11T15:51:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7f62577fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62580c9030 0x7f62580b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7f62577fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62580c9030 0x7f62580b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40734/0 (socket says 172.21.15.27:40734) 2023-08-11T15:51:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7f625ce24640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62580ceac0 0x7f6258159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7f625ce24640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62580ceac0 0x7f6258159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46336/0 (socket says 172.21.15.27:46336) 2023-08-11T15:51:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7fe4e27fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe4e40ccb70 0x7fe4e406f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48456/0 (socket says 172.21.15.27:48456) 2023-08-11T15:51:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.316+0000 7fe4e27fc640 1 -- 172.21.15.27:0/325330764 learned_addr learned my addr 172.21.15.27:0/325330764 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.316+0000 7fe4e37fe640 1 --2- 172.21.15.27:0/325330764 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4e406c150 0x7fe4e406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.316+0000 7fee43fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fee4406a2f0 0x7fee4406f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:43904/0 (socket says 172.21.15.27:43904) 2023-08-11T15:51:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.316+0000 7fee43fff640 1 -- 172.21.15.27:0/2161856754 learned_addr learned my addr 172.21.15.27:0/2161856754 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.315+0000 7f4ec668e640 1 -- start start 2023-08-11T15:51:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.316+0000 7fee437fe640 1 --2- 172.21.15.27:0/2161856754 >> v2:172.21.15.27:3300/0 conn(0x7fee440c9e90 0x7fee440c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.316+0000 7fe4e2ffd640 1 --2- 172.21.15.27:0/325330764 >> v2:172.21.15.27:3300/0 conn(0x7fe4e40c14c0 0x7fe4e40ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7fee42ffd640 1 -- 172.21.15.27:0/2161856754 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040814293 0 0) 0x7fee4406cd80 con 0x7fee4406a2f0 2023-08-11T15:51:30.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7fee42ffd640 1 -- 172.21.15.27:0/2161856754 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee2c003520 con 0x7fee4406a2f0 2023-08-11T15:51:30.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7fee48e5c640 1 --2- 172.21.15.27:0/2161856754 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee4406c8f0 0x7fee4406f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7f4ec668e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb80e75b0 0x7f4eb803bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7f4ec668e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4eb8006e70 0x7f4eb80072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7f4ec668e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb8006140 con 0x7f4eb8007810 2023-08-11T15:51:30.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7f4ec668e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb80e7990 con 0x7f4eb8006e70 2023-08-11T15:51:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7f4ec668e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80069e0 con 0x7f4eb80e75b0 2023-08-11T15:51:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7fe4e37fe640 1 -- 172.21.15.27:0/325330764 >> v2:172.21.15.27:3300/0 conn(0x7fe4e40c14c0 msgr2=0x7fe4e40ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7fe4e37fe640 1 --2- 172.21.15.27:0/325330764 >> v2:172.21.15.27:3300/0 conn(0x7fe4e40c14c0 0x7fe4e40ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7fe4e37fe640 1 -- 172.21.15.27:0/325330764 >> v1:172.21.15.27:6789/0 conn(0x7fe4e40ccb70 legacy=0x7fe4e406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.317+0000 7fe4e37fe640 1 -- 172.21.15.27:0/325330764 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e40cb350 con 0x7fe4e406c150 2023-08-11T15:51:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7f4ebf7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4eb8006e70 0x7f4eb80072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7f4ebf7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4eb8006e70 0x7f4eb80072d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40746/0 (socket says 172.21.15.27:40746) 2023-08-11T15:51:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7f4ebf7fe640 1 -- 172.21.15.27:0/1275330580 learned_addr learned my addr 172.21.15.27:0/1275330580 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7f4ebffff640 1 --2- 172.21.15.27:0/1275330580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb80e75b0 0x7f4eb803bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7f6257fff640 1 -- 172.21.15.27:0/1662145139 learned_addr learned my addr 172.21.15.27:0/1662145139 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7f625e887640 1 -- 172.21.15.27:0/1662145139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62580b3070 con 0x7f62580ceac0 2023-08-11T15:51:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7f625e887640 1 -- 172.21.15.27:0/1662145139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62580b3db0 con 0x7f62580c9030 2023-08-11T15:51:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7faa7914e640 1 Processor -- start 2023-08-11T15:51:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7f625e887640 1 -- 172.21.15.27:0/1662145139 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62580c2e50 con 0x7f625806ffe0 2023-08-11T15:51:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7faa7914e640 1 -- start start 2023-08-11T15:51:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7f4ebffff640 1 -- 172.21.15.27:0/1275330580 >> v2:172.21.15.119:3300/0 conn(0x7f4eb8006e70 msgr2=0x7f4eb80072d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.319+0000 7f4ebffff640 1 --2- 172.21.15.27:0/1275330580 >> v2:172.21.15.119:3300/0 conn(0x7f4eb8006e70 0x7f4eb80072d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.319+0000 7f625ce24640 1 -- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 msgr2=0x7f62580b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.319+0000 7f625ce24640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 0x7f62580b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.319+0000 7f625ce24640 1 -- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62580c9030 msgr2=0x7f62580b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.319+0000 7f625ce24640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62580c9030 0x7f62580b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.319+0000 7f625ce24640 1 -- 172.21.15.27:0/1662145139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c009400 con 0x7f62580ceac0 2023-08-11T15:51:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.319+0000 7fe4e37fe640 1 --2- 172.21.15.27:0/325330764 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4e406c150 0x7fe4e406c5e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40097a0 tx=0x7fe4d402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0c903895729dc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7fee48e5c640 1 -- 172.21.15.27:0/2161856754 >> v2:172.21.15.27:3300/0 conn(0x7fee440c9e90 msgr2=0x7fee440c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7fee48e5c640 1 --2- 172.21.15.27:0/2161856754 >> v2:172.21.15.27:3300/0 conn(0x7fee440c9e90 0x7fee440c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7fee48e5c640 1 -- 172.21.15.27:0/2161856754 >> v1:172.21.15.119:6789/0 conn(0x7fee4406a2f0 legacy=0x7fee4406f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.318+0000 7fee48e5c640 1 -- 172.21.15.27:0/2161856754 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4406be10 con 0x7fee4406c8f0 2023-08-11T15:51:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.319+0000 7f4ebffff640 1 -- 172.21.15.27:0/1275330580 >> v1:172.21.15.119:6789/0 conn(0x7f4eb8007810 legacy=0x7f4eb80eb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.319+0000 7f4ebffff640 1 -- 172.21.15.27:0/1275330580 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb803c2c0 con 0x7f4eb80e75b0 2023-08-11T15:51:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.319+0000 7f4ebf7fe640 1 --2- 172.21.15.27:0/1275330580 >> v2:172.21.15.119:3300/0 conn(0x7f4eb8006e70 0x7f4eb80072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7f625ce24640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62580ceac0 0x7f6258159a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f62580ca480 tx=0x7f625000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7fe4e1ffb640 1 -- 172.21.15.27:0/325330764 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4031780 con 0x7fe4e406c150 2023-08-11T15:51:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7fe4e1ffb640 1 -- 172.21.15.27:0/325330764 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d40318e0 con 0x7fe4e406c150 2023-08-11T15:51:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7fe4e1ffb640 1 -- 172.21.15.27:0/325330764 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4031c70 con 0x7fe4e406c150 2023-08-11T15:51:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7fee48e5c640 1 --2- 172.21.15.27:0/2161856754 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee4406c8f0 0x7fee4406f940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fee340097a0 tx=0x7fee3402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d38b6e4a7fc16a84 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7f4ebffff640 1 --2- 172.21.15.27:0/1275330580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb80e75b0 0x7f4eb803bd80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ec002c740 tx=0x7f4ec00677b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=422115988b9e9203 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7fee42ffd640 1 -- 172.21.15.27:0/2161856754 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34030610 con 0x7fee4406c8f0 2023-08-11T15:51:30.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7fee42ffd640 1 -- 172.21.15.27:0/2161856754 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee34030770 con 0x7fee4406c8f0 2023-08-11T15:51:30.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7fee42ffd640 1 -- 172.21.15.27:0/2161856754 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34030b40 con 0x7fee4406c8f0 2023-08-11T15:51:30.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7faa7914e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7fe4e966a640 1 -- 172.21.15.27:0/325330764 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4e406c150 msgr2=0x7fe4e406c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7fe4e966a640 1 --2- 172.21.15.27:0/325330764 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4e406c150 0x7fe4e406c5e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40097a0 tx=0x7fe4d402b820 comp rx=0 tx=0).stop 2023-08-11T15:51:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7faa7914e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa740ceac0 0x7faa740cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7faa7914e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa740cd390 con 0x7faa74160cd0 2023-08-11T15:51:30.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7f6254ff9640 1 -- 172.21.15.27:0/1662145139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625000ab70 con 0x7f62580ceac0 2023-08-11T15:51:30.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7f6254ff9640 1 -- 172.21.15.27:0/1662145139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625000acd0 con 0x7f62580ceac0 2023-08-11T15:51:30.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7f6254ff9640 1 -- 172.21.15.27:0/1662145139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250012620 con 0x7f62580ceac0 2023-08-11T15:51:30.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7f625e887640 1 -- 172.21.15.27:0/1662145139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62580c33d0 con 0x7f62580ceac0 2023-08-11T15:51:30.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7f625e887640 1 -- 172.21.15.27:0/1662145139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62580c3990 con 0x7f62580ceac0 2023-08-11T15:51:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7faa7914e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa740d0b20 con 0x7faa740ceac0 2023-08-11T15:51:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.320+0000 7faa7914e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa741610b0 con 0x7faa740cc9f0 2023-08-11T15:51:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7faa7259c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7faa7259c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53548/0 (socket says 172.21.15.27:53548) 2023-08-11T15:51:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7faa7259c640 1 -- 172.21.15.27:0/2372672461 learned_addr learned my addr 172.21.15.27:0/2372672461 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7f4ebeffd640 1 -- 172.21.15.27:0/1275330580 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00a5de0 con 0x7f4eb80e75b0 2023-08-11T15:51:30.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7f4ebeffd640 1 -- 172.21.15.27:0/1275330580 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec00ad440 con 0x7f4eb80e75b0 2023-08-11T15:51:30.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7fee4965d640 1 -- 172.21.15.27:0/2161856754 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee4406c8f0 msgr2=0x7fee4406f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7fee4965d640 1 --2- 172.21.15.27:0/2161856754 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee4406c8f0 0x7fee4406f940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fee340097a0 tx=0x7fee3402b960 comp rx=0 tx=0).stop 2023-08-11T15:51:30.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7fe4e966a640 1 -- 172.21.15.27:0/325330764 shutdown_connections 2023-08-11T15:51:30.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7fe4e966a640 1 --2- 172.21.15.27:0/325330764 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4e406c150 0x7fe4e406c5e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7fe4e966a640 1 --2- 172.21.15.27:0/325330764 >> v2:172.21.15.27:3300/0 conn(0x7fe4e40c14c0 0x7fe4e40ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7f4ec668e640 1 -- 172.21.15.27:0/1275330580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb80e75b0 msgr2=0x7f4eb803bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7f4ec668e640 1 --2- 172.21.15.27:0/1275330580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb80e75b0 0x7f4eb803bd80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ec002c740 tx=0x7f4ec00677b0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.321+0000 7fe4e966a640 1 -- 172.21.15.27:0/325330764 >> 172.21.15.27:0/325330764 conn(0x7fe4e40afb00 msgr2=0x7fe4e40afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7fee4965d640 1 -- 172.21.15.27:0/2161856754 shutdown_connections 2023-08-11T15:51:30.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7fee4965d640 1 --2- 172.21.15.27:0/2161856754 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee4406c8f0 0x7fee4406f940 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7fee4965d640 1 --2- 172.21.15.27:0/2161856754 >> v2:172.21.15.27:3300/0 conn(0x7fee440c9e90 0x7fee440c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7fee4965d640 1 -- 172.21.15.27:0/2161856754 >> 172.21.15.27:0/2161856754 conn(0x7fee44069800 msgr2=0x7fee44069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7f4ec668e640 1 -- 172.21.15.27:0/1275330580 shutdown_connections 2023-08-11T15:51:30.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7f4ec668e640 1 --2- 172.21.15.27:0/1275330580 >> v2:172.21.15.119:3300/0 conn(0x7f4eb8006e70 0x7f4eb80072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7f4ec668e640 1 --2- 172.21.15.27:0/1275330580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb80e75b0 0x7f4eb803bd80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7f4ec668e640 1 -- 172.21.15.27:0/1275330580 >> 172.21.15.27:0/1275330580 conn(0x7f4eb80559b0 msgr2=0x7f4eb8055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.323+0000 7fee4965d640 1 -- 172.21.15.27:0/2161856754 shutdown_connections 2023-08-11T15:51:30.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.323+0000 7f7f1ba8e640 1 Processor -- start 2023-08-11T15:51:30.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.323+0000 7f7f1ba8e640 1 -- start start 2023-08-11T15:51:30.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.323+0000 7f625e887640 1 -- 172.21.15.27:0/1662145139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6238005250 con 0x7f62580ceac0 2023-08-11T15:51:30.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.324+0000 7fe4e966a640 1 -- 172.21.15.27:0/325330764 shutdown_connections 2023-08-11T15:51:30.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7faa7359e640 1 --2- 172.21.15.27:0/2372672461 >> v2:172.21.15.119:3300/0 conn(0x7faa740ceac0 0x7faa740cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7faa7259c640 1 -- 172.21.15.27:0/2372672461 >> v2:172.21.15.119:3300/0 conn(0x7faa740ceac0 msgr2=0x7faa740cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:30.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7faa7259c640 1 --2- 172.21.15.27:0/2372672461 >> v2:172.21.15.119:3300/0 conn(0x7faa740ceac0 0x7faa740cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7faa7259c640 1 -- 172.21.15.27:0/2372672461 >> v1:172.21.15.119:6789/0 conn(0x7faa74160cd0 legacy=0x7faa740d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7faa7259c640 1 -- 172.21.15.27:0/2372672461 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa740cf490 con 0x7faa740cc9f0 2023-08-11T15:51:30.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.322+0000 7faa7359e640 1 --2- 172.21.15.27:0/2372672461 >> v2:172.21.15.119:3300/0 conn(0x7faa740ceac0 0x7faa740cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:30.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.324+0000 7fe4e966a640 1 -- 172.21.15.27:0/325330764 wait complete. 2023-08-11T15:51:30.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.324+0000 7faa7259c640 1 --2- 172.21.15.27:0/2372672461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740cce50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faa68009870 tx=0x7faa6802b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59b52bc6a3c6df24 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.325+0000 7f4ec668e640 1 -- 172.21.15.27:0/1275330580 shutdown_connections 2023-08-11T15:51:30.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.323+0000 7fee4965d640 1 -- 172.21.15.27:0/2161856754 wait complete. 2023-08-11T15:51:30.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.325+0000 7fee4965d640 1 Processor -- start 2023-08-11T15:51:30.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.325+0000 7faa71d9b640 1 -- 172.21.15.27:0/2372672461 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68030860 con 0x7faa740cc9f0 2023-08-11T15:51:30.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.325+0000 7faa71d9b640 1 -- 172.21.15.27:0/2372672461 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa680309c0 con 0x7faa740cc9f0 2023-08-11T15:51:30.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.325+0000 7faa71d9b640 1 -- 172.21.15.27:0/2372672461 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68030d90 con 0x7faa740cc9f0 2023-08-11T15:51:30.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.326+0000 7faa7914e640 1 -- 172.21.15.27:0/2372672461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 msgr2=0x7faa740cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.326+0000 7faa7914e640 1 --2- 172.21.15.27:0/2372672461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740cce50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faa68009870 tx=0x7faa6802b9a0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.325+0000 7f7f1ba8e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7f0c007600 0x7f7f0c005ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.325+0000 7f7f1ba8e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0c007170 con 0x7f7f0c050f00 2023-08-11T15:51:30.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.325+0000 7f7f1ba8e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0c006560 con 0x7f7f0c0e11c0 2023-08-11T15:51:30.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.325+0000 7f7f1ba8e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0c006840 con 0x7f7f0c007600 2023-08-11T15:51:30.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.326+0000 7f7f1982a640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7f0c0e11c0 0x7f7f0c039e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:43932/0 (socket says 172.21.15.27:43932) 2023-08-11T15:51:30.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.326+0000 7f7f1982a640 1 -- 172.21.15.27:0/516081432 learned_addr learned my addr 172.21.15.27:0/516081432 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:30.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.326+0000 7f7f19029640 1 --2- 172.21.15.27:0/516081432 >> v2:172.21.15.27:3300/0 conn(0x7f7f0c007600 0x7f7f0c005ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.326+0000 7fe4e966a640 1 Processor -- start 2023-08-11T15:51:30.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.325+0000 7f4ec668e640 1 -- 172.21.15.27:0/1275330580 wait complete. 2023-08-11T15:51:30.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.327+0000 7fee4965d640 1 -- start start 2023-08-11T15:51:30.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.328+0000 7f4ec668e640 1 Processor -- start 2023-08-11T15:51:30.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7f6254ff9640 1 -- 172.21.15.27:0/1662145139 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6250007070 con 0x7f62580ceac0 2023-08-11T15:51:30.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.327+0000 7fe4e966a640 1 -- start start 2023-08-11T15:51:30.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fe4e966a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4e406c150 0x7fe4e40c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fe4e966a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e40c14c0 0x7fe4e415c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fe4e966a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e40ccb70 0x7fe4e40c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fe4e966a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e415c860 con 0x7fe4e40c14c0 2023-08-11T15:51:30.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fe4e966a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40c7e20 con 0x7fe4e40ccb70 2023-08-11T15:51:30.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fe4e966a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40c6630 con 0x7fe4e406c150 2023-08-11T15:51:30.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fe4e27fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e40c14c0 0x7fe4e415c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fe4e27fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e40c14c0 0x7fe4e415c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46380/0 (socket says 172.21.15.27:46380) 2023-08-11T15:51:30.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fe4e27fc640 1 -- 172.21.15.27:0/2568654792 learned_addr learned my addr 172.21.15.27:0/2568654792 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fe4e27fc640 1 -- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4e406c150 msgr2=0x7fe4e40c70d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T15:51:30.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fe4e27fc640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4e406c150 0x7fe4e40c70d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fe4e27fc640 1 -- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e40ccb70 msgr2=0x7fe4e40c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fe4e27fc640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e40ccb70 0x7fe4e40c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fe4e27fc640 1 -- 172.21.15.27:0/2568654792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d4009400 con 0x7fe4e40c14c0 2023-08-11T15:51:30.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.331+0000 7fe4e27fc640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e40c14c0 0x7fe4e415c320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00cba0 tx=0x7fe4dc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fee4965d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee4406a2f0 0x7fee44146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fee4965d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4406c8f0 0x7fee44070000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fee4965d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee440c9e90 0x7fee44070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fee4965d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee440b12a0 con 0x7fee4406c8f0 2023-08-11T15:51:30.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fee4965d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee440b0c00 con 0x7fee440c9e90 2023-08-11T15:51:30.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fee4965d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee44146af0 con 0x7fee4406a2f0 2023-08-11T15:51:30.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fee437fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4406c8f0 0x7fee44070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fee437fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4406c8f0 0x7fee44070000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46374/0 (socket says 172.21.15.27:46374) 2023-08-11T15:51:30.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7fee437fe640 1 -- 172.21.15.27:0/3461844050 learned_addr learned my addr 172.21.15.27:0/3461844050 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fee48e5c640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee440c9e90 0x7fee44070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fee437fe640 1 -- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee4406a2f0 msgr2=0x7fee44146050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:51:30.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fee437fe640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee4406a2f0 0x7fee44146050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fee437fe640 1 -- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee440c9e90 msgr2=0x7fee44070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fee437fe640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee440c9e90 0x7fee44070540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fee437fe640 1 -- 172.21.15.27:0/3461844050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee34009400 con 0x7fee4406c8f0 2023-08-11T15:51:30.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7fee48e5c640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee440c9e90 0x7fee44070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:30.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.331+0000 7fee437fe640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4406c8f0 0x7fee44070000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fee3c007ef0 tx=0x7fee3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.328+0000 7f4ec668e640 1 -- start start 2023-08-11T15:51:30.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7f4ec668e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb8006e70 0x7f4eb80ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7f4ec668e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb8007810 0x7f4eb804eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.331+0000 7f4ebf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb8007810 0x7f4eb804eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.331+0000 7f4ebf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb8007810 0x7f4eb804eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40770/0 (socket says 172.21.15.27:40770) 2023-08-11T15:51:30.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.331+0000 7f4ebffff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb8006e70 0x7f4eb80ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7faa7914e640 1 -- 172.21.15.27:0/2372672461 shutdown_connections 2023-08-11T15:51:30.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7faa7914e640 1 --2- 172.21.15.27:0/2372672461 >> v2:172.21.15.119:3300/0 conn(0x7faa740ceac0 0x7faa740cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7faa7914e640 1 --2- 172.21.15.27:0/2372672461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740cce50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7faa7914e640 1 -- 172.21.15.27:0/2372672461 >> 172.21.15.27:0/2372672461 conn(0x7faa7406a4a0 msgr2=0x7faa7406a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7faa7914e640 1 -- 172.21.15.27:0/2372672461 shutdown_connections 2023-08-11T15:51:30.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.327+0000 7f7f19029640 1 -- 172.21.15.27:0/516081432 >> v1:172.21.15.119:6789/0 conn(0x7f7f0c0e11c0 legacy=0x7f7f0c039e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.327+0000 7f7f19029640 1 -- 172.21.15.27:0/516081432 >> v1:172.21.15.27:6789/0 conn(0x7f7f0c050f00 legacy=0x7f7f0c04bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:30.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.327+0000 7f7f19029640 1 -- 172.21.15.27:0/516081432 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c006b20 con 0x7f7f0c007600 2023-08-11T15:51:30.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.329+0000 7f7f19029640 1 --2- 172.21.15.27:0/516081432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c007600 0x7f7f0c005ec0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7f100097a0 tx=0x7f7f1002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7c3886c2daca1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.331+0000 7f6254ff9640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f624406a120 0x7f624406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.331+0000 7f6254ff9640 1 -- 172.21.15.27:0/1662145139 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6250088bc0 con 0x7f62580ceac0 2023-08-11T15:51:30.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.331+0000 7f6257fff640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f624406a120 0x7f624406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.331+0000 7f4ebffff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb8006e70 0x7f4eb80ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53566/0 (socket says 172.21.15.27:53566) 2023-08-11T15:51:30.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.330+0000 7faa7914e640 1 -- 172.21.15.27:0/2372672461 wait complete. 2023-08-11T15:51:30.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.331+0000 7f4ebf7fe640 1 -- 172.21.15.27:0/2207724834 learned_addr learned my addr 172.21.15.27:0/2207724834 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.332+0000 7f4ec668e640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb80ca060 0x7f4eb80e6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.332+0000 7f4ec4c2b640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb80ca060 0x7f4eb80e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.333+0000 7f6257fff640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f624406a120 0x7f624406c5e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f624c02b980 tx=0x7f624c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.334+0000 7fee40ff9640 1 -- 172.21.15.27:0/3461844050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c014620 con 0x7fee4406c8f0 2023-08-11T15:51:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.335+0000 7fee4965d640 1 -- 172.21.15.27:0/3461844050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee440b0ec0 con 0x7fee4406c8f0 2023-08-11T15:51:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.335+0000 7fee4965d640 1 -- 172.21.15.27:0/3461844050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee441586f0 con 0x7fee4406c8f0 2023-08-11T15:51:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.333+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80df120 con 0x7f4eb80ca060 2023-08-11T15:51:30.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.333+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80de830 con 0x7f4eb8007810 2023-08-11T15:51:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.333+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80cd000 con 0x7f4eb8006e70 2023-08-11T15:51:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.334+0000 7f4ec4c2b640 1 -- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb8006e70 msgr2=0x7f4eb80ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.334+0000 7f4ec4c2b640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb8006e70 0x7f4eb80ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.334+0000 7f4ec4c2b640 1 -- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb8007810 msgr2=0x7f4eb804eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.334+0000 7f4ec4c2b640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb8007810 0x7f4eb804eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.334+0000 7f4ec4c2b640 1 -- 172.21.15.27:0/2207724834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac0094d0 con 0x7f4eb80ca060 2023-08-11T15:51:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.335+0000 7f4ec4c2b640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb80ca060 0x7f4eb80e6050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8038940 tx=0x7f4eb400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.333+0000 7faa7914e640 1 Processor -- start 2023-08-11T15:51:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.334+0000 7faa7914e640 1 -- start start 2023-08-11T15:51:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.334+0000 7f7f18828640 1 -- 172.21.15.27:0/516081432 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10031640 con 0x7f7f0c007600 2023-08-11T15:51:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.335+0000 7f6254ff9640 1 -- 172.21.15.27:0/1662145139 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6250054050 con 0x7f62580ceac0 2023-08-11T15:51:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.335+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/2568654792 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc00ad40 con 0x7fe4e40c14c0 2023-08-11T15:51:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.335+0000 7faa7914e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7faa7914e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa740ceac0 0x7faa740b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7faa7259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa740ceac0 0x7faa740b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7faa72d9d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7fee40ff9640 1 -- 172.21.15.27:0/3461844050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c014780 con 0x7fee4406c8f0 2023-08-11T15:51:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7fee40ff9640 1 -- 172.21.15.27:0/3461844050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c014af0 con 0x7fee4406c8f0 2023-08-11T15:51:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7faa72d9d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53578/0 (socket says 172.21.15.27:53578) 2023-08-11T15:51:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7faa7259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa740ceac0 0x7faa740b7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46390/0 (socket says 172.21.15.27:46390) 2023-08-11T15:51:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7faa7914e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74160cd0 0x7faa740c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.337+0000 7f4ebcff9640 1 -- 172.21.15.27:0/2207724834 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4005220 con 0x7f4eb80ca060 2023-08-11T15:51:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.337+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec0073490 con 0x7f4eb80ca060 2023-08-11T15:51:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7faa72d9d640 1 -- 172.21.15.27:0/3916055576 learned_addr learned my addr 172.21.15.27:0/3916055576 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa740c5f50 con 0x7faa740ceac0 2023-08-11T15:51:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa740c7f00 con 0x7faa74160cd0 2023-08-11T15:51:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa740c61f0 con 0x7faa740cc9f0 2023-08-11T15:51:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.335+0000 7fe4e966a640 1 -- 172.21.15.27:0/2568654792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e40c5a60 con 0x7fe4e40c14c0 2023-08-11T15:51:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.336+0000 7fe4e966a640 1 -- 172.21.15.27:0/2568654792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e40c5d00 con 0x7fe4e40c14c0 2023-08-11T15:51:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.337+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb80dead0 con 0x7f4eb80ca060 2023-08-11T15:51:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.338+0000 7f4ebcff9640 1 -- 172.21.15.27:0/2207724834 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb4001d60 con 0x7f4eb80ca060 2023-08-11T15:51:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.338+0000 7f7f18828640 1 -- 172.21.15.27:0/516081432 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f100317a0 con 0x7f7f0c007600 2023-08-11T15:51:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.338+0000 7f7f18828640 1 -- 172.21.15.27:0/516081432 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10031b70 con 0x7f7f0c007600 2023-08-11T15:51:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.339+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/2568654792 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc01c440 con 0x7fe4e40c14c0 2023-08-11T15:51:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.339+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/2568654792 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc00aea0 con 0x7fe4e40c14c0 2023-08-11T15:51:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.339+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea0005250 con 0x7f4eb80ca060 2023-08-11T15:51:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.339+0000 7f4ebcff9640 1 -- 172.21.15.27:0/2207724834 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4007030 con 0x7f4eb80ca060 2023-08-11T15:51:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.337+0000 7faa7359e640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74160cd0 0x7faa740c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.338+0000 7faa7359e640 1 -- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 msgr2=0x7faa740c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.338+0000 7faa7359e640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.338+0000 7faa7359e640 1 -- 172.21.15.27:0/3916055576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa740ceac0 msgr2=0x7faa740b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.338+0000 7faa7359e640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa740ceac0 0x7faa740b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.338+0000 7faa7359e640 1 -- 172.21.15.27:0/3916055576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680094d0 con 0x7faa74160cd0 2023-08-11T15:51:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.339+0000 7fee40ff9640 1 -- 172.21.15.27:0/3461844050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fee3c014c50 con 0x7fee4406c8f0 2023-08-11T15:51:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.339+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/516081432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c007600 msgr2=0x7f7f0c005ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.339+0000 7f7f1ba8e640 1 --2- 172.21.15.27:0/516081432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c007600 0x7f7f0c005ec0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7f100097a0 tx=0x7f7f1002b8f0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.340+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/2568654792 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe4dc012400 con 0x7fe4e40c14c0 2023-08-11T15:51:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.340+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/516081432 shutdown_connections 2023-08-11T15:51:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.340+0000 7f7f1ba8e640 1 --2- 172.21.15.27:0/516081432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c007600 0x7f7f0c005ec0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.340+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/516081432 >> 172.21.15.27:0/516081432 conn(0x7f7f0c009fe0 msgr2=0x7f7f0c00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.341+0000 7fe4e966a640 1 -- 172.21.15.27:0/2568654792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c4005250 con 0x7fe4e40c14c0 2023-08-11T15:51:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.341+0000 7fee40ff9640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3006a050 0x7fee3006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.342+0000 7fe4cb7fe640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4d006a050 0x7fe4d006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.342+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/2568654792 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe4dc0887c0 con 0x7fe4e40c14c0 2023-08-11T15:51:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.342+0000 7fee43fff640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3006a050 0x7fee3006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.342+0000 7fee4965d640 1 -- 172.21.15.27:0/3461844050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee24005250 con 0x7fee4406c8f0 2023-08-11T15:51:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.342+0000 7f54053b2640 1 Processor -- start 2023-08-11T15:51:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.342+0000 7fe4e2ffd640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4d006a050 0x7fe4d006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.340+0000 7f95482a7640 1 Processor -- start 2023-08-11T15:51:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.341+0000 7f95482a7640 1 -- start start 2023-08-11T15:51:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.342+0000 7f95482a7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 0x7f95400cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7f95482a7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95400cd390 con 0x7f9540160cd0 2023-08-11T15:51:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7f95482a7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95401610b0 con 0x7f95400ceac0 2023-08-11T15:51:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7f95482a7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95400d0b20 con 0x7f95400cc9f0 2023-08-11T15:51:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7fee40ff9640 1 -- 172.21.15.27:0/3461844050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fee3c0589e0 con 0x7fee4406c8f0 2023-08-11T15:51:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7f54053b2640 1 -- start start 2023-08-11T15:51:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7faa7359e640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74160cd0 0x7faa740c72b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa6400c5c0 tx=0x7faa6400ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/516081432 shutdown_connections 2023-08-11T15:51:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7f9545842640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 0x7f95400cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7f9545842640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 0x7f95400cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53584/0 (socket says 172.21.15.27:53584) 2023-08-11T15:51:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7f9546844640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f95400ceac0 0x7f95400cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:43936/0 (socket says 172.21.15.27:43936) 2023-08-11T15:51:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7f9545842640 1 -- 172.21.15.27:0/779312831 learned_addr learned my addr 172.21.15.27:0/779312831 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.344+0000 7fee43fff640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3006a050 0x7fee3006c510 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee380014f0 tx=0x7fee380078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.344+0000 7fe4e2ffd640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4d006a050 0x7fe4d006c510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40c0660 tx=0x7fe4d8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.344+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/516081432 wait complete. 2023-08-11T15:51:30.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.343+0000 7f4ebcff9640 1 -- 172.21.15.27:0/2207724834 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4eb4007190 con 0x7f4eb80ca060 2023-08-11T15:51:30.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.344+0000 7f4ebcff9640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ea406a050 0x7f4ea406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.344+0000 7f4ebcff9640 1 -- 172.21.15.27:0/2207724834 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4eb4088130 con 0x7f4eb80ca060 2023-08-11T15:51:30.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.345+0000 7f4ebffff640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ea406a050 0x7f4ea406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.344+0000 7f9545842640 1 -- 172.21.15.27:0/779312831 >> v1:172.21.15.119:6789/0 conn(0x7f95400ceac0 legacy=0x7f95400cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.344+0000 7f9545842640 1 -- 172.21.15.27:0/779312831 >> v1:172.21.15.27:6789/0 conn(0x7f9540160cd0 legacy=0x7f95400d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.344+0000 7f9545842640 1 -- 172.21.15.27:0/779312831 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95400cf730 con 0x7f95400cc9f0 2023-08-11T15:51:30.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.345+0000 7faa5b7fe640 1 -- 172.21.15.27:0/3916055576 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64011800 con 0x7faa74160cd0 2023-08-11T15:51:30.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.345+0000 7faa5b7fe640 1 -- 172.21.15.27:0/3916055576 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa64011960 con 0x7faa74160cd0 2023-08-11T15:51:30.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.345+0000 7faa5b7fe640 1 -- 172.21.15.27:0/3916055576 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64011d50 con 0x7faa74160cd0 2023-08-11T15:51:30.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.345+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa740c64d0 con 0x7faa74160cd0 2023-08-11T15:51:30.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.345+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa740b7b50 con 0x7faa74160cd0 2023-08-11T15:51:30.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.346+0000 7f7f1ba8e640 1 Processor -- start 2023-08-11T15:51:30.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.346+0000 7f4ebffff640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ea406a050 0x7f4ea406c510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8006b20 tx=0x7f4ec00a9000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.346+0000 7f9545842640 1 --2- 172.21.15.27:0/779312831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 0x7f95400cce50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f953c0098a0 tx=0x7f953c02ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24257e2fa269709b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.346+0000 7f7f1ba8e640 1 -- start start 2023-08-11T15:51:30.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f9545041640 1 -- 172.21.15.27:0/779312831 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c02f870 con 0x7f95400cc9f0 2023-08-11T15:51:30.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f9545041640 1 -- 172.21.15.27:0/779312831 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c02f9d0 con 0x7f95400cc9f0 2023-08-11T15:51:30.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f9545041640 1 -- 172.21.15.27:0/779312831 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c02fd40 con 0x7f95400cc9f0 2023-08-11T15:51:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f54053b2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53f803b950 0x7f53f8006fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f53fffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f53f80e0000 0x7f53f8053fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:43938/0 (socket says 172.21.15.27:43938) 2023-08-11T15:51:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f54053b2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f53f8007520 0x7f53f8007990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f53fffff640 1 -- 172.21.15.27:0/1458961180 learned_addr learned my addr 172.21.15.27:0/1458961180 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f54053b2640 1 -- 172.21.15.27:0/1458961180 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f80546d0 con 0x7f53f80e0000 2023-08-11T15:51:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f53ff7fe640 1 --2- 172.21.15.27:0/1458961180 >> v2:172.21.15.119:3300/0 conn(0x7f53f803b950 0x7f53f8006fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f7f1ba8e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f0c007600 0x7f7f0c0d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f7f1ba8e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f0c050f00 0x7f7f0c0e3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f7f1ba8e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0e11c0 0x7f7f0c0e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f7f1ba8e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c04bbc0 con 0x7f7f0c0e11c0 2023-08-11T15:51:30.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f7f1ba8e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0e41d0 con 0x7f7f0c007600 2023-08-11T15:51:30.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f7f1ba8e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c17d160 con 0x7f7f0c050f00 2023-08-11T15:51:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f54053b2640 1 -- 172.21.15.27:0/1458961180 --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f80559d0 con 0x7f53f8007520 2023-08-11T15:51:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa54005250 con 0x7faa74160cd0 2023-08-11T15:51:30.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f5404bb1640 1 --2- 172.21.15.27:0/1458961180 >> v2:172.21.15.27:3300/0 conn(0x7f53f8007520 0x7f53f8007990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f95482a7640 1 -- 172.21.15.27:0/779312831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 msgr2=0x7f95400cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.347+0000 7f95482a7640 1 --2- 172.21.15.27:0/779312831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 0x7f95400cce50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f953c0098a0 tx=0x7f953c02ba50 comp rx=0 tx=0).stop 2023-08-11T15:51:30.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f95482a7640 1 -- 172.21.15.27:0/779312831 shutdown_connections 2023-08-11T15:51:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f54053b2640 1 -- 172.21.15.27:0/1458961180 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f8054930 con 0x7f53f803b950 2023-08-11T15:51:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f95482a7640 1 --2- 172.21.15.27:0/779312831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 0x7f95400cce50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f95482a7640 1 -- 172.21.15.27:0/779312831 >> 172.21.15.27:0/779312831 conn(0x7f954006a4a0 msgr2=0x7f954006a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f95482a7640 1 -- 172.21.15.27:0/779312831 shutdown_connections 2023-08-11T15:51:30.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f95482a7640 1 -- 172.21.15.27:0/779312831 wait complete. 2023-08-11T15:51:30.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f7f19029640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f0c050f00 0x7f7f0c0e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f7f19029640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f0c050f00 0x7f7f0c0e3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53596/0 (socket says 172.21.15.27:53596) 2023-08-11T15:51:30.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f7f19029640 1 -- 172.21.15.27:0/2515996568 learned_addr learned my addr 172.21.15.27:0/2515996568 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f53feffd640 1 -- 172.21.15.27:0/1458961180 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812066974 0 0) 0x7f53f80546d0 con 0x7f53f80e0000 2023-08-11T15:51:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.348+0000 7f53feffd640 1 -- 172.21.15.27:0/1458961180 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53f0003420 con 0x7f53f80e0000 2023-08-11T15:51:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.349+0000 7f7f1982a640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f0c007600 0x7f7f0c0d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.349+0000 7f53ff7fe640 1 -- 172.21.15.27:0/1458961180 >> v2:172.21.15.27:3300/0 conn(0x7f53f8007520 msgr2=0x7f53f8007990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.349+0000 7f53ff7fe640 1 --2- 172.21.15.27:0/1458961180 >> v2:172.21.15.27:3300/0 conn(0x7f53f8007520 0x7f53f8007990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.349+0000 7f53ff7fe640 1 -- 172.21.15.27:0/1458961180 >> v1:172.21.15.119:6789/0 conn(0x7f53f80e0000 legacy=0x7f53f8053fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.349+0000 7f53ff7fe640 1 -- 172.21.15.27:0/1458961180 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f8054c10 con 0x7f53f803b950 2023-08-11T15:51:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.349+0000 7f7f1a02b640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0e11c0 0x7f7f0c0e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.350+0000 7f7f1982a640 1 -- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f0c050f00 msgr2=0x7f7f0c0e3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.350+0000 7f7f1982a640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f0c050f00 0x7f7f0c0e3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.350+0000 7f7f1982a640 1 -- 172.21.15.27:0/2515996568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0e11c0 msgr2=0x7f7f0c0e3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.350+0000 7f7f1982a640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0e11c0 0x7f7f0c0e3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.350+0000 7f7f1982a640 1 -- 172.21.15.27:0/2515996568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f10009400 con 0x7f7f0c007600 2023-08-11T15:51:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.350+0000 7f4ebcff9640 1 -- 172.21.15.27:0/2207724834 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f4eb4053490 con 0x7f4eb80ca060 2023-08-11T15:51:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.351+0000 7f95482a7640 1 Processor -- start 2023-08-11T15:51:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.351+0000 7f95482a7640 1 -- start start 2023-08-11T15:51:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.350+0000 7faa5b7fe640 1 -- 172.21.15.27:0/3916055576 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faa6401a450 con 0x7faa74160cd0 2023-08-11T15:51:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.351+0000 7faa5b7fe640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa6006a150 0x7faa6006c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.351+0000 7faa5b7fe640 1 -- 172.21.15.27:0/3916055576 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faa64088c80 con 0x7faa74160cd0 2023-08-11T15:51:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.351+0000 7f5404bb1640 1 --2- 172.21.15.27:0/1458961180 >> v2:172.21.15.27:3300/0 conn(0x7f53f8007520 0x7f53f8007990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.351+0000 7faa72d9d640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa6006a150 0x7faa6006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.352+0000 7f7f1982a640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f0c007600 0x7f7f0c0d3b30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400c470 tx=0x7f7f0400c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.352+0000 7f53ff7fe640 1 --2- 172.21.15.27:0/1458961180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f803b950 0x7f53f8006fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f53f8050210 tx=0x7f53f402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc0e5fb0f74fd08 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.352+0000 7f95482a7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 0x7f95400c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.352+0000 7f95482a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95400ceac0 0x7f95401f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.353+0000 7faa72d9d640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa6006a150 0x7faa6006c610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faa740d0920 tx=0x7faa6c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.353+0000 7f95482a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9540160cd0 0x7f95400cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.353+0000 7f95482a7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95400cc5f0 con 0x7f95400ceac0 2023-08-11T15:51:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.353+0000 7f95482a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95400c99c0 con 0x7f9540160cd0 2023-08-11T15:51:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.353+0000 7f95482a7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95400cbc50 con 0x7f95400cc9f0 2023-08-11T15:51:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.353+0000 7f9545842640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95400ceac0 0x7f95401f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.353+0000 7f9545842640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95400ceac0 0x7f95401f5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46414/0 (socket says 172.21.15.27:46414) 2023-08-11T15:51:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.353+0000 7f9545842640 1 -- 172.21.15.27:0/4252988015 learned_addr learned my addr 172.21.15.27:0/4252988015 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.353+0000 7f9546043640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 0x7f95400c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.353+0000 7f9546844640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9540160cd0 0x7f95400cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.354+0000 7f9545842640 1 -- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 msgr2=0x7f95400c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.354+0000 7f9545842640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 0x7f95400c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.354+0000 7f9545842640 1 -- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9540160cd0 msgr2=0x7f95400cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.354+0000 7f9545842640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9540160cd0 0x7f95400cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.354+0000 7f9545842640 1 -- 172.21.15.27:0/4252988015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c0094d0 con 0x7f95400ceac0 2023-08-11T15:51:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.354+0000 7f9545842640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95400ceac0 0x7f95401f5d60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f953c0098a0 tx=0x7f953c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.360+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/2568654792 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe4dc053c30 con 0x7fe4e40c14c0 2023-08-11T15:51:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.365+0000 7f53feffd640 1 -- 172.21.15.27:0/1458961180 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4031640 con 0x7f53f803b950 2023-08-11T15:51:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.365+0000 7f7f0a7fc640 1 -- 172.21.15.27:0/2515996568 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04005ce0 con 0x7f7f0c007600 2023-08-11T15:51:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.365+0000 7f7f0a7fc640 1 -- 172.21.15.27:0/2515996568 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f04005e40 con 0x7f7f0c007600 2023-08-11T15:51:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.365+0000 7f7f0a7fc640 1 -- 172.21.15.27:0/2515996568 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f040072d0 con 0x7f7f0c007600 2023-08-11T15:51:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.365+0000 7f53feffd640 1 -- 172.21.15.27:0/1458961180 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f40317a0 con 0x7f53f803b950 2023-08-11T15:51:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.365+0000 7f53feffd640 1 -- 172.21.15.27:0/1458961180 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4031b10 con 0x7f53f803b950 2023-08-11T15:51:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.366+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/2515996568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0c053f20 con 0x7f7f0c007600 2023-08-11T15:51:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.366+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/2515996568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0c0d5880 con 0x7f7f0c007600 2023-08-11T15:51:30.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.366+0000 7faa5b7fe640 1 -- 172.21.15.27:0/3916055576 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7faa640541a0 con 0x7faa74160cd0 2023-08-11T15:51:30.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.366+0000 7f54053b2640 1 -- 172.21.15.27:0/1458961180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f803b950 msgr2=0x7f53f8006fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.366+0000 7f54053b2640 1 --2- 172.21.15.27:0/1458961180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f803b950 0x7f53f8006fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f53f8050210 tx=0x7f53f402b8f0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.367+0000 7f54053b2640 1 -- 172.21.15.27:0/1458961180 shutdown_connections 2023-08-11T15:51:30.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.367+0000 7f54053b2640 1 --2- 172.21.15.27:0/1458961180 >> v2:172.21.15.27:3300/0 conn(0x7f53f8007520 0x7f53f8007990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.367+0000 7f54053b2640 1 --2- 172.21.15.27:0/1458961180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f803b950 0x7f53f8006fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.367+0000 7f54053b2640 1 -- 172.21.15.27:0/1458961180 >> 172.21.15.27:0/1458961180 conn(0x7f53f8055c10 msgr2=0x7f53f8056020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.367+0000 7f54053b2640 1 -- 172.21.15.27:0/1458961180 shutdown_connections 2023-08-11T15:51:30.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.367+0000 7f54053b2640 1 -- 172.21.15.27:0/1458961180 wait complete. 2023-08-11T15:51:30.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.367+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/2515996568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0c005330 con 0x7f7f0c007600 2023-08-11T15:51:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.369+0000 7f9536ffd640 1 -- 172.21.15.27:0/4252988015 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c003a20 con 0x7f95400ceac0 2023-08-11T15:51:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.369+0000 7f9536ffd640 1 -- 172.21.15.27:0/4252988015 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c003ba0 con 0x7f95400ceac0 2023-08-11T15:51:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.369+0000 7f9536ffd640 1 -- 172.21.15.27:0/4252988015 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c041410 con 0x7f95400ceac0 2023-08-11T15:51:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.369+0000 7f95482a7640 1 -- 172.21.15.27:0/4252988015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95400cbf30 con 0x7f95400ceac0 2023-08-11T15:51:30.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.369+0000 7f95482a7640 1 -- 172.21.15.27:0/4252988015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9540152c10 con 0x7f95400ceac0 2023-08-11T15:51:30.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.370+0000 7f54053b2640 1 Processor -- start 2023-08-11T15:51:30.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.371+0000 7f54053b2640 1 -- start start 2023-08-11T15:51:30.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.371+0000 7fee40ff9640 1 -- 172.21.15.27:0/3461844050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fee3c054620 con 0x7fee4406c8f0 2023-08-11T15:51:30.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.370+0000 7f7f0a7fc640 1 -- 172.21.15.27:0/2515996568 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7f04007430 con 0x7f7f0c007600 2023-08-11T15:51:30.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.371+0000 7f95482a7640 1 -- 172.21.15.27:0/4252988015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95400cc730 con 0x7f95400ceac0 2023-08-11T15:51:30.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.372+0000 7f7f0a7fc640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ef806a0d0 0x7f7ef806c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.372+0000 7f7f0a7fc640 1 -- 172.21.15.27:0/2515996568 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7f04087c30 con 0x7f7f0c007600 2023-08-11T15:51:30.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.373+0000 7f54053b2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f8007520 0x7f53f80e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.373+0000 7f7f19029640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ef806a0d0 0x7f7ef806c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.373+0000 7f54053b2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f803b950 0x7f53f80d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.374+0000 7f53fffff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f8007520 0x7f53f80e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.374+0000 7f53fffff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f8007520 0x7f53f80e03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53628/0 (socket says 172.21.15.27:53628) 2023-08-11T15:51:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.374+0000 7f54053b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f80e0000 0x7f53f80506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.374+0000 7f54053b2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f804cce0 con 0x7f53f803b950 2023-08-11T15:51:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.374+0000 7f54053b2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80e0e10 con 0x7f53f80e0000 2023-08-11T15:51:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.374+0000 7f54053b2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80e1080 con 0x7f53f8007520 2023-08-11T15:51:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.374+0000 7f53fffff640 1 -- 172.21.15.27:0/1483577441 learned_addr learned my addr 172.21.15.27:0/1483577441 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.374+0000 7f53ff7fe640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f803b950 0x7f53f80d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.375+0000 7f5404bb1640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f80e0000 0x7f53f80506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.374+0000 7f9536ffd640 1 -- 172.21.15.27:0/4252988015 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f953c037400 con 0x7f95400ceac0 2023-08-11T15:51:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.375+0000 7f9536ffd640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f952406a0a0 0x7f952406c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.375+0000 7f9536ffd640 1 -- 172.21.15.27:0/4252988015 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f953c03e070 con 0x7f95400ceac0 2023-08-11T15:51:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.375+0000 7f5404bb1640 1 -- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f8007520 msgr2=0x7f53f80e03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.375+0000 7f5404bb1640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f8007520 0x7f53f80e03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.375+0000 7f5404bb1640 1 -- 172.21.15.27:0/1483577441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f803b950 msgr2=0x7f53f80d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.375+0000 7f5404bb1640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f803b950 0x7f53f80d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.375+0000 7f5404bb1640 1 -- 172.21.15.27:0/1483577441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec009400 con 0x7f53f80e0000 2023-08-11T15:51:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.376+0000 7f5404bb1640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f80e0000 0x7f53f80506b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00e880 tx=0x7f53ec00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.377+0000 7f9546043640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f952406a0a0 0x7f952406c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.377+0000 7f7f19029640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ef806a0d0 0x7f7ef806c590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0d4f70 tx=0x7f7f1003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.378+0000 7f9546043640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f952406a0a0 0x7f952406c560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f95300016e0 tx=0x7f9530009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.384+0000 7f9536ffd640 1 -- 172.21.15.27:0/4252988015 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f953c075fc0 con 0x7f95400ceac0 2023-08-11T15:51:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.385+0000 7f53fcff9640 1 -- 172.21.15.27:0/1483577441 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec004020 con 0x7f53f80e0000 2023-08-11T15:51:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.385+0000 7f53fcff9640 1 -- 172.21.15.27:0/1483577441 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec004180 con 0x7f53f80e0000 2023-08-11T15:51:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.385+0000 7f53fcff9640 1 -- 172.21.15.27:0/1483577441 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec00c940 con 0x7f53f80e0000 2023-08-11T15:51:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.385+0000 7f54053b2640 1 -- 172.21.15.27:0/1483577441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f4009400 con 0x7f53f80e0000 2023-08-11T15:51:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.385+0000 7f54053b2640 1 -- 172.21.15.27:0/1483577441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f80d4d60 con 0x7f53f80e0000 2023-08-11T15:51:30.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.387+0000 7f54053b2640 1 -- 172.21.15.27:0/1483577441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f804ce90 con 0x7f53f80e0000 2023-08-11T15:51:30.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.387+0000 7f7f0a7fc640 1 -- 172.21.15.27:0/2515996568 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f7f04053150 con 0x7f7f0c007600 2023-08-11T15:51:30.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.390+0000 7f53fcff9640 1 -- 172.21.15.27:0/1483577441 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f53ec00caa0 con 0x7f53f80e0000 2023-08-11T15:51:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.392+0000 7f53fcff9640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f53e406a0d0 0x7f53e406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.392+0000 7f53fcff9640 1 -- 172.21.15.27:0/1483577441 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f53ec088b00 con 0x7f53f80e0000 2023-08-11T15:51:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.393+0000 7f53fffff640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f53e406a0d0 0x7f53e406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.394+0000 7f53fffff640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f53e406a0d0 0x7f53e406c590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5400078de0 tx=0x7f5400068830 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.402+0000 7f53fcff9640 1 -- 172.21.15.27:0/1483577441 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f53ec054120 con 0x7f53f80e0000 2023-08-11T15:51:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.736+0000 7f625e887640 1 -- 172.21.15.27:0/1662145139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f62380057d0 con 0x7f62580ceac0 2023-08-11T15:51:30.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.737+0000 7f6254ff9640 1 -- 172.21.15.27:0/1662145139 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6250017090 con 0x7f62580ceac0 2023-08-11T15:51:30.738 INFO:teuthology.orchestra.run.smithi027.stdout:60129542156 2023-08-11T15:51:30.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.741+0000 7f623e7fc640 1 -- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f624406a120 msgr2=0x7f624406c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.741+0000 7f623e7fc640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f624406a120 0x7f624406c5e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f624c02b980 tx=0x7f624c03b040 comp rx=0 tx=0).stop 2023-08-11T15:51:30.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.741+0000 7f623e7fc640 1 -- 172.21.15.27:0/1662145139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62580ceac0 msgr2=0x7f6258159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.741+0000 7f623e7fc640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62580ceac0 0x7f6258159a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f62580ca480 tx=0x7f625000cf00 comp rx=0 tx=0).stop 2023-08-11T15:51:30.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.742+0000 7f623e7fc640 1 -- 172.21.15.27:0/1662145139 shutdown_connections 2023-08-11T15:51:30.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.742+0000 7f623e7fc640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f624406a120 0x7f624406c5e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.742+0000 7f623e7fc640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62580ceac0 0x7f6258159a30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.742+0000 7f623e7fc640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62580c9030 0x7f62580b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.742+0000 7f623e7fc640 1 --2- 172.21.15.27:0/1662145139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f625806ffe0 0x7f62580b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.742+0000 7f623e7fc640 1 -- 172.21.15.27:0/1662145139 >> 172.21.15.27:0/1662145139 conn(0x7f62580cdfd0 msgr2=0x7f625814c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.743+0000 7f623e7fc640 1 -- 172.21.15.27:0/1662145139 shutdown_connections 2023-08-11T15:51:30.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.743+0000 7f623e7fc640 1 -- 172.21.15.27:0/1662145139 wait complete. 2023-08-11T15:51:30.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.752+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4ea00050c0 con 0x7f4eb80ca060 2023-08-11T15:51:30.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.753+0000 7f4ebcff9640 1 -- 172.21.15.27:0/2207724834 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4eb405c020 con 0x7f4eb80ca060 2023-08-11T15:51:30.758 INFO:teuthology.orchestra.run.smithi027.stdout:60129542155 2023-08-11T15:51:30.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.757+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ea406a050 msgr2=0x7f4ea406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.757+0000 7f4ec668e640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ea406a050 0x7f4ea406c510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8006b20 tx=0x7f4ec00a9000 comp rx=0 tx=0).stop 2023-08-11T15:51:30.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.757+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb80ca060 msgr2=0x7f4eb80e6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.757+0000 7f4ec668e640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb80ca060 0x7f4eb80e6050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8038940 tx=0x7f4eb400c820 comp rx=0 tx=0).stop 2023-08-11T15:51:30.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.758+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 shutdown_connections 2023-08-11T15:51:30.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.758+0000 7f4ec668e640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ea406a050 0x7f4ea406c510 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.758+0000 7f4ec668e640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb80ca060 0x7f4eb80e6050 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.758+0000 7f4ec668e640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb8007810 0x7f4eb804eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.758+0000 7f4ec668e640 1 --2- 172.21.15.27:0/2207724834 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4eb8006e70 0x7f4eb80ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.758+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 >> 172.21.15.27:0/2207724834 conn(0x7f4eb80559b0 msgr2=0x7f4eb8004b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.759+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 shutdown_connections 2023-08-11T15:51:30.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.759+0000 7f4ec668e640 1 -- 172.21.15.27:0/2207724834 wait complete. 2023-08-11T15:51:30.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542157 got 60129542156 for osd.0 2023-08-11T15:51:30.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.3 2023-08-11T15:51:30.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.788+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faa540057d0 con 0x7faa74160cd0 2023-08-11T15:51:30.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.788+0000 7faa5b7fe640 1 -- 172.21.15.27:0/3916055576 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faa640577c0 con 0x7faa74160cd0 2023-08-11T15:51:30.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.789+0000 7fe4e966a640 1 -- 172.21.15.27:0/2568654792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe4c40050c0 con 0x7fe4e40c14c0 2023-08-11T15:51:30.790 INFO:teuthology.orchestra.run.smithi027.stdout:60129542154 2023-08-11T15:51:30.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.789+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/2568654792 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe4dc057210 con 0x7fe4e40c14c0 2023-08-11T15:51:30.791 INFO:teuthology.orchestra.run.smithi027.stdout:60129542154 2023-08-11T15:51:30.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.791+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa6006a150 msgr2=0x7faa6006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.791+0000 7faa7914e640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa6006a150 0x7faa6006c610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faa740d0920 tx=0x7faa6c009040 comp rx=0 tx=0).stop 2023-08-11T15:51:30.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.791+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74160cd0 msgr2=0x7faa740c72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.791+0000 7faa7914e640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74160cd0 0x7faa740c72b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa6400c5c0 tx=0x7faa6400ca80 comp rx=0 tx=0).stop 2023-08-11T15:51:30.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.791+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 shutdown_connections 2023-08-11T15:51:30.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.791+0000 7faa7914e640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa6006a150 0x7faa6006c610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.791+0000 7faa7914e640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74160cd0 0x7faa740c72b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.791+0000 7faa7914e640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa740ceac0 0x7faa740b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.791+0000 7faa7914e640 1 --2- 172.21.15.27:0/3916055576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa740cc9f0 0x7faa740c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.791+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 >> 172.21.15.27:0/3916055576 conn(0x7faa7406a4a0 msgr2=0x7faa74163460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.792+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 shutdown_connections 2023-08-11T15:51:30.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.792+0000 7faa7914e640 1 -- 172.21.15.27:0/3916055576 wait complete. 2023-08-11T15:51:30.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.793+0000 7fe4e966a640 1 -- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4d006a050 msgr2=0x7fe4d006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.793+0000 7fe4e966a640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4d006a050 0x7fe4d006c510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40c0660 tx=0x7fe4d8009190 comp rx=0 tx=0).stop 2023-08-11T15:51:30.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.793+0000 7fe4e966a640 1 -- 172.21.15.27:0/2568654792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e40c14c0 msgr2=0x7fe4e415c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.793+0000 7fe4e966a640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e40c14c0 0x7fe4e415c320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00cba0 tx=0x7fe4dc00a430 comp rx=0 tx=0).stop 2023-08-11T15:51:30.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.793+0000 7fe4e966a640 1 -- 172.21.15.27:0/2568654792 shutdown_connections 2023-08-11T15:51:30.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.793+0000 7fe4e966a640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4d006a050 0x7fe4d006c510 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.793+0000 7fe4e966a640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e40ccb70 0x7fe4e40c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.793+0000 7fe4e966a640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e40c14c0 0x7fe4e415c320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.794+0000 7fe4e966a640 1 --2- 172.21.15.27:0/2568654792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4e406c150 0x7fe4e40c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.794+0000 7fe4e966a640 1 -- 172.21.15.27:0/2568654792 >> 172.21.15.27:0/2568654792 conn(0x7fe4e40afb00 msgr2=0x7fe4e41649c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.794+0000 7fe4e966a640 1 -- 172.21.15.27:0/2568654792 shutdown_connections 2023-08-11T15:51:30.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.794+0000 7fe4e966a640 1 -- 172.21.15.27:0/2568654792 wait complete. 2023-08-11T15:51:30.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.798+0000 7f95482a7640 1 -- 172.21.15.27:0/4252988015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9540073990 con 0x7f95400ceac0 2023-08-11T15:51:30.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.799+0000 7f9536ffd640 1 -- 172.21.15.27:0/4252988015 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f953c0795e0 con 0x7f95400ceac0 2023-08-11T15:51:30.800 INFO:teuthology.orchestra.run.smithi027.stdout:60129542156 2023-08-11T15:51:30.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 -- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f952406a0a0 msgr2=0x7f952406c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f952406a0a0 0x7f952406c560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f95300016e0 tx=0x7f9530009040 comp rx=0 tx=0).stop 2023-08-11T15:51:30.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 -- 172.21.15.27:0/4252988015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95400ceac0 msgr2=0x7f95401f5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95400ceac0 0x7f95401f5d60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f953c0098a0 tx=0x7f953c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 -- 172.21.15.27:0/4252988015 shutdown_connections 2023-08-11T15:51:30.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f952406a0a0 0x7f952406c560 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9540160cd0 0x7f95400cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95400ceac0 0x7f95401f5d60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 --2- 172.21.15.27:0/4252988015 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f95400cc9f0 0x7f95400c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 -- 172.21.15.27:0/4252988015 >> 172.21.15.27:0/4252988015 conn(0x7f954006a4a0 msgr2=0x7f9540161300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 -- 172.21.15.27:0/4252988015 shutdown_connections 2023-08-11T15:51:30.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.802+0000 7f95482a7640 1 -- 172.21.15.27:0/4252988015 wait complete. 2023-08-11T15:51:30.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.803+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/2515996568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7f0c0d74e0 con 0x7f7f0c007600 2023-08-11T15:51:30.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.804+0000 7f7f0a7fc640 1 -- 172.21.15.27:0/2515996568 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f04056770 con 0x7f7f0c007600 2023-08-11T15:51:30.806 INFO:teuthology.orchestra.run.smithi027.stdout:60129542155 2023-08-11T15:51:30.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.806+0000 7fee4965d640 1 -- 172.21.15.27:0/3461844050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fee240057d0 con 0x7fee4406c8f0 2023-08-11T15:51:30.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7fee40ff9640 1 -- 172.21.15.27:0/3461844050 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fee3c057c40 con 0x7fee4406c8f0 2023-08-11T15:51:30.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ef806a0d0 msgr2=0x7f7ef806c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ef806a0d0 0x7f7ef806c590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0d4f70 tx=0x7f7f1003d040 comp rx=0 tx=0).stop 2023-08-11T15:51:30.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f0c007600 msgr2=0x7f7f0c0d3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f0c007600 0x7f7f0c0d3b30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400c470 tx=0x7f7f0400c930 comp rx=0 tx=0).stop 2023-08-11T15:51:30.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/2515996568 shutdown_connections 2023-08-11T15:51:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ef806a0d0 0x7f7ef806c590 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f0c050f00 0x7f7f0c0e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0e11c0 0x7f7f0c0e3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 --2- 172.21.15.27:0/2515996568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f0c007600 0x7f7f0c0d3b30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/2515996568 >> 172.21.15.27:0/2515996568 conn(0x7f7f0c009fe0 msgr2=0x7f7f0c004b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.811 INFO:teuthology.orchestra.run.smithi027.stdout:60129542154 2023-08-11T15:51:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/2515996568 shutdown_connections 2023-08-11T15:51:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.807+0000 7f7f1ba8e640 1 -- 172.21.15.27:0/2515996568 wait complete. 2023-08-11T15:51:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 -- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3006a050 msgr2=0x7fee3006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3006a050 0x7fee3006c510 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee380014f0 tx=0x7fee380078f0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 -- 172.21.15.27:0/3461844050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4406c8f0 msgr2=0x7fee44070000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4406c8f0 0x7fee44070000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fee3c007ef0 tx=0x7fee3c00e4b0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 -- 172.21.15.27:0/3461844050 shutdown_connections 2023-08-11T15:51:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3006a050 0x7fee3006c510 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee440c9e90 0x7fee44070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4406c8f0 0x7fee44070000 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 --2- 172.21.15.27:0/3461844050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee4406a2f0 0x7fee44146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 -- 172.21.15.27:0/3461844050 >> 172.21.15.27:0/3461844050 conn(0x7fee44069800 msgr2=0x7fee440c3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 -- 172.21.15.27:0/3461844050 shutdown_connections 2023-08-11T15:51:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.811+0000 7fee4965d640 1 -- 172.21.15.27:0/3461844050 wait complete. 2023-08-11T15:51:30.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.2 2023-08-11T15:51:30.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.1 2023-08-11T15:51:30.824 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542157 got 60129542156 for osd.5 2023-08-11T15:51:30.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.6 2023-08-11T15:51:30.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.828+0000 7f54053b2640 1 -- 172.21.15.27:0/1483577441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f53f804ce90 con 0x7f53f80e0000 2023-08-11T15:51:30.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.828+0000 7f53fcff9640 1 -- 172.21.15.27:0/1483577441 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f53f804ce90 con 0x7f53f80e0000 2023-08-11T15:51:30.831 INFO:teuthology.orchestra.run.smithi027.stdout:60129542155 2023-08-11T15:51:30.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 -- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f53e406a0d0 msgr2=0x7f53e406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f53e406a0d0 0x7f53e406c590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5400078de0 tx=0x7f5400068830 comp rx=0 tx=0).stop 2023-08-11T15:51:30.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 -- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f80e0000 msgr2=0x7f53f80506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:30.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f80e0000 0x7f53f80506b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00e880 tx=0x7f53ec00ed40 comp rx=0 tx=0).stop 2023-08-11T15:51:30.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 -- 172.21.15.27:0/1483577441 shutdown_connections 2023-08-11T15:51:30.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f53e406a0d0 0x7f53e406c590 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f80e0000 0x7f53f80506b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f803b950 0x7f53f80d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 --2- 172.21.15.27:0/1483577441 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f8007520 0x7f53f80e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:30.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 -- 172.21.15.27:0/1483577441 >> 172.21.15.27:0/1483577441 conn(0x7f53f8055c10 msgr2=0x7f53f80de850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:30.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 -- 172.21.15.27:0/1483577441 shutdown_connections 2023-08-11T15:51:30.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:30.831+0000 7f54053b2640 1 -- 172.21.15.27:0/1483577441 wait complete. 2023-08-11T15:51:30.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.4 2023-08-11T15:51:30.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.7 2023-08-11T15:51:31.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-11T15:51:31.781 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-11T15:51:31.816 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-11T15:51:31.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-11T15:51:31.825 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-11T15:51:31.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-11T15:51:31.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-11T15:51:31.850 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-11T15:51:32.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.056+0000 7f7b71071640 1 Processor -- start 2023-08-11T15:51:32.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.057+0000 7f7b71071640 1 -- start start 2023-08-11T15:51:32.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.059+0000 7f7b71071640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7b6c0cd170 0x7f7b6c159460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.059+0000 7f7b71071640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b6c0c9030 0x7f7b6c15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.059+0000 7f7b71071640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c06b9c0 con 0x7f7b6c0ceac0 2023-08-11T15:51:32.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.059+0000 7f7b71071640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6c0c97a0 con 0x7f7b6c0cd170 2023-08-11T15:51:32.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.059+0000 7f7b71071640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6c0c9a80 con 0x7f7b6c0c9030 2023-08-11T15:51:32.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.060+0000 7f7b6a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7b6c0cd170 0x7f7b6c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.060+0000 7f7b6a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7b6c0cd170 0x7f7b6c159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46422/0 (socket says 172.21.15.27:46422) 2023-08-11T15:51:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.060+0000 7f7b6a59c640 1 -- 172.21.15.27:0/4214058472 learned_addr learned my addr 172.21.15.27:0/4214058472 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.061+0000 7f7b6b59e640 1 --2- 172.21.15.27:0/4214058472 >> v2:172.21.15.119:3300/0 conn(0x7f7b6c0c9030 0x7f7b6c15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.061+0000 7f7b6a59c640 1 -- 172.21.15.27:0/4214058472 >> v2:172.21.15.119:3300/0 conn(0x7f7b6c0c9030 msgr2=0x7f7b6c15c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.061+0000 7f7b6a59c640 1 --2- 172.21.15.27:0/4214058472 >> v2:172.21.15.119:3300/0 conn(0x7f7b6c0c9030 0x7f7b6c15c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.061+0000 7f7b6a59c640 1 -- 172.21.15.27:0/4214058472 >> v1:172.21.15.27:6789/0 conn(0x7f7b6c0ceac0 legacy=0x7f7b6c15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.061+0000 7f7b6a59c640 1 -- 172.21.15.27:0/4214058472 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c0c9d60 con 0x7f7b6c0cd170 2023-08-11T15:51:32.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.062+0000 7f7b6a59c640 1 --2- 172.21.15.27:0/4214058472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b6c0cd170 0x7f7b6c159460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0ca460 tx=0x7f7b6402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1c5cdc8dba6bff server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.070+0000 7f7b69d9b640 1 -- 172.21.15.27:0/4214058472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6402cb10 con 0x7f7b6c0cd170 2023-08-11T15:51:32.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.070+0000 7f7b69d9b640 1 -- 172.21.15.27:0/4214058472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6402cc70 con 0x7f7b6c0cd170 2023-08-11T15:51:32.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.071+0000 7f7b71071640 1 -- 172.21.15.27:0/4214058472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b6c0cd170 msgr2=0x7f7b6c159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.071+0000 7f7b71071640 1 --2- 172.21.15.27:0/4214058472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b6c0cd170 0x7f7b6c159460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0ca460 tx=0x7f7b6402bf70 comp rx=0 tx=0).stop 2023-08-11T15:51:32.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.071+0000 7f7b71071640 1 -- 172.21.15.27:0/4214058472 shutdown_connections 2023-08-11T15:51:32.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.071+0000 7f7b71071640 1 --2- 172.21.15.27:0/4214058472 >> v2:172.21.15.119:3300/0 conn(0x7f7b6c0c9030 0x7f7b6c15c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.071+0000 7f7b71071640 1 --2- 172.21.15.27:0/4214058472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b6c0cd170 0x7f7b6c159460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.071+0000 7f7b71071640 1 -- 172.21.15.27:0/4214058472 >> 172.21.15.27:0/4214058472 conn(0x7f7b6c0cdfd0 msgr2=0x7f7b6c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.071+0000 7f7b71071640 1 -- 172.21.15.27:0/4214058472 shutdown_connections 2023-08-11T15:51:32.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.071+0000 7f7b71071640 1 -- 172.21.15.27:0/4214058472 wait complete. 2023-08-11T15:51:32.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.074+0000 7f7b71071640 1 Processor -- start 2023-08-11T15:51:32.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.074+0000 7f7b71071640 1 -- start start 2023-08-11T15:51:32.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.076+0000 7f7b71071640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0c9030 0x7f7b6c0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.076+0000 7f7b71071640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cd170 0x7f7b6c0c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.077+0000 7f7b71071640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b6c0ceac0 0x7f7b6c0c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.077+0000 7f7b6ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0c9030 0x7f7b6c0b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.077+0000 7f7b6ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0c9030 0x7f7b6c0b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40846/0 (socket says 172.21.15.27:40846) 2023-08-11T15:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.077+0000 7f7b6a59c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cd170 0x7f7b6c0c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.077+0000 7f7b6b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b6c0ceac0 0x7f7b6c0c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.077+0000 7f7b6a59c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cd170 0x7f7b6c0c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53642/0 (socket says 172.21.15.27:53642) 2023-08-11T15:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.077+0000 7f7b6ad9d640 1 -- 172.21.15.27:0/2994846887 learned_addr learned my addr 172.21.15.27:0/2994846887 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.077+0000 7f7b71071640 1 -- 172.21.15.27:0/2994846887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c14fdf0 con 0x7f7b6c0ceac0 2023-08-11T15:51:32.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.077+0000 7f7b71071640 1 -- 172.21.15.27:0/2994846887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c0c2740 con 0x7f7b6c0c9030 2023-08-11T15:51:32.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.077+0000 7f7b71071640 1 -- 172.21.15.27:0/2994846887 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c0c2260 con 0x7f7b6c0cd170 2023-08-11T15:51:32.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.078+0000 7f7b6ad9d640 1 -- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cd170 msgr2=0x7f7b6c0c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.078+0000 7f7b6ad9d640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cd170 0x7f7b6c0c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.078+0000 7f7b6ad9d640 1 -- 172.21.15.27:0/2994846887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b6c0ceac0 msgr2=0x7f7b6c0c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.078+0000 7f7b6ad9d640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b6c0ceac0 0x7f7b6c0c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.078+0000 7f7b6ad9d640 1 -- 172.21.15.27:0/2994846887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64009b30 con 0x7f7b6c0c9030 2023-08-11T15:51:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.078+0000 7f7b6b59e640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b6c0ceac0 0x7f7b6c0c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.079+0000 7f7b6ad9d640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0c9030 0x7f7b6c0b0bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c430 tx=0x7f7b5c00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.080+0000 7f7b537fe640 1 -- 172.21.15.27:0/2994846887 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c011790 con 0x7f7b6c0c9030 2023-08-11T15:51:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.080+0000 7f7b537fe640 1 -- 172.21.15.27:0/2994846887 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c0118f0 con 0x7f7b6c0c9030 2023-08-11T15:51:32.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.080+0000 7f7b537fe640 1 -- 172.21.15.27:0/2994846887 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c011ce0 con 0x7f7b6c0c9030 2023-08-11T15:51:32.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.080+0000 7f7b71071640 1 -- 172.21.15.27:0/2994846887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c0c1ca0 con 0x7f7b6c0c9030 2023-08-11T15:51:32.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.080+0000 7f7b71071640 1 -- 172.21.15.27:0/2994846887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c0b1110 con 0x7f7b6c0c9030 2023-08-11T15:51:32.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.084+0000 7f7b537fe640 1 -- 172.21.15.27:0/2994846887 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7b5c01c460 con 0x7f7b6c0c9030 2023-08-11T15:51:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.086+0000 7f7b537fe640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7b5806a050 0x7f7b5806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.086+0000 7f7b537fe640 1 -- 172.21.15.27:0/2994846887 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7b5c088680 con 0x7f7b6c0c9030 2023-08-11T15:51:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.087+0000 7f7b71071640 1 -- 172.21.15.27:0/2994846887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4c005250 con 0x7f7b6c0c9030 2023-08-11T15:51:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.087+0000 7f7b6a59c640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7b5806a050 0x7f7b5806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.088+0000 7f7b6a59c640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7b5806a050 0x7f7b5806c510 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b64035000 tx=0x7f7b64030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.101+0000 7f7b537fe640 1 -- 172.21.15.27:0/2994846887 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f7b5c053a90 con 0x7f7b6c0c9030 2023-08-11T15:51:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.114+0000 7f7401b4e640 1 Processor -- start 2023-08-11T15:51:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.115+0000 7f7401b4e640 1 -- start start 2023-08-11T15:51:32.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.116+0000 7f7401b4e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f73fc15f8d0 0x7f73fc0ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.116+0000 7f7401b4e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc0b3df0 con 0x7f73fc0d1a50 2023-08-11T15:51:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.116+0000 7f7401b4e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc15fcb0 con 0x7f73fc0b5390 2023-08-11T15:51:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.116+0000 7f7401b4e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73fc0d2050 con 0x7f73fc15f8d0 2023-08-11T15:51:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.117+0000 7f73fbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f73fc0d1a50 0x7f73fc15ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48486/0 (socket says 172.21.15.27:48486) 2023-08-11T15:51:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.117+0000 7f73fbfff640 1 -- 172.21.15.27:0/442718989 learned_addr learned my addr 172.21.15.27:0/442718989 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.117+0000 7f73fb7fe640 1 --2- 172.21.15.27:0/442718989 >> v2:172.21.15.27:3300/0 conn(0x7f73fc15f8d0 0x7f73fc0ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.117+0000 7f73fb7fe640 1 -- 172.21.15.27:0/442718989 >> v1:172.21.15.119:6789/0 conn(0x7f73fc0b5390 legacy=0x7f73fc0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.117+0000 7f73fb7fe640 1 -- 172.21.15.27:0/442718989 >> v1:172.21.15.27:6789/0 conn(0x7f73fc0d1a50 legacy=0x7f73fc15ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.118+0000 7f73fb7fe640 1 -- 172.21.15.27:0/442718989 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc0b4090 con 0x7f73fc15f8d0 2023-08-11T15:51:32.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.119+0000 7f2377fff640 1 Processor -- start 2023-08-11T15:51:32.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.119+0000 7f2377fff640 1 -- start start 2023-08-11T15:51:32.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.119+0000 7f73fb7fe640 1 --2- 172.21.15.27:0/442718989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc15f8d0 0x7f73fc0ccf10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73e40097a0 tx=0x7f73e402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96e780de1614d91 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.121+0000 7f2377fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f237815c350 0x7f23780ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.121+0000 7f2377fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23780b7bd0 0x7f23780cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.121+0000 7f2377fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237815d970 con 0x7f237815c730 2023-08-11T15:51:32.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.121+0000 7f2377fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23780d0740 con 0x7f23780b7bd0 2023-08-11T15:51:32.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.121+0000 7f2377fff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237815d3b0 con 0x7f237815c350 2023-08-11T15:51:32.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.121+0000 7f93ea191640 1 Processor -- start 2023-08-11T15:51:32.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.122+0000 7f23777fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f237815c730 0x7f237815cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48496/0 (socket says 172.21.15.27:48496) 2023-08-11T15:51:32.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.122+0000 7f23777fe640 1 -- 172.21.15.27:0/2594149509 learned_addr learned my addr 172.21.15.27:0/2594149509 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:32.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.122+0000 7f93ea191640 1 -- start start 2023-08-11T15:51:32.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.122+0000 7f23767fc640 1 --2- 172.21.15.27:0/2594149509 >> v2:172.21.15.27:3300/0 conn(0x7f23780b7bd0 0x7f23780cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.123+0000 7f23767fc640 1 -- 172.21.15.27:0/2594149509 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f237815c350 msgr2=0x7f23780ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.123+0000 7f2376ffd640 1 --2- 172.21.15.27:0/2594149509 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f237815c350 0x7f23780ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.123+0000 7f23767fc640 1 --2- 172.21.15.27:0/2594149509 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f237815c350 0x7f23780ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.123+0000 7f23767fc640 1 -- 172.21.15.27:0/2594149509 >> v1:172.21.15.27:6789/0 conn(0x7f237815c730 legacy=0x7f237815cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.123+0000 7f23767fc640 1 -- 172.21.15.27:0/2594149509 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237815d690 con 0x7f23780b7bd0 2023-08-11T15:51:32.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.124+0000 7f93ea191640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.124+0000 7f73fa7fc640 1 -- 172.21.15.27:0/442718989 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4030710 con 0x7f73fc15f8d0 2023-08-11T15:51:32.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.124+0000 7f73fa7fc640 1 -- 172.21.15.27:0/442718989 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e402e050 con 0x7f73fc15f8d0 2023-08-11T15:51:32.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.124+0000 7f93ea191640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93e40c9e90 0x7f93e406f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.124+0000 7f93ea191640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e406c460 con 0x7f93e40c9530 2023-08-11T15:51:32.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.124+0000 7f93ea191640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e4158980 con 0x7f93e40c9e90 2023-08-11T15:51:32.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.124+0000 7f93ea191640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40cc510 con 0x7f93e406a2f0 2023-08-11T15:51:32.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f7401b4e640 1 -- 172.21.15.27:0/442718989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc15f8d0 msgr2=0x7f73fc0ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f7401b4e640 1 --2- 172.21.15.27:0/442718989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc15f8d0 0x7f73fc0ccf10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73e40097a0 tx=0x7f73e402b790 comp rx=0 tx=0).stop 2023-08-11T15:51:32.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f93e918f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f7401b4e640 1 -- 172.21.15.27:0/442718989 shutdown_connections 2023-08-11T15:51:32.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f7401b4e640 1 --2- 172.21.15.27:0/442718989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc15f8d0 0x7f73fc0ccf10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f7401b4e640 1 -- 172.21.15.27:0/442718989 >> 172.21.15.27:0/442718989 conn(0x7f73fc06a250 msgr2=0x7f73fc06a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f93e9990640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f93e40c9530 0x7f93e406f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48500/0 (socket says 172.21.15.27:48500) 2023-08-11T15:51:32.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f93e9990640 1 -- 172.21.15.27:0/317374149 learned_addr learned my addr 172.21.15.27:0/317374149 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:32.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f93e918f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e406c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53652/0 (socket says 172.21.15.27:53652) 2023-08-11T15:51:32.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f23767fc640 1 --2- 172.21.15.27:0/2594149509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23780b7bd0 0x7f23780cd4e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f23680097a0 tx=0x7f236802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b228929242c5065 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f7401b4e640 1 -- 172.21.15.27:0/442718989 shutdown_connections 2023-08-11T15:51:32.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.125+0000 7f7401b4e640 1 -- 172.21.15.27:0/442718989 wait complete. 2023-08-11T15:51:32.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.126+0000 7f2375ffb640 1 -- 172.21.15.27:0/2594149509 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368030710 con 0x7f23780b7bd0 2023-08-11T15:51:32.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.126+0000 7f2375ffb640 1 -- 172.21.15.27:0/2594149509 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236802e050 con 0x7f23780b7bd0 2023-08-11T15:51:32.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.126+0000 7f2375ffb640 1 -- 172.21.15.27:0/2594149509 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368030a90 con 0x7f23780b7bd0 2023-08-11T15:51:32.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f93e3fff640 1 -- 172.21.15.27:0/317374149 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029004043 0 0) 0x7f93e406c460 con 0x7f93e40c9530 2023-08-11T15:51:32.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f2377fff640 1 -- 172.21.15.27:0/2594149509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23780b7bd0 msgr2=0x7f23780cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f2377fff640 1 --2- 172.21.15.27:0/2594149509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23780b7bd0 0x7f23780cd4e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f23680097a0 tx=0x7f236802b710 comp rx=0 tx=0).stop 2023-08-11T15:51:32.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f2377fff640 1 -- 172.21.15.27:0/2594149509 shutdown_connections 2023-08-11T15:51:32.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f2377fff640 1 --2- 172.21.15.27:0/2594149509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23780b7bd0 0x7f23780cd4e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f2377fff640 1 --2- 172.21.15.27:0/2594149509 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f237815c350 0x7f23780ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f2377fff640 1 -- 172.21.15.27:0/2594149509 >> 172.21.15.27:0/2594149509 conn(0x7f237806a250 msgr2=0x7f237806a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.128+0000 7f2377fff640 1 -- 172.21.15.27:0/2594149509 shutdown_connections 2023-08-11T15:51:32.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f93e918f640 1 -- 172.21.15.27:0/317374149 >> v2:172.21.15.119:3300/0 conn(0x7f93e40c9e90 msgr2=0x7f93e406f230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T15:51:32.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f93e918f640 1 --2- 172.21.15.27:0/317374149 >> v2:172.21.15.119:3300/0 conn(0x7f93e40c9e90 0x7f93e406f230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f93e918f640 1 -- 172.21.15.27:0/317374149 >> v1:172.21.15.27:6789/0 conn(0x7f93e40c9530 legacy=0x7f93e406f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.127+0000 7f93e918f640 1 -- 172.21.15.27:0/317374149 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40cc7f0 con 0x7f93e406a2f0 2023-08-11T15:51:32.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.128+0000 7f7401b4e640 1 Processor -- start 2023-08-11T15:51:32.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.128+0000 7f7401b4e640 1 -- start start 2023-08-11T15:51:32.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.129+0000 7f93e918f640 1 --2- 172.21.15.27:0/317374149 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e406c8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009d90 tx=0x7f93d802bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b96ceab05598bf8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.128+0000 7f2377fff640 1 -- 172.21.15.27:0/2594149509 wait complete. 2023-08-11T15:51:32.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.130+0000 7f7401b4e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc0b5390 0x7f73fc156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f2377fff640 1 Processor -- start 2023-08-11T15:51:32.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.130+0000 7f93e3fff640 1 -- 172.21.15.27:0/317374149 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d802cd30 con 0x7f93e406a2f0 2023-08-11T15:51:32.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.130+0000 7f93e3fff640 1 -- 172.21.15.27:0/317374149 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d802ce90 con 0x7f93e406a2f0 2023-08-11T15:51:32.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.130+0000 7f93e3fff640 1 -- 172.21.15.27:0/317374149 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8034660 con 0x7f93e406a2f0 2023-08-11T15:51:32.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f93ea191640 1 -- 172.21.15.27:0/317374149 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 msgr2=0x7f93e406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f93ea191640 1 --2- 172.21.15.27:0/317374149 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e406c8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009d90 tx=0x7f93d802bf80 comp rx=0 tx=0).stop 2023-08-11T15:51:32.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.130+0000 7f7401b4e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc0d1a50 0x7f73fc06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f7401b4e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f73fc15f8d0 0x7f73fc0ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f73fb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc0b5390 0x7f73fc156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f73fb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc0b5390 0x7f73fc156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46446/0 (socket says 172.21.15.27:46446) 2023-08-11T15:51:32.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f73fb7fe640 1 -- 172.21.15.27:0/3274389604 learned_addr learned my addr 172.21.15.27:0/3274389604 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0c5fa0 con 0x7f73fc0b5390 2023-08-11T15:51:32.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f73faffd640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc0d1a50 0x7f73fc06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc06d6a0 con 0x7f73fc0d1a50 2023-08-11T15:51:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc06cb20 con 0x7f73fc15f8d0 2023-08-11T15:51:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f73fbfff640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f73fc15f8d0 0x7f73fc0ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.132+0000 7f93ea191640 1 -- 172.21.15.27:0/317374149 shutdown_connections 2023-08-11T15:51:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.132+0000 7f93ea191640 1 --2- 172.21.15.27:0/317374149 >> v2:172.21.15.119:3300/0 conn(0x7f93e40c9e90 0x7f93e406f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.132+0000 7f93ea191640 1 --2- 172.21.15.27:0/317374149 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e406c8f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.132+0000 7f93ea191640 1 -- 172.21.15.27:0/317374149 >> 172.21.15.27:0/317374149 conn(0x7f93e4069800 msgr2=0x7f93e4069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.132+0000 7f73fb7fe640 1 -- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f73fc15f8d0 msgr2=0x7f73fc0ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.132+0000 7f73fb7fe640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f73fc15f8d0 0x7f73fc0ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.132+0000 7f73fb7fe640 1 -- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc0d1a50 msgr2=0x7f73fc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.132+0000 7f93ea191640 1 -- 172.21.15.27:0/317374149 shutdown_connections 2023-08-11T15:51:32.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.131+0000 7f2377fff640 1 -- start start 2023-08-11T15:51:32.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.133+0000 7f73fb7fe640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc0d1a50 0x7f73fc06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.133+0000 7f73fb7fe640 1 -- 172.21.15.27:0/3274389604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e4009400 con 0x7f73fc0b5390 2023-08-11T15:51:32.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.133+0000 7f73faffd640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc0d1a50 0x7f73fc06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.133+0000 7f2377fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23780b7bd0 0x7f2378156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.133+0000 7f2377fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f237815c350 0x7f2378158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.134+0000 7f2377fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237815c730 0x7f23780b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.134+0000 7f2376ffd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23780b7bd0 0x7f2378156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.134+0000 7f2376ffd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23780b7bd0 0x7f2378156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53670/0 (socket says 172.21.15.27:53670) 2023-08-11T15:51:32.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.134+0000 7f73fb7fe640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc0b5390 0x7f73fc156580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f73e40098c0 tx=0x7f73e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.134+0000 7f23777fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237815c730 0x7f23780b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.134+0000 7f23777fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237815c730 0x7f23780b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46452/0 (socket says 172.21.15.27:46452) 2023-08-11T15:51:32.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.135+0000 7f23767fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f237815c350 0x7f2378158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.135+0000 7f23767fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f237815c350 0x7f2378158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40872/0 (socket says 172.21.15.27:40872) 2023-08-11T15:51:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.135+0000 7f2376ffd640 1 -- 172.21.15.27:0/2129354030 learned_addr learned my addr 172.21.15.27:0/2129354030 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.134+0000 7f73f3fff640 1 -- 172.21.15.27:0/3274389604 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4032e40 con 0x7f73fc0b5390 2023-08-11T15:51:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.135+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378158f50 con 0x7f237815c730 2023-08-11T15:51:32.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.135+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23780c9340 con 0x7f237815c350 2023-08-11T15:51:32.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.135+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23780c95e0 con 0x7f23780b7bd0 2023-08-11T15:51:32.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.132+0000 7f93ea191640 1 -- 172.21.15.27:0/317374149 wait complete. 2023-08-11T15:51:32.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.135+0000 7f93ea191640 1 Processor -- start 2023-08-11T15:51:32.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.135+0000 7f73f3fff640 1 -- 172.21.15.27:0/3274389604 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e403c410 con 0x7f73fc0b5390 2023-08-11T15:51:32.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.136+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc06ce00 con 0x7f73fc0b5390 2023-08-11T15:51:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.136+0000 7f2376ffd640 1 -- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f237815c350 msgr2=0x7f2378158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.136+0000 7f2376ffd640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f237815c350 0x7f2378158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.136+0000 7f2376ffd640 1 -- 172.21.15.27:0/2129354030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237815c730 msgr2=0x7f23780b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.136+0000 7f2376ffd640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237815c730 0x7f23780b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.136+0000 7f2376ffd640 1 -- 172.21.15.27:0/2129354030 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368009400 con 0x7f23780b7bd0 2023-08-11T15:51:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.137+0000 7f2376ffd640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23780b7bd0 0x7f2378156070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c7f0 tx=0x7f236c00ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.137+0000 7f23767fc640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f237815c350 0x7f2378158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.137+0000 7f235f7fe640 1 -- 172.21.15.27:0/2129354030 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c004220 con 0x7f23780b7bd0 2023-08-11T15:51:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.137+0000 7f235f7fe640 1 -- 172.21.15.27:0/2129354030 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c004380 con 0x7f23780b7bd0 2023-08-11T15:51:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.137+0000 7f235f7fe640 1 -- 172.21.15.27:0/2129354030 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c007860 con 0x7f23780b7bd0 2023-08-11T15:51:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.136+0000 7f93ea191640 1 -- start start 2023-08-11T15:51:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.137+0000 7f93ea191640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e4201270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.138+0000 7f93ea191640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e40c9530 0x7f93e41ff530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.138+0000 7f93ea191640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e40c9e90 0x7f93e4201960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.138+0000 7f93e918f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e4201270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.138+0000 7f93e918f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e4201270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53674/0 (socket says 172.21.15.27:53674) 2023-08-11T15:51:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.139+0000 7f93e898e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e40c9530 0x7f93e41ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.139+0000 7f93e898e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e40c9530 0x7f93e41ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46460/0 (socket says 172.21.15.27:46460) 2023-08-11T15:51:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.137+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23780c98c0 con 0x7f23780b7bd0 2023-08-11T15:51:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.138+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23781591a0 con 0x7f23780b7bd0 2023-08-11T15:51:32.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.139+0000 7f93e918f640 1 -- 172.21.15.27:0/2123989151 learned_addr learned my addr 172.21.15.27:0/2123989151 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.139+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e42017b0 con 0x7f93e40c9530 2023-08-11T15:51:32.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.139+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e415bad0 con 0x7f93e40c9e90 2023-08-11T15:51:32.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.139+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40ad860 con 0x7f93e406a2f0 2023-08-11T15:51:32.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.140+0000 7f93e9990640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e40c9e90 0x7f93e4201960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.140+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc06d3c0 con 0x7f73fc0b5390 2023-08-11T15:51:32.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.140+0000 7f93e898e640 1 -- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 msgr2=0x7f93e4201270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.140+0000 7f93e898e640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e4201270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.140+0000 7f93e898e640 1 -- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e40c9e90 msgr2=0x7f93e4201960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.140+0000 7f93e898e640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e40c9e90 0x7f93e4201960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.140+0000 7f93e898e640 1 -- 172.21.15.27:0/2123989151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d8009a70 con 0x7f93e40c9530 2023-08-11T15:51:32.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.141+0000 7f93e898e640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e40c9530 0x7f93e41ff530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f93d000b3e0 tx=0x7f93d000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.141+0000 7f73f3fff640 1 -- 172.21.15.27:0/3274389604 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e403c7a0 con 0x7f73fc0b5390 2023-08-11T15:51:32.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.141+0000 7f73f3fff640 1 -- 172.21.15.27:0/3274389604 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f73e4039070 con 0x7f73fc0b5390 2023-08-11T15:51:32.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.142+0000 7f93e1ffb640 1 -- 172.21.15.27:0/2123989151 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0011830 con 0x7f93e40c9530 2023-08-11T15:51:32.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.142+0000 7f93e1ffb640 1 -- 172.21.15.27:0/2123989151 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d0011990 con 0x7f93e40c9530 2023-08-11T15:51:32.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.142+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e40adb40 con 0x7f93e40c9530 2023-08-11T15:51:32.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.142+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2358005250 con 0x7f23780b7bd0 2023-08-11T15:51:32.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.142+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e40ae100 con 0x7f93e40c9530 2023-08-11T15:51:32.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.142+0000 7f93e1ffb640 1 -- 172.21.15.27:0/2123989151 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0011d40 con 0x7f93e40c9530 2023-08-11T15:51:32.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.144+0000 7f73f3fff640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f73dc06a1a0 0x7f73dc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.144+0000 7f73f3fff640 1 -- 172.21.15.27:0/3274389604 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f73e40a9e00 con 0x7f73fc0b5390 2023-08-11T15:51:32.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.142+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e0005250 con 0x7f73fc0b5390 2023-08-11T15:51:32.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.145+0000 7f73faffd640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f73dc06a1a0 0x7f73dc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.145+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c4005250 con 0x7f93e40c9530 2023-08-11T15:51:32.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.144+0000 7f235f7fe640 1 -- 172.21.15.27:0/2129354030 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f236c0079c0 con 0x7f23780b7bd0 2023-08-11T15:51:32.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.146+0000 7f235f7fe640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f236406a050 0x7f236406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.146+0000 7f235f7fe640 1 -- 172.21.15.27:0/2129354030 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f236c088630 con 0x7f23780b7bd0 2023-08-11T15:51:32.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.147+0000 7f23767fc640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f236406a050 0x7f236406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.147+0000 7f93e1ffb640 1 -- 172.21.15.27:0/2123989151 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f93d001a400 con 0x7f93e40c9530 2023-08-11T15:51:32.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.148+0000 7f73faffd640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f73dc06a1a0 0x7f73dc06c660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0c54b0 tx=0x7f73ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.149+0000 7f23767fc640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f236406a050 0x7f236406c510 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2368034000 tx=0x7f2368008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.149+0000 7f93e1ffb640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f93c806a120 0x7f93c806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.149+0000 7f93e1ffb640 1 -- 172.21.15.27:0/2123989151 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f93d0088700 con 0x7f93e40c9530 2023-08-11T15:51:32.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.150+0000 7f93e918f640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f93c806a120 0x7f93c806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.151+0000 7f93e918f640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f93c806a120 0x7f93c806c5e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f93d8000f80 tx=0x7f93d8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.155+0000 7f73f3fff640 1 -- 172.21.15.27:0/3274389604 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f73e4075380 con 0x7f73fc0b5390 2023-08-11T15:51:32.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.164+0000 7f93e1ffb640 1 -- 172.21.15.27:0/2123989151 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f93d0053c20 con 0x7f93e40c9530 2023-08-11T15:51:32.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.162+0000 7f235f7fe640 1 -- 172.21.15.27:0/2129354030 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f236c053a40 con 0x7f23780b7bd0 2023-08-11T15:51:32.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.196+0000 7f17e15fc640 1 Processor -- start 2023-08-11T15:51:32.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.196+0000 7f17e15fc640 1 -- start start 2023-08-11T15:51:32.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.198+0000 7f17e15fc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 0x7f17dc06b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.198+0000 7f17e15fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17dc06b940 0x7f17dc0c3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.198+0000 7f17e15fc640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17dc0cd150 con 0x7f17dc0c3ed0 2023-08-11T15:51:32.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.198+0000 7f17e15fc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17dc163b90 con 0x7f17dc06b940 2023-08-11T15:51:32.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.198+0000 7f17e15fc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0c3500 con 0x7f17dc1637b0 2023-08-11T15:51:32.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.199+0000 7f17dbfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 0x7f17dc06b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.199+0000 7f17dbfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 0x7f17dc06b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53684/0 (socket says 172.21.15.27:53684) 2023-08-11T15:51:32.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.199+0000 7f17dbfff640 1 -- 172.21.15.27:0/7886607 learned_addr learned my addr 172.21.15.27:0/7886607 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.199+0000 7f17e0dfb640 1 --1- 172.21.15.27:0/7886607 >> v1:172.21.15.27:6789/0 conn(0x7f17dc0c3ed0 0x7f17dc0892f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48508/0 (socket says 172.21.15.27:48508) 2023-08-11T15:51:32.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.201+0000 7f17db7fe640 1 --2- 172.21.15.27:0/7886607 >> v2:172.21.15.119:3300/0 conn(0x7f17dc06b940 0x7f17dc0c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.201+0000 7f17daffd640 1 -- 172.21.15.27:0/7886607 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823030053 0 0) 0x7f17dc0cd150 con 0x7f17dc0c3ed0 2023-08-11T15:51:32.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.201+0000 7f17daffd640 1 -- 172.21.15.27:0/7886607 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17c4003520 con 0x7f17dc0c3ed0 2023-08-11T15:51:32.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.202+0000 7f17dbfff640 1 -- 172.21.15.27:0/7886607 >> v2:172.21.15.119:3300/0 conn(0x7f17dc06b940 msgr2=0x7f17dc0c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.202+0000 7f17dbfff640 1 --2- 172.21.15.27:0/7886607 >> v2:172.21.15.119:3300/0 conn(0x7f17dc06b940 0x7f17dc0c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.202+0000 7f17dbfff640 1 -- 172.21.15.27:0/7886607 >> v1:172.21.15.27:6789/0 conn(0x7f17dc0c3ed0 legacy=0x7f17dc0892f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.202+0000 7f17daffd640 1 -- 172.21.15.27:0/7886607 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 145512986 0 0) 0x7f17c4003520 con 0x7f17dc0c3ed0 2023-08-11T15:51:32.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.202+0000 7f17dbfff640 1 -- 172.21.15.27:0/7886607 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc16a290 con 0x7f17dc1637b0 2023-08-11T15:51:32.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.204+0000 7f17dbfff640 1 --2- 172.21.15.27:0/7886607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 0x7f17dc06b400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f17d4014e60 tx=0x7f17d40347a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=377c98e34c41587a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.205+0000 7f17daffd640 1 -- 172.21.15.27:0/7886607 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc001670 con 0x7f17dc1637b0 2023-08-11T15:51:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.205+0000 7f17daffd640 1 -- 172.21.15.27:0/7886607 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d40394b0 con 0x7f17dc1637b0 2023-08-11T15:51:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.205+0000 7f17daffd640 1 -- 172.21.15.27:0/7886607 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4039820 con 0x7f17dc1637b0 2023-08-11T15:51:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.206+0000 7f17e15fc640 1 -- 172.21.15.27:0/7886607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 msgr2=0x7f17dc06b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.206+0000 7f17e15fc640 1 --2- 172.21.15.27:0/7886607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 0x7f17dc06b400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f17d4014e60 tx=0x7f17d40347a0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.206+0000 7f17e15fc640 1 -- 172.21.15.27:0/7886607 shutdown_connections 2023-08-11T15:51:32.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.206+0000 7f17e15fc640 1 --2- 172.21.15.27:0/7886607 >> v2:172.21.15.119:3300/0 conn(0x7f17dc06b940 0x7f17dc0c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.206+0000 7f17e15fc640 1 --2- 172.21.15.27:0/7886607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 0x7f17dc06b400 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.206+0000 7f17e15fc640 1 -- 172.21.15.27:0/7886607 >> 172.21.15.27:0/7886607 conn(0x7f17dc069800 msgr2=0x7f17dc069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.206+0000 7f17e15fc640 1 -- 172.21.15.27:0/7886607 shutdown_connections 2023-08-11T15:51:32.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.206+0000 7f17e15fc640 1 -- 172.21.15.27:0/7886607 wait complete. 2023-08-11T15:51:32.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.209+0000 7f17e15fc640 1 Processor -- start 2023-08-11T15:51:32.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.209+0000 7f17e15fc640 1 -- start start 2023-08-11T15:51:32.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.211+0000 7f17e15fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17dc06b940 0x7f17dc15de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.211+0000 7f17e15fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc0c3ed0 0x7f17dc0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.211+0000 7f17e15fc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 0x7f17dc0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.211+0000 7f17e15fc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0b1f50 con 0x7f17dc06b940 2023-08-11T15:51:32.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.211+0000 7f17e15fc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0b2370 con 0x7f17dc0c3ed0 2023-08-11T15:51:32.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.211+0000 7f17e15fc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc15e3b0 con 0x7f17dc1637b0 2023-08-11T15:51:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.212+0000 7f17dbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17dc06b940 0x7f17dc15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.212+0000 7f17dbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17dc06b940 0x7f17dc15de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46462/0 (socket says 172.21.15.27:46462) 2023-08-11T15:51:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.212+0000 7f17dbfff640 1 -- 172.21.15.27:0/1928378884 learned_addr learned my addr 172.21.15.27:0/1928378884 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.212+0000 7f17e0dfb640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 0x7f17dc0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.212+0000 7f1e29555640 1 Processor -- start 2023-08-11T15:51:32.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.212+0000 7f17db7fe640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc0c3ed0 0x7f17dc0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.213+0000 7f17dbfff640 1 -- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 msgr2=0x7f17dc0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.213+0000 7f17dbfff640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 0x7f17dc0b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.213+0000 7f17dbfff640 1 -- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc0c3ed0 msgr2=0x7f17dc0b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.213+0000 7f17dbfff640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc0c3ed0 0x7f17dc0b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.213+0000 7f17dbfff640 1 -- 172.21.15.27:0/1928378884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d4014ac0 con 0x7f17dc06b940 2023-08-11T15:51:32.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.214+0000 7f17dbfff640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17dc06b940 0x7f17dc15de70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f17d4014f80 tx=0x7f17d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.213+0000 7f1e29555640 1 -- start start 2023-08-11T15:51:32.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f1e29555640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e1c0df530 0x7f1e1c0075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f1e29555640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e1c006010 0x7f1e1c054290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f1e29555640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c006510 con 0x7f1e1c04fca0 2023-08-11T15:51:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f1e29555640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c0df910 con 0x7f1e1c006010 2023-08-11T15:51:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f1e29555640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c0067f0 con 0x7f1e1c0df530 2023-08-11T15:51:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f1e237fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1e1c04fca0 0x7f1e1c0500b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48512/0 (socket says 172.21.15.27:48512) 2023-08-11T15:51:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f1e237fe640 1 -- 172.21.15.27:0/2260835134 learned_addr learned my addr 172.21.15.27:0/2260835134 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.216+0000 7f1e227fc640 1 --2- 172.21.15.27:0/2260835134 >> v2:172.21.15.27:3300/0 conn(0x7f1e1c006010 0x7f1e1c054290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.217+0000 7f1e227fc640 1 -- 172.21.15.27:0/2260835134 >> v2:172.21.15.119:3300/0 conn(0x7f1e1c0df530 msgr2=0x7f1e1c0075b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:51:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.217+0000 7f1e227fc640 1 --2- 172.21.15.27:0/2260835134 >> v2:172.21.15.119:3300/0 conn(0x7f1e1c0df530 0x7f1e1c0075b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f17d8ff9640 1 -- 172.21.15.27:0/1928378884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4039820 con 0x7f17dc06b940 2023-08-11T15:51:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f17d8ff9640 1 -- 172.21.15.27:0/1928378884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d4039980 con 0x7f17dc06b940 2023-08-11T15:51:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f17d8ff9640 1 -- 172.21.15.27:0/1928378884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4039d10 con 0x7f17dc06b940 2023-08-11T15:51:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f17e15fc640 1 -- 172.21.15.27:0/1928378884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc0b12d0 con 0x7f17dc06b940 2023-08-11T15:51:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.215+0000 7f17e15fc640 1 -- 172.21.15.27:0/1928378884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc0b1890 con 0x7f17dc06b940 2023-08-11T15:51:32.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.218+0000 7f1e227fc640 1 -- 172.21.15.27:0/2260835134 >> v1:172.21.15.27:6789/0 conn(0x7f1e1c04fca0 legacy=0x7f1e1c0500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.218+0000 7f1e227fc640 1 -- 172.21.15.27:0/2260835134 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c006ad0 con 0x7f1e1c006010 2023-08-11T15:51:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.220+0000 7f1e227fc640 1 --2- 172.21.15.27:0/2260835134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e1c006010 0x7f1e1c054290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c04fb60 tx=0x7f1e1402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edea9ae1d321bbb server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.221+0000 7f1e21ffb640 1 -- 172.21.15.27:0/2260835134 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14030650 con 0x7f1e1c006010 2023-08-11T15:51:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.221+0000 7f1e21ffb640 1 -- 172.21.15.27:0/2260835134 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e140307b0 con 0x7f1e1c006010 2023-08-11T15:51:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.222+0000 7f1e29555640 1 -- 172.21.15.27:0/2260835134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e1c006010 msgr2=0x7f1e1c054290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.222+0000 7f1e29555640 1 --2- 172.21.15.27:0/2260835134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e1c006010 0x7f1e1c054290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c04fb60 tx=0x7f1e1402ba20 comp rx=0 tx=0).stop 2023-08-11T15:51:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.223+0000 7f1e29555640 1 -- 172.21.15.27:0/2260835134 shutdown_connections 2023-08-11T15:51:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.223+0000 7f1e29555640 1 --2- 172.21.15.27:0/2260835134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e1c006010 0x7f1e1c054290 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.223+0000 7f1e29555640 1 --2- 172.21.15.27:0/2260835134 >> v2:172.21.15.119:3300/0 conn(0x7f1e1c0df530 0x7f1e1c0075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.223+0000 7f1e29555640 1 -- 172.21.15.27:0/2260835134 >> 172.21.15.27:0/2260835134 conn(0x7f1e1c054c40 msgr2=0x7f1e1c055050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.221+0000 7f17d8ff9640 1 -- 172.21.15.27:0/1928378884 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f17d403c440 con 0x7f17dc06b940 2023-08-11T15:51:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.223+0000 7f17d8ff9640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f17c806a120 0x7f17c806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.223+0000 7f17d8ff9640 1 -- 172.21.15.27:0/1928378884 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f17d40b3db0 con 0x7f17dc06b940 2023-08-11T15:51:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.224+0000 7f17db7fe640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f17c806a120 0x7f17c806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.224+0000 7f1e29555640 1 -- 172.21.15.27:0/2260835134 shutdown_connections 2023-08-11T15:51:32.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.224+0000 7f1e29555640 1 -- 172.21.15.27:0/2260835134 wait complete. 2023-08-11T15:51:32.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.224+0000 7f17e15fc640 1 -- 172.21.15.27:0/1928378884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17bc005250 con 0x7f17dc06b940 2023-08-11T15:51:32.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.226+0000 7f17db7fe640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f17c806a120 0x7f17c806c5e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0ca440 tx=0x7f17d000f040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.227+0000 7f1e29555640 1 Processor -- start 2023-08-11T15:51:32.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.227+0000 7f1e29555640 1 -- start start 2023-08-11T15:51:32.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.228+0000 7f1e29555640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e1c04fca0 0x7f1e1c0d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.228+0000 7f1e29555640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e1c0df530 0x7f1e1c0e2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.228+0000 7f1e29555640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e1c0e26a0 0x7f1e1c0de330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.229+0000 7f1e29555640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0e1140 con 0x7f1e1c04fca0 2023-08-11T15:51:32.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.229+0000 7f1e29555640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0d5de0 con 0x7f1e1c0df530 2023-08-11T15:51:32.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.229+0000 7f1e29555640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0d6030 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.229+0000 7f1e22ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e1c04fca0 0x7f1e1c0d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.229+0000 7f1e227fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e1c0df530 0x7f1e1c0e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.229+0000 7f1e227fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e1c0df530 0x7f1e1c0e2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40902/0 (socket says 172.21.15.27:40902) 2023-08-11T15:51:32.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.229+0000 7f1e227fc640 1 -- 172.21.15.27:0/2887743768 learned_addr learned my addr 172.21.15.27:0/2887743768 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.229+0000 7f1e237fe640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e1c0e26a0 0x7f1e1c0de330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.230+0000 7f1e237fe640 1 -- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e1c0df530 msgr2=0x7f1e1c0e2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.230+0000 7f1e237fe640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e1c0df530 0x7f1e1c0e2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.230+0000 7f1e237fe640 1 -- 172.21.15.27:0/2887743768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e1c04fca0 msgr2=0x7f1e1c0d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.231+0000 7f1e237fe640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e1c04fca0 0x7f1e1c0d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.231+0000 7f1e237fe640 1 -- 172.21.15.27:0/2887743768 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e14009400 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.231+0000 7f1e22ffd640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e1c04fca0 0x7f1e1c0d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:32.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.231+0000 7f1e237fe640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e1c0e26a0 0x7f1e1c0de330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1e24063520 tx=0x7f1e2406cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.232+0000 7f1e0b7fe640 1 -- 172.21.15.27:0/2887743768 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e24066cc0 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.232+0000 7f1e0b7fe640 1 -- 172.21.15.27:0/2887743768 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2406ab90 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.232+0000 7f1e29555640 1 -- 172.21.15.27:0/2887743768 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c0d6310 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.232+0000 7f1e0b7fe640 1 -- 172.21.15.27:0/2887743768 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e24072c60 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.232+0000 7f1e29555640 1 -- 172.21.15.27:0/2887743768 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c0d68d0 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.233+0000 7f1e29555640 1 -- 172.21.15.27:0/2887743768 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1c050da0 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.234+0000 7f17d8ff9640 1 -- 172.21.15.27:0/1928378884 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f17d407f360 con 0x7f17dc06b940 2023-08-11T15:51:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.236+0000 7f1e0b7fe640 1 -- 172.21.15.27:0/2887743768 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e24072dc0 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.237+0000 7f1e0b7fe640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e1006a050 0x7f1e1006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.237+0000 7f1e0b7fe640 1 -- 172.21.15.27:0/2887743768 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1e24101790 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.238+0000 7f1e22ffd640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e1006a050 0x7f1e1006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.240+0000 7f1e22ffd640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e1006a050 0x7f1e1006c510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c04bf00 tx=0x7f1e18009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.248+0000 7f1e0b7fe640 1 -- 172.21.15.27:0/2887743768 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f1e240cccb0 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.250+0000 7fe4472de640 1 Processor -- start 2023-08-11T15:51:32.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.250+0000 7fe4472de640 1 -- start start 2023-08-11T15:51:32.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.252+0000 7fe4472de640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4380de3c0 0x7fe438006150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.252+0000 7fe4472de640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4380eb040 0x7fe4380eb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.252+0000 7fe4472de640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe438057010 con 0x7fe438006690 2023-08-11T15:51:32.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.252+0000 7fe4472de640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4380eba10 con 0x7fe4380de3c0 2023-08-11T15:51:32.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.252+0000 7fe4472de640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe438058e80 con 0x7fe4380eb040 2023-08-11T15:51:32.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.253+0000 7fe444879640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe438006690 0x7fe4380380d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48528/0 (socket says 172.21.15.27:48528) 2023-08-11T15:51:32.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.253+0000 7fe444879640 1 -- 172.21.15.27:0/1964928271 learned_addr learned my addr 172.21.15.27:0/1964928271 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:32.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.254+0000 7fe44507a640 1 --2- 172.21.15.27:0/1964928271 >> v2:172.21.15.27:3300/0 conn(0x7fe4380de3c0 0x7fe438006150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.254+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100809866 0 0) 0x7fe438057010 con 0x7fe438006690 2023-08-11T15:51:32.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.254+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe42c003520 con 0x7fe438006690 2023-08-11T15:51:32.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.254+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3776854250 0 0) 0x7fe42c003520 con 0x7fe438006690 2023-08-11T15:51:32.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 >> v2:172.21.15.119:3300/0 conn(0x7fe4380eb040 msgr2=0x7fe4380eb4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:51:32.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 --2- 172.21.15.27:0/1964928271 >> v2:172.21.15.119:3300/0 conn(0x7fe4380eb040 0x7fe4380eb4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 >> v2:172.21.15.27:3300/0 conn(0x7fe4380de3c0 msgr2=0x7fe438006150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 --2- 172.21.15.27:0/1964928271 >> v2:172.21.15.27:3300/0 conn(0x7fe4380de3c0 0x7fe438006150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe438056ce0 con 0x7fe438006690 2023-08-11T15:51:32.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fe440073490 con 0x7fe438006690 2023-08-11T15:51:32.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 >> v1:172.21.15.27:6789/0 conn(0x7fe438006690 legacy=0x7fe4380380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe42c0039a0 0x7fe42c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 0x7fe42c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe42c005180 0x7fe42c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380eba10 con 0x7fe42c0047c0 2023-08-11T15:51:32.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe438058e80 con 0x7fe42c005180 2023-08-11T15:51:32.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c00c100 con 0x7fe42c0039a0 2023-08-11T15:51:32.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.255+0000 7fe44507a640 1 --2- 172.21.15.27:0/1964928271 >> v2:172.21.15.27:3300/0 conn(0x7fe4380de3c0 0x7fe438006150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:32.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.256+0000 7fe444879640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 0x7fe42c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.256+0000 7fe44507a640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe42c0039a0 0x7fe42c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.256+0000 7fe44587b640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe42c005180 0x7fe42c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.257+0000 7f298e9d5640 1 Processor -- start 2023-08-11T15:51:32.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.257+0000 7fe444879640 1 -- 172.21.15.27:0/1964928271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe42c0039a0 msgr2=0x7fe42c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.257+0000 7fe444879640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe42c0039a0 0x7fe42c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.257+0000 7fe444879640 1 -- 172.21.15.27:0/1964928271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe42c005180 msgr2=0x7fe42c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.257+0000 7fe444879640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe42c005180 0x7fe42c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.257+0000 7fe444879640 1 -- 172.21.15.27:0/1964928271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c003520 con 0x7fe42c0047c0 2023-08-11T15:51:32.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.257+0000 7fe44587b640 1 -- 172.21.15.27:0/1964928271 reap_dead start 2023-08-11T15:51:32.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.258+0000 7f298e9d5640 1 -- start start 2023-08-11T15:51:32.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.260+0000 7f298e9d5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29800503b0 0x7f29800075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.261+0000 7f298e9d5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2980006510 con 0x7f2980006010 2023-08-11T15:51:32.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.261+0000 7f298e9d5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29800067f0 con 0x7f29800df530 2023-08-11T15:51:32.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.261+0000 7f298e9d5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29800df910 con 0x7f29800503b0 2023-08-11T15:51:32.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.261+0000 7f29877fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29800503b0 0x7f29800075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.262+0000 7f29877fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29800503b0 0x7f29800075b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46490/0 (socket says 172.21.15.27:46490) 2023-08-11T15:51:32.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.262+0000 7f29877fe640 1 -- 172.21.15.27:0/3013739234 learned_addr learned my addr 172.21.15.27:0/3013739234 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.262+0000 7fe444879640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 0x7fe42c004c40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe440067a30 tx=0x7fe42c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1bc929abfc7aee server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.262+0000 7f29877fe640 1 -- 172.21.15.27:0/3013739234 >> v1:172.21.15.119:6789/0 conn(0x7f29800df530 legacy=0x7f298004fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.262+0000 7f29877fe640 1 -- 172.21.15.27:0/3013739234 >> v1:172.21.15.27:6789/0 conn(0x7f2980006010 legacy=0x7f2980038200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.262+0000 7f29877fe640 1 -- 172.21.15.27:0/3013739234 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980006ad0 con 0x7f29800503b0 2023-08-11T15:51:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.263+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44006a980 con 0x7fe42c0047c0 2023-08-11T15:51:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.263+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44006aae0 con 0x7fe42c0047c0 2023-08-11T15:51:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.263+0000 7fe437fff640 1 -- 172.21.15.27:0/1964928271 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44006c070 con 0x7fe42c0047c0 2023-08-11T15:51:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.265+0000 7fe4472de640 1 -- 172.21.15.27:0/1964928271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 msgr2=0x7fe42c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.265+0000 7fe4472de640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 0x7fe42c004c40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe440067a30 tx=0x7fe42c0057f0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.266+0000 7fe4472de640 1 -- 172.21.15.27:0/1964928271 shutdown_connections 2023-08-11T15:51:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.266+0000 7fe4472de640 1 --2- 172.21.15.27:0/1964928271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 0x7fe42c004c40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.266+0000 7fe4472de640 1 -- 172.21.15.27:0/1964928271 >> 172.21.15.27:0/1964928271 conn(0x7fe438055a80 msgr2=0x7fe438055e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.266+0000 7f29877fe640 1 --2- 172.21.15.27:0/3013739234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29800503b0 0x7f29800075b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f297c00a910 tx=0x7f297c0301a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8a5e8f4e669929 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.266+0000 7fe4472de640 1 -- 172.21.15.27:0/1964928271 shutdown_connections 2023-08-11T15:51:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.266+0000 7f2986ffd640 1 -- 172.21.15.27:0/3013739234 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c03a400 con 0x7f29800503b0 2023-08-11T15:51:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.267+0000 7f2986ffd640 1 -- 172.21.15.27:0/3013739234 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c034070 con 0x7f29800503b0 2023-08-11T15:51:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.267+0000 7f2986ffd640 1 -- 172.21.15.27:0/3013739234 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c03a780 con 0x7f29800503b0 2023-08-11T15:51:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.267+0000 7fe4472de640 1 -- 172.21.15.27:0/1964928271 wait complete. 2023-08-11T15:51:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.268+0000 7f298e9d5640 1 -- 172.21.15.27:0/3013739234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29800503b0 msgr2=0x7f29800075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.268+0000 7f298e9d5640 1 --2- 172.21.15.27:0/3013739234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29800503b0 0x7f29800075b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f297c00a910 tx=0x7f297c0301a0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.268+0000 7f298e9d5640 1 -- 172.21.15.27:0/3013739234 shutdown_connections 2023-08-11T15:51:32.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.268+0000 7f298e9d5640 1 --2- 172.21.15.27:0/3013739234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29800503b0 0x7f29800075b0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f297c00a910 tx=0x7f297c0301a0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.268+0000 7f298e9d5640 1 -- 172.21.15.27:0/3013739234 >> 172.21.15.27:0/3013739234 conn(0x7f2980054c40 msgr2=0x7f2980055050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.269+0000 7f298e9d5640 1 -- 172.21.15.27:0/3013739234 shutdown_connections 2023-08-11T15:51:32.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.270+0000 7fe4472de640 1 Processor -- start 2023-08-11T15:51:32.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.270+0000 7f298e9d5640 1 -- 172.21.15.27:0/3013739234 wait complete. 2023-08-11T15:51:32.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.271+0000 7fe4472de640 1 -- start start 2023-08-11T15:51:32.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.272+0000 7fe4472de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 0x7fe4380cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.273+0000 7fe4472de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380de7a0 0x7fe4380ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.273+0000 7f298e9d5640 1 Processor -- start 2023-08-11T15:51:32.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.273+0000 7fe4472de640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe438047bf0 0x7fe438048070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.273+0000 7fe4472de640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380dfc00 con 0x7fe42c0047c0 2023-08-11T15:51:32.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.273+0000 7fe4472de640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380deff0 con 0x7fe4380de7a0 2023-08-11T15:51:32.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.273+0000 7fe4472de640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380df2d0 con 0x7fe438047bf0 2023-08-11T15:51:32.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.273+0000 7fe44507a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 0x7fe4380cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.273+0000 7fe44507a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 0x7fe4380cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46502/0 (socket says 172.21.15.27:46502) 2023-08-11T15:51:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.274+0000 7fe44507a640 1 -- 172.21.15.27:0/3694981967 learned_addr learned my addr 172.21.15.27:0/3694981967 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.274+0000 7fe444879640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380de7a0 0x7fe4380ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.274+0000 7fe44587b640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe438047bf0 0x7fe438048070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.274+0000 7f298e9d5640 1 -- start start 2023-08-11T15:51:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.275+0000 7fe444879640 1 -- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe438047bf0 msgr2=0x7fe438048070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.275+0000 7fe444879640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe438047bf0 0x7fe438048070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.275+0000 7fe444879640 1 -- 172.21.15.27:0/3694981967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 msgr2=0x7fe4380cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.275+0000 7fe444879640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 0x7fe4380cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.275+0000 7fe444879640 1 -- 172.21.15.27:0/3694981967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4380df5b0 con 0x7fe4380de7a0 2023-08-11T15:51:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.275+0000 7fe444879640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380de7a0 0x7fe4380ce750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe4400735f0 tx=0x7fe438056e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7f298e9d5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2980006010 0x7f2980186250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7f298e9d5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29800df530 0x7f2980186c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7fe435ffb640 1 -- 172.21.15.27:0/3694981967 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44006c800 con 0x7fe4380de7a0 2023-08-11T15:51:32.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7fe435ffb640 1 -- 172.21.15.27:0/3694981967 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4400be400 con 0x7fe4380de7a0 2023-08-11T15:51:32.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7fe435ffb640 1 -- 172.21.15.27:0/3694981967 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4400be570 con 0x7fe4380de7a0 2023-08-11T15:51:32.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7fe4472de640 1 -- 172.21.15.27:0/3694981967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4380df890 con 0x7fe4380de7a0 2023-08-11T15:51:32.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7f298e9d5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2980187190 0x7f29801885d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7fe4472de640 1 -- 172.21.15.27:0/3694981967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4380cec90 con 0x7fe4380de7a0 2023-08-11T15:51:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7f2987fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2980006010 0x7f2980186250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7f2987fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2980006010 0x7f2980186250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:53746/0 (socket says 172.21.15.27:53746) 2023-08-11T15:51:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.276+0000 7f29877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29800df530 0x7f2980186c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.277+0000 7f29877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29800df530 0x7f2980186c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40918/0 (socket says 172.21.15.27:40918) 2023-08-11T15:51:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.277+0000 7f298cf72640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2980187190 0x7f29801885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.277+0000 7f298cf72640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2980187190 0x7f29801885d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46506/0 (socket says 172.21.15.27:46506) 2023-08-11T15:51:32.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.277+0000 7f2987fff640 1 -- 172.21.15.27:0/3402264365 learned_addr learned my addr 172.21.15.27:0/3402264365 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.277+0000 7f298e9d5640 1 -- 172.21.15.27:0/3402264365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29800e0ad0 con 0x7f2980187190 2023-08-11T15:51:32.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.277+0000 7f298e9d5640 1 -- 172.21.15.27:0/3402264365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29800e0d70 con 0x7f29800df530 2023-08-11T15:51:32.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.277+0000 7f298e9d5640 1 -- 172.21.15.27:0/3402264365 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800e1050 con 0x7f2980006010 2023-08-11T15:51:32.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.278+0000 7fe4472de640 1 -- 172.21.15.27:0/3694981967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe420005250 con 0x7fe4380de7a0 2023-08-11T15:51:32.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.278+0000 7f29877fe640 1 -- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2980006010 msgr2=0x7f2980186250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.278+0000 7f29877fe640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2980006010 0x7f2980186250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.278+0000 7f29877fe640 1 -- 172.21.15.27:0/3402264365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2980187190 msgr2=0x7f29801885d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.278+0000 7f29877fe640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2980187190 0x7f29801885d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.278+0000 7f29877fe640 1 -- 172.21.15.27:0/3402264365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c00a570 con 0x7f29800df530 2023-08-11T15:51:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.278+0000 7f298cf72640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2980187190 0x7f29801885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.279+0000 7f29877fe640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29800df530 0x7f2980186c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f297c038000 tx=0x7f297c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.281+0000 7fe435ffb640 1 -- 172.21.15.27:0/3694981967 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe4400b2070 con 0x7fe4380de7a0 2023-08-11T15:51:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.283+0000 7fe435ffb640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42406a0d0 0x7fe42406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.283+0000 7fe435ffb640 1 -- 172.21.15.27:0/3694981967 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe440123d20 con 0x7fe4380de7a0 2023-08-11T15:51:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.284+0000 7fe44507a640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42406a0d0 0x7fe42406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.285+0000 7fe44507a640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42406a0d0 0x7fe42406c590 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00ac10 tx=0x7fe43c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.291+0000 7f2984ff9640 1 -- 172.21.15.27:0/3402264365 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c004220 con 0x7f29800df530 2023-08-11T15:51:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.291+0000 7f2984ff9640 1 -- 172.21.15.27:0/3402264365 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c00f070 con 0x7f29800df530 2023-08-11T15:51:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.291+0000 7f2984ff9640 1 -- 172.21.15.27:0/3402264365 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c003610 con 0x7f29800df530 2023-08-11T15:51:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.291+0000 7f298e9d5640 1 -- 172.21.15.27:0/3402264365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29800e1330 con 0x7f29800df530 2023-08-11T15:51:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.291+0000 7f298e9d5640 1 -- 172.21.15.27:0/3402264365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2980186790 con 0x7f29800df530 2023-08-11T15:51:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.291+0000 7fe435ffb640 1 -- 172.21.15.27:0/3694981967 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe4400ef240 con 0x7fe4380de7a0 2023-08-11T15:51:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.293+0000 7f29727fc640 1 -- 172.21.15.27:0/3402264365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2960005250 con 0x7f29800df530 2023-08-11T15:51:32.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.297+0000 7f2984ff9640 1 -- 172.21.15.27:0/3402264365 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f297c030d50 con 0x7f29800df530 2023-08-11T15:51:32.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.298+0000 7f2984ff9640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f296c06a120 0x7f296c06c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.299+0000 7f2984ff9640 1 -- 172.21.15.27:0/3402264365 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f297c0aa1b0 con 0x7f29800df530 2023-08-11T15:51:32.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.300+0000 7f2987fff640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f296c06a120 0x7f296c06c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.301+0000 7f2987fff640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f296c06a120 0x7f296c06c5e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2988078c80 tx=0x7f2988068830 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.308+0000 7f2984ff9640 1 -- 172.21.15.27:0/3402264365 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f297c075760 con 0x7f29800df530 2023-08-11T15:51:32.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.507+0000 7f7b71071640 1 -- 172.21.15.27:0/2994846887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7b4c005500 con 0x7f7b6c0c9030 2023-08-11T15:51:32.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.508+0000 7f7b537fe640 1 -- 172.21.15.27:0/2994846887 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b5c0570b0 con 0x7f7b6c0c9030 2023-08-11T15:51:32.513 INFO:teuthology.orchestra.run.smithi027.stdout:60129542157 2023-08-11T15:51:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.512+0000 7f7b517fa640 1 -- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7b5806a050 msgr2=0x7f7b5806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.512+0000 7f7b517fa640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7b5806a050 0x7f7b5806c510 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b64035000 tx=0x7f7b64030000 comp rx=0 tx=0).stop 2023-08-11T15:51:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.513+0000 7f7b517fa640 1 -- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0c9030 msgr2=0x7f7b6c0b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.513+0000 7f7b517fa640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0c9030 0x7f7b6c0b0bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c430 tx=0x7f7b5c00c8f0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.513+0000 7f7b517fa640 1 -- 172.21.15.27:0/2994846887 shutdown_connections 2023-08-11T15:51:32.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.513+0000 7f7b517fa640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7b5806a050 0x7f7b5806c510 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.513+0000 7f7b517fa640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cd170 0x7f7b6c0c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.514+0000 7f7b517fa640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b6c0ceac0 0x7f7b6c0c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.514+0000 7f7b517fa640 1 --2- 172.21.15.27:0/2994846887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0c9030 0x7f7b6c0b0bd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.514+0000 7f7b517fa640 1 -- 172.21.15.27:0/2994846887 >> 172.21.15.27:0/2994846887 conn(0x7f7b6c0cdfd0 msgr2=0x7f7b6c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.514+0000 7f7b517fa640 1 -- 172.21.15.27:0/2994846887 shutdown_connections 2023-08-11T15:51:32.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.514+0000 7f7b517fa640 1 -- 172.21.15.27:0/2994846887 wait complete. 2023-08-11T15:51:32.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542157 got 60129542157 for osd.0 2023-08-11T15:51:32.543 DEBUG:teuthology.parallel:result is None 2023-08-11T15:51:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.556+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f93c4005500 con 0x7f93e40c9530 2023-08-11T15:51:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.557+0000 7f93e1ffb640 1 -- 172.21.15.27:0/2123989151 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93d0057240 con 0x7f93e40c9530 2023-08-11T15:51:32.559 INFO:teuthology.orchestra.run.smithi027.stdout:60129542155 2023-08-11T15:51:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.564+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f73e00057d0 con 0x7f73fc0b5390 2023-08-11T15:51:32.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.565+0000 7f73f3fff640 1 -- 172.21.15.27:0/3274389604 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f73e4037070 con 0x7f73fc0b5390 2023-08-11T15:51:32.566 INFO:teuthology.orchestra.run.smithi027.stdout:60129542155 2023-08-11T15:51:32.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.565+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f93c806a120 msgr2=0x7f93c806c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.565+0000 7f93ea191640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f93c806a120 0x7f93c806c5e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f93d8000f80 tx=0x7f93d8030000 comp rx=0 tx=0).stop 2023-08-11T15:51:32.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.565+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e40c9530 msgr2=0x7f93e41ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.565+0000 7f93ea191640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e40c9530 0x7f93e41ff530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f93d000b3e0 tx=0x7f93d000b8a0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.566+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 shutdown_connections 2023-08-11T15:51:32.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.566+0000 7f93ea191640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f93c806a120 0x7f93c806c5e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.566+0000 7f93ea191640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e40c9e90 0x7f93e4201960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.566+0000 7f93ea191640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e40c9530 0x7f93e41ff530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.566+0000 7f93ea191640 1 --2- 172.21.15.27:0/2123989151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93e406a2f0 0x7f93e4201270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.566+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 >> 172.21.15.27:0/2123989151 conn(0x7f93e4069800 msgr2=0x7f93e40c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.566+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 shutdown_connections 2023-08-11T15:51:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.566+0000 7f93ea191640 1 -- 172.21.15.27:0/2123989151 wait complete. 2023-08-11T15:51:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.569+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f73dc06a1a0 msgr2=0x7f73dc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.569+0000 7f7401b4e640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f73dc06a1a0 0x7f73dc06c660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0c54b0 tx=0x7f73ec009040 comp rx=0 tx=0).stop 2023-08-11T15:51:32.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.569+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc0b5390 msgr2=0x7f73fc156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.569+0000 7f7401b4e640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc0b5390 0x7f73fc156580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f73e40098c0 tx=0x7f73e40046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.570+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 shutdown_connections 2023-08-11T15:51:32.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.570+0000 7f7401b4e640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f73dc06a1a0 0x7f73dc06c660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.570+0000 7f7401b4e640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f73fc15f8d0 0x7f73fc0ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.570+0000 7f7401b4e640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73fc0d1a50 0x7f73fc06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.571+0000 7f7401b4e640 1 --2- 172.21.15.27:0/3274389604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73fc0b5390 0x7f73fc156580 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.571+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 >> 172.21.15.27:0/3274389604 conn(0x7f73fc06a250 msgr2=0x7f73fc0b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.571+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 shutdown_connections 2023-08-11T15:51:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.571+0000 7f7401b4e640 1 -- 172.21.15.27:0/3274389604 wait complete. 2023-08-11T15:51:32.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.586+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2358005500 con 0x7f23780b7bd0 2023-08-11T15:51:32.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.587+0000 7f235f7fe640 1 -- 172.21.15.27:0/2129354030 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f236c057060 con 0x7f23780b7bd0 2023-08-11T15:51:32.589 INFO:teuthology.orchestra.run.smithi027.stdout:60129542156 2023-08-11T15:51:32.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.591+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f236406a050 msgr2=0x7f236406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.591+0000 7f2377fff640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f236406a050 0x7f236406c510 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2368034000 tx=0x7f2368008030 comp rx=0 tx=0).stop 2023-08-11T15:51:32.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.591+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23780b7bd0 msgr2=0x7f2378156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.591+0000 7f2377fff640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23780b7bd0 0x7f2378156070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c7f0 tx=0x7f236c00ccb0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.592+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 shutdown_connections 2023-08-11T15:51:32.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.592+0000 7f2377fff640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f236406a050 0x7f236406c510 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.592+0000 7f2377fff640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237815c730 0x7f23780b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.592+0000 7f2377fff640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f237815c350 0x7f2378158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.593+0000 7f2377fff640 1 --2- 172.21.15.27:0/2129354030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f23780b7bd0 0x7f2378156070 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.593+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 >> 172.21.15.27:0/2129354030 conn(0x7f237806a250 msgr2=0x7f23780c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.593+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 shutdown_connections 2023-08-11T15:51:32.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.593+0000 7f2377fff640 1 -- 172.21.15.27:0/2129354030 wait complete. 2023-08-11T15:51:32.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.2 2023-08-11T15:51:32.597 DEBUG:teuthology.parallel:result is None 2023-08-11T15:51:32.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.1 2023-08-11T15:51:32.601 DEBUG:teuthology.parallel:result is None 2023-08-11T15:51:32.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.3 2023-08-11T15:51:32.622 DEBUG:teuthology.parallel:result is None 2023-08-11T15:51:32.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.624+0000 7f1e29555640 1 -- 172.21.15.27:0/2887743768 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1e1c008b50 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.626+0000 7f1e0b7fe640 1 -- 172.21.15.27:0/2887743768 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1e24098030 con 0x7f1e1c0e26a0 2023-08-11T15:51:32.630 INFO:teuthology.orchestra.run.smithi027.stdout:60129542157 2023-08-11T15:51:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.628+0000 7f1e29555640 1 -- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e1006a050 msgr2=0x7f1e1006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.628+0000 7f1e29555640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e1006a050 0x7f1e1006c510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c04bf00 tx=0x7f1e18009060 comp rx=0 tx=0).stop 2023-08-11T15:51:32.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.628+0000 7f1e29555640 1 -- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e1c0e26a0 msgr2=0x7f1e1c0de330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.628+0000 7f1e29555640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e1c0e26a0 0x7f1e1c0de330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1e24063520 tx=0x7f1e2406cd70 comp rx=0 tx=0).stop 2023-08-11T15:51:32.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.628+0000 7f1e29555640 1 -- 172.21.15.27:0/2887743768 shutdown_connections 2023-08-11T15:51:32.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.628+0000 7f1e29555640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e1006a050 0x7f1e1006c510 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.628+0000 7f1e29555640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e1c0e26a0 0x7f1e1c0de330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.628+0000 7f1e29555640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e1c0df530 0x7f1e1c0e2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.628+0000 7f1e29555640 1 --2- 172.21.15.27:0/2887743768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e1c04fca0 0x7f1e1c0d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.628+0000 7f1e29555640 1 -- 172.21.15.27:0/2887743768 >> 172.21.15.27:0/2887743768 conn(0x7f1e1c054c40 msgr2=0x7f1e1c0dd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.629+0000 7f1e29555640 1 -- 172.21.15.27:0/2887743768 shutdown_connections 2023-08-11T15:51:32.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.629+0000 7f1e29555640 1 -- 172.21.15.27:0/2887743768 wait complete. 2023-08-11T15:51:32.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.632+0000 7f29727fc640 1 -- 172.21.15.27:0/3402264365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f29600057d0 con 0x7f29800df530 2023-08-11T15:51:32.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.633+0000 7f2984ff9640 1 -- 172.21.15.27:0/3402264365 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f297c078d80 con 0x7f29800df530 2023-08-11T15:51:32.635 INFO:teuthology.orchestra.run.smithi027.stdout:60129542155 2023-08-11T15:51:32.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.636+0000 7f29727fc640 1 -- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f296c06a120 msgr2=0x7f296c06c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.636+0000 7f29727fc640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f296c06a120 0x7f296c06c5e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2988078c80 tx=0x7f2988068830 comp rx=0 tx=0).stop 2023-08-11T15:51:32.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.636+0000 7f29727fc640 1 -- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29800df530 msgr2=0x7f2980186c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.636+0000 7f29727fc640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29800df530 0x7f2980186c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f297c038000 tx=0x7f297c004730 comp rx=0 tx=0).stop 2023-08-11T15:51:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.636+0000 7f29727fc640 1 -- 172.21.15.27:0/3402264365 shutdown_connections 2023-08-11T15:51:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.636+0000 7f29727fc640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f296c06a120 0x7f296c06c5e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.636+0000 7f29727fc640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2980187190 0x7f29801885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.636+0000 7f29727fc640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29800df530 0x7f2980186c50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.636+0000 7f29727fc640 1 --2- 172.21.15.27:0/3402264365 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2980006010 0x7f2980186250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.636+0000 7f29727fc640 1 -- 172.21.15.27:0/3402264365 >> 172.21.15.27:0/3402264365 conn(0x7f2980054c40 msgr2=0x7f29800dcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.637+0000 7f29727fc640 1 -- 172.21.15.27:0/3402264365 shutdown_connections 2023-08-11T15:51:32.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.637+0000 7f29727fc640 1 -- 172.21.15.27:0/3402264365 wait complete. 2023-08-11T15:51:32.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.640+0000 7f17e15fc640 1 -- 172.21.15.27:0/1928378884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f17bc005500 con 0x7f17dc06b940 2023-08-11T15:51:32.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.641+0000 7f17d8ff9640 1 -- 172.21.15.27:0/1928378884 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17d4056020 con 0x7f17dc06b940 2023-08-11T15:51:32.643 INFO:teuthology.orchestra.run.smithi027.stdout:60129542156 2023-08-11T15:51:32.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 -- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f17c806a120 msgr2=0x7f17c806c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f17c806a120 0x7f17c806c5e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0ca440 tx=0x7f17d000f040 comp rx=0 tx=0).stop 2023-08-11T15:51:32.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 -- 172.21.15.27:0/1928378884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17dc06b940 msgr2=0x7f17dc15de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17dc06b940 0x7f17dc15de70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f17d4014f80 tx=0x7f17d4004a40 comp rx=0 tx=0).stop 2023-08-11T15:51:32.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 -- 172.21.15.27:0/1928378884 shutdown_connections 2023-08-11T15:51:32.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f17c806a120 0x7f17c806c5e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17dc1637b0 0x7f17dc0b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17dc0c3ed0 0x7f17dc0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 --2- 172.21.15.27:0/1928378884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17dc06b940 0x7f17dc15de70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 -- 172.21.15.27:0/1928378884 >> 172.21.15.27:0/1928378884 conn(0x7f17dc069800 msgr2=0x7f17dc0895c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 -- 172.21.15.27:0/1928378884 shutdown_connections 2023-08-11T15:51:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.643+0000 7f17e15fc640 1 -- 172.21.15.27:0/1928378884 wait complete. 2023-08-11T15:51:32.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.646+0000 7fe4472de640 1 -- 172.21.15.27:0/3694981967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe4200057d0 con 0x7fe4380de7a0 2023-08-11T15:51:32.649 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542157 got 60129542157 for osd.5 2023-08-11T15:51:32.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.647+0000 7fe435ffb640 1 -- 172.21.15.27:0/3694981967 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe4400f2860 con 0x7fe4380de7a0 2023-08-11T15:51:32.649 INFO:teuthology.orchestra.run.smithi027.stdout:60129542156 2023-08-11T15:51:32.650 DEBUG:teuthology.parallel:result is None 2023-08-11T15:51:32.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.649+0000 7fe4472de640 1 -- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42406a0d0 msgr2=0x7fe42406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42406a0d0 0x7fe42406c590 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00ac10 tx=0x7fe43c008040 comp rx=0 tx=0).stop 2023-08-11T15:51:32.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 -- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380de7a0 msgr2=0x7fe4380ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380de7a0 0x7fe4380ce750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe4400735f0 tx=0x7fe438056e70 comp rx=0 tx=0).stop 2023-08-11T15:51:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 -- 172.21.15.27:0/3694981967 shutdown_connections 2023-08-11T15:51:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42406a0d0 0x7fe42406c590 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe438047bf0 0x7fe438048070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380de7a0 0x7fe4380ce750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 --2- 172.21.15.27:0/3694981967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42c0047c0 0x7fe4380cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 -- 172.21.15.27:0/3694981967 >> 172.21.15.27:0/3694981967 conn(0x7fe438055a80 msgr2=0x7fe438006ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 -- 172.21.15.27:0/3694981967 shutdown_connections 2023-08-11T15:51:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.650+0000 7fe4472de640 1 -- 172.21.15.27:0/3694981967 wait complete. 2023-08-11T15:51:32.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.4 2023-08-11T15:51:32.657 DEBUG:teuthology.parallel:result is None 2023-08-11T15:51:32.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.6 2023-08-11T15:51:32.663 DEBUG:teuthology.parallel:result is None 2023-08-11T15:51:32.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.7 2023-08-11T15:51:32.668 DEBUG:teuthology.parallel:result is None 2023-08-11T15:51:32.668 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-11T15:51:32.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:51:32.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.936+0000 7f6c1e322640 1 Processor -- start 2023-08-11T15:51:32.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.936+0000 7f6c1e322640 1 -- start start 2023-08-11T15:51:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.938+0000 7f6c1e322640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c1815c960 0x7f6c1806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.938+0000 7f6c1e322640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1806c9f0 con 0x7f6c181584e0 2023-08-11T15:51:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.938+0000 7f6c1e322640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c180cd8c0 con 0x7f6c181588c0 2023-08-11T15:51:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.938+0000 7f6c1e322640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c180cc110 con 0x7f6c1815c960 2023-08-11T15:51:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.938+0000 7f6c1d320640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6c181584e0 0x7f6c180cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48550/0 (socket says 172.21.15.27:48550) 2023-08-11T15:51:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.938+0000 7f6c1d320640 1 -- 172.21.15.27:0/1360163257 learned_addr learned my addr 172.21.15.27:0/1360163257 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:32.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.938+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297308945 0 0) 0x7f6c14001110 con 0x7f6c181584e0 2023-08-11T15:51:32.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c04003440 con 0x7f6c181584e0 2023-08-11T15:51:32.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c1db21640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c1815c960 0x7f6c1806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4066845721 0 0) 0x7f6c04003440 con 0x7f6c181584e0 2023-08-11T15:51:32.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c1815c960 msgr2=0x7f6c1806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c1815c960 0x7f6c1806c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 >> v1:172.21.15.119:6789/0 conn(0x7f6c181588c0 legacy=0x7f6c1815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180cc3f0 con 0x7f6c181584e0 2023-08-11T15:51:32.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f6c14001800 con 0x7f6c181584e0 2023-08-11T15:51:32.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 >> v1:172.21.15.27:6789/0 conn(0x7f6c181584e0 legacy=0x7f6c180cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c1815c960 0x7f6c04003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c040040a0 0x7f6c0400a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c0400b040 0x7f6c0400d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c180cc110 con 0x7f6c040040a0 2023-08-11T15:51:32.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0400dd10 con 0x7f6c0400b040 2023-08-11T15:51:32.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0400ded0 con 0x7f6c1815c960 2023-08-11T15:51:32.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.939+0000 7f6c1cb1f640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c040040a0 0x7f6c0400a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.940+0000 7f6c1db21640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c0400b040 0x7f6c0400d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.940+0000 7f6c1d320640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c1815c960 0x7f6c04003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.940+0000 7f6c1cb1f640 1 -- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c1815c960 msgr2=0x7f6c04003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.940+0000 7f6c1cb1f640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c1815c960 0x7f6c04003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.940+0000 7f6c1cb1f640 1 -- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c0400b040 msgr2=0x7f6c0400d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:32.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.940+0000 7f6c1cb1f640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c0400b040 0x7f6c0400d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.940+0000 7f6c1cb1f640 1 -- 172.21.15.27:0/1360163257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04003440 con 0x7f6c040040a0 2023-08-11T15:51:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.941+0000 7f6c1cb1f640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c040040a0 0x7f6c0400a860 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6c08003db0 tx=0x7f6c040036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812fab743c863b96 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.941+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08032600 con 0x7f6c040040a0 2023-08-11T15:51:32.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.941+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0802e050 con 0x7f6c040040a0 2023-08-11T15:51:32.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.941+0000 7f6c0ffff640 1 -- 172.21.15.27:0/1360163257 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c080329e0 con 0x7f6c040040a0 2023-08-11T15:51:32.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.942+0000 7f6c1e322640 1 -- 172.21.15.27:0/1360163257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c040040a0 msgr2=0x7f6c0400a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.942+0000 7f6c1e322640 1 --2- 172.21.15.27:0/1360163257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c040040a0 0x7f6c0400a860 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6c08003db0 tx=0x7f6c040036f0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.942+0000 7f6c1db21640 1 -- 172.21.15.27:0/1360163257 reap_dead start 2023-08-11T15:51:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.942+0000 7f6c1e322640 1 -- 172.21.15.27:0/1360163257 shutdown_connections 2023-08-11T15:51:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.942+0000 7f6c1e322640 1 -- 172.21.15.27:0/1360163257 >> 172.21.15.27:0/1360163257 conn(0x7f6c180afb70 msgr2=0x7f6c180aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.942+0000 7f6c1e322640 1 -- 172.21.15.27:0/1360163257 shutdown_connections 2023-08-11T15:51:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.942+0000 7f6c1e322640 1 -- 172.21.15.27:0/1360163257 wait complete. 2023-08-11T15:51:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.944+0000 7f6c1e322640 1 Processor -- start 2023-08-11T15:51:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.944+0000 7f6c1e322640 1 -- start start 2023-08-11T15:51:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.945+0000 7f6c1e322640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c040040a0 0x7f6c18153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.945+0000 7f6c1e322640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c180ac740 0x7f6c1815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.945+0000 7f6c1e322640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c1815b400 0x7f6c180ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.945+0000 7f6c1e322640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c181548e0 con 0x7f6c1815b400 2023-08-11T15:51:32.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.945+0000 7f6c1e322640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1815b890 con 0x7f6c180ac740 2023-08-11T15:51:32.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.945+0000 7f6c1e322640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c181f2080 con 0x7f6c040040a0 2023-08-11T15:51:32.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.945+0000 7f6c1db21640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c1815b400 0x7f6c180ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.945+0000 7f6c1db21640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c1815b400 0x7f6c180ad9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46532/0 (socket says 172.21.15.27:46532) 2023-08-11T15:51:32.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.945+0000 7f6c1db21640 1 -- 172.21.15.27:0/1094569961 learned_addr learned my addr 172.21.15.27:0/1094569961 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:32.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.945+0000 7f6c1d320640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c040040a0 0x7f6c18153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c1db21640 1 -- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c040040a0 msgr2=0x7f6c18153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c1db21640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c040040a0 0x7f6c18153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c1db21640 1 -- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c180ac740 msgr2=0x7f6c1815aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:32.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c1cb1f640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c180ac740 0x7f6c1815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c1db21640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c180ac740 0x7f6c1815aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c1db21640 1 -- 172.21.15.27:0/1094569961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c18154360 con 0x7f6c1815b400 2023-08-11T15:51:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c1db21640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c1815b400 0x7f6c180ad9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6c180b0810 tx=0x7f6c10009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c0dffb640 1 -- 172.21.15.27:0/1094569961 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10007de0 con 0x7f6c1815b400 2023-08-11T15:51:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c0dffb640 1 -- 172.21.15.27:0/1094569961 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1000f050 con 0x7f6c1815b400 2023-08-11T15:51:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c1e322640 1 -- 172.21.15.27:0/1094569961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c180ab980 con 0x7f6c1815b400 2023-08-11T15:51:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c0dffb640 1 -- 172.21.15.27:0/1094569961 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10014650 con 0x7f6c1815b400 2023-08-11T15:51:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.946+0000 7f6c1e322640 1 -- 172.21.15.27:0/1094569961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c180abf40 con 0x7f6c1815b400 2023-08-11T15:51:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.947+0000 7f6c1e322640 1 -- 172.21.15.27:0/1094569961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1804e420 con 0x7f6c1815b400 2023-08-11T15:51:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.948+0000 7f6c0dffb640 1 -- 172.21.15.27:0/1094569961 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6c100147b0 con 0x7f6c1815b400 2023-08-11T15:51:32.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.949+0000 7f6c0dffb640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6bfc06a080 0x7f6bfc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:32.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.950+0000 7f6c0dffb640 1 -- 172.21.15.27:0/1094569961 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6c10088d30 con 0x7f6c1815b400 2023-08-11T15:51:32.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.950+0000 7f6c1d320640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6bfc06a080 0x7f6bfc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:32.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.951+0000 7f6c1d320640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6bfc06a080 0x7f6bfc06c540 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c140012b0 tx=0x7f6c14000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:32.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:32.953+0000 7f6c0dffb640 1 -- 172.21.15.27:0/1094569961 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6c10057b20 con 0x7f6c1815b400 2023-08-11T15:51:33.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.136+0000 7f6c1e322640 1 -- 172.21.15.27:0/1094569961 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c180cc9b0 con 0x7f6bfc06a080 2023-08-11T15:51:33.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.165+0000 7f6c0dffb640 1 -- 172.21.15.27:0/1094569961 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367163 (secure 0 0 0) 0x7f6c14005e40 con 0x7f6bfc06a080 2023-08-11T15:51:33.170 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:33.172 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:51:33.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 -- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6bfc06a080 msgr2=0x7f6bfc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6bfc06a080 0x7f6bfc06c540 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c140012b0 tx=0x7f6c14000ed0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 -- 172.21.15.27:0/1094569961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c1815b400 msgr2=0x7f6c180ad9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c1815b400 0x7f6c180ad9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6c180b0810 tx=0x7f6c10009da0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 -- 172.21.15.27:0/1094569961 shutdown_connections 2023-08-11T15:51:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6bfc06a080 0x7f6bfc06c540 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c1815b400 0x7f6c180ad9a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c180ac740 0x7f6c1815aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 --2- 172.21.15.27:0/1094569961 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6c040040a0 0x7f6c18153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 -- 172.21.15.27:0/1094569961 >> 172.21.15.27:0/1094569961 conn(0x7f6c180afb70 msgr2=0x7f6c18154ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 -- 172.21.15.27:0/1094569961 shutdown_connections 2023-08-11T15:51:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.171+0000 7f6c1e322640 1 -- 172.21.15.27:0/1094569961 wait complete. 2023-08-11T15:51:33.190 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-08-11T15:51:32.722468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232028,"kb_used_data":3560,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754742692,"statfs":{"total":773094113280,"available":772856516608,"internally_reserved":0,"allocated":3645440,"data_stored":2175582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005576"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449971+0000","last_change":"2023-08-11T15:51:05.843998+0000","last_active":"2023-08-11T15:51:07.449971+0000","last_peered":"2023-08-11T15:51:07.449971+0000","last_clean":"2023-08-11T15:51:07.449971+0000","last_became_active":"2023-08-11T15:51:05.843613+0000","last_became_peered":"2023-08-11T15:51:05.843613+0000","last_unstale":"2023-08-11T15:51:07.449971+0000","last_undegraded":"2023-08-11T15:51:07.449971+0000","last_fullsized":"2023-08-11T15:51:07.449971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:35:45.480000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842354+0000","last_change":"2023-08-11T15:51:00.746010+0000","last_active":"2023-08-11T15:51:06.842354+0000","last_peered":"2023-08-11T15:51:06.842354+0000","last_clean":"2023-08-11T15:51:06.842354+0000","last_became_active":"2023-08-11T15:51:00.745732+0000","last_became_peered":"2023-08-11T15:51:00.745732+0000","last_unstale":"2023-08-11T15:51:06.842354+0000","last_undegraded":"2023-08-11T15:51:06.842354+0000","last_fullsized":"2023-08-11T15:51:06.842354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:02.960249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028896+0000","last_change":"2023-08-11T15:51:01.746812+0000","last_active":"2023-08-11T15:51:08.028896+0000","last_peered":"2023-08-11T15:51:08.028896+0000","last_clean":"2023-08-11T15:51:08.028896+0000","last_became_active":"2023-08-11T15:51:01.745729+0000","last_became_peered":"2023-08-11T15:51:01.745729+0000","last_unstale":"2023-08-11T15:51:08.028896+0000","last_undegraded":"2023-08-11T15:51:08.028896+0000","last_fullsized":"2023-08-11T15:51:08.028896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:46.849896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028532+0000","last_change":"2023-08-11T15:51:05.846118+0000","last_active":"2023-08-11T15:51:08.028532+0000","last_peered":"2023-08-11T15:51:08.028532+0000","last_clean":"2023-08-11T15:51:08.028532+0000","last_became_active":"2023-08-11T15:51:05.845703+0000","last_became_peered":"2023-08-11T15:51:05.845703+0000","last_unstale":"2023-08-11T15:51:08.028532+0000","last_undegraded":"2023-08-11T15:51:08.028532+0000","last_fullsized":"2023-08-11T15:51:08.028532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:15.472354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278289+0000","last_change":"2023-08-11T15:51:00.732689+0000","last_active":"2023-08-11T15:51:07.278289+0000","last_peered":"2023-08-11T15:51:07.278289+0000","last_clean":"2023-08-11T15:51:07.278289+0000","last_became_active":"2023-08-11T15:51:00.732185+0000","last_became_peered":"2023-08-11T15:51:00.732185+0000","last_unstale":"2023-08-11T15:51:07.278289+0000","last_undegraded":"2023-08-11T15:51:07.278289+0000","last_fullsized":"2023-08-11T15:51:07.278289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:19.559486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277505+0000","last_change":"2023-08-11T15:51:01.740441+0000","last_active":"2023-08-11T15:51:07.277505+0000","last_peered":"2023-08-11T15:51:07.277505+0000","last_clean":"2023-08-11T15:51:07.277505+0000","last_became_active":"2023-08-11T15:51:01.740104+0000","last_became_peered":"2023-08-11T15:51:01.740104+0000","last_unstale":"2023-08-11T15:51:07.277505+0000","last_undegraded":"2023-08-11T15:51:07.277505+0000","last_fullsized":"2023-08-11T15:51:07.277505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842536+0000","last_change":"2023-08-11T15:51:05.850642+0000","last_active":"2023-08-11T15:51:06.842536+0000","last_peered":"2023-08-11T15:51:06.842536+0000","last_clean":"2023-08-11T15:51:06.842536+0000","last_became_active":"2023-08-11T15:51:05.850276+0000","last_became_peered":"2023-08-11T15:51:05.850276+0000","last_unstale":"2023-08-11T15:51:06.842536+0000","last_undegraded":"2023-08-11T15:51:06.842536+0000","last_fullsized":"2023-08-11T15:51:06.842536+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:06:02.062687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845184+0000","last_change":"2023-08-11T15:51:00.753395+0000","last_active":"2023-08-11T15:51:06.845184+0000","last_peered":"2023-08-11T15:51:06.845184+0000","last_clean":"2023-08-11T15:51:06.845184+0000","last_became_active":"2023-08-11T15:51:00.753094+0000","last_became_peered":"2023-08-11T15:51:00.753094+0000","last_unstale":"2023-08-11T15:51:06.845184+0000","last_undegraded":"2023-08-11T15:51:06.845184+0000","last_fullsized":"2023-08-11T15:51:06.845184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:32.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029427+0000","last_change":"2023-08-11T15:51:01.748176+0000","last_active":"2023-08-11T15:51:08.029427+0000","last_peered":"2023-08-11T15:51:08.029427+0000","last_clean":"2023-08-11T15:51:08.029427+0000","last_became_active":"2023-08-11T15:51:01.747880+0000","last_became_peered":"2023-08-11T15:51:01.747880+0000","last_unstale":"2023-08-11T15:51:08.029427+0000","last_undegraded":"2023-08-11T15:51:08.029427+0000","last_fullsized":"2023-08-11T15:51:08.029427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:40:28.656635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842819+0000","last_change":"2023-08-11T15:51:05.854549+0000","last_active":"2023-08-11T15:51:06.842819+0000","last_peered":"2023-08-11T15:51:06.842819+0000","last_clean":"2023-08-11T15:51:06.842819+0000","last_became_active":"2023-08-11T15:51:05.853942+0000","last_became_peered":"2023-08-11T15:51:05.853942+0000","last_unstale":"2023-08-11T15:51:06.842819+0000","last_undegraded":"2023-08-11T15:51:06.842819+0000","last_fullsized":"2023-08-11T15:51:06.842819+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:12.616756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844270+0000","last_change":"2023-08-11T15:51:00.754010+0000","last_active":"2023-08-11T15:51:06.844270+0000","last_peered":"2023-08-11T15:51:06.844270+0000","last_clean":"2023-08-11T15:51:06.844270+0000","last_became_active":"2023-08-11T15:51:00.753640+0000","last_became_peered":"2023-08-11T15:51:00.753640+0000","last_unstale":"2023-08-11T15:51:06.844270+0000","last_undegraded":"2023-08-11T15:51:06.844270+0000","last_fullsized":"2023-08-11T15:51:06.844270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:59.887617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029380+0000","last_change":"2023-08-11T15:51:01.748240+0000","last_active":"2023-08-11T15:51:08.029380+0000","last_peered":"2023-08-11T15:51:08.029380+0000","last_clean":"2023-08-11T15:51:08.029380+0000","last_became_active":"2023-08-11T15:51:01.747946+0000","last_became_peered":"2023-08-11T15:51:01.747946+0000","last_unstale":"2023-08-11T15:51:08.029380+0000","last_undegraded":"2023-08-11T15:51:08.029380+0000","last_fullsized":"2023-08-11T15:51:08.029380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:08:40.170218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450286+0000","last_change":"2023-08-11T15:51:05.843794+0000","last_active":"2023-08-11T15:51:07.450286+0000","last_peered":"2023-08-11T15:51:07.450286+0000","last_clean":"2023-08-11T15:51:07.450286+0000","last_became_active":"2023-08-11T15:51:05.843006+0000","last_became_peered":"2023-08-11T15:51:05.843006+0000","last_unstale":"2023-08-11T15:51:07.450286+0000","last_undegraded":"2023-08-11T15:51:07.450286+0000","last_fullsized":"2023-08-11T15:51:07.450286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:53:03.920014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841384+0000","last_change":"2023-08-11T15:51:00.753341+0000","last_active":"2023-08-11T15:51:06.841384+0000","last_peered":"2023-08-11T15:51:06.841384+0000","last_clean":"2023-08-11T15:51:06.841384+0000","last_became_active":"2023-08-11T15:51:00.753070+0000","last_became_peered":"2023-08-11T15:51:00.753070+0000","last_unstale":"2023-08-11T15:51:06.841384+0000","last_undegraded":"2023-08-11T15:51:06.841384+0000","last_fullsized":"2023-08-11T15:51:06.841384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:07.508785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575377+0000","last_change":"2023-08-11T15:51:01.745198+0000","last_active":"2023-08-11T15:51:08.575377+0000","last_peered":"2023-08-11T15:51:08.575377+0000","last_clean":"2023-08-11T15:51:08.575377+0000","last_became_active":"2023-08-11T15:51:01.744845+0000","last_became_peered":"2023-08-11T15:51:01.744845+0000","last_unstale":"2023-08-11T15:51:08.575377+0000","last_undegraded":"2023-08-11T15:51:08.575377+0000","last_fullsized":"2023-08-11T15:51:08.575377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:07:54.433272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029435+0000","last_change":"2023-08-11T15:51:05.846476+0000","last_active":"2023-08-11T15:51:08.029435+0000","last_peered":"2023-08-11T15:51:08.029435+0000","last_clean":"2023-08-11T15:51:08.029435+0000","last_became_active":"2023-08-11T15:51:05.846010+0000","last_became_peered":"2023-08-11T15:51:05.846010+0000","last_unstale":"2023-08-11T15:51:08.029435+0000","last_undegraded":"2023-08-11T15:51:08.029435+0000","last_fullsized":"2023-08-11T15:51:08.029435+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:55:13.061478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029848+0000","last_change":"2023-08-11T15:51:00.748391+0000","last_active":"2023-08-11T15:51:08.029848+0000","last_peered":"2023-08-11T15:51:08.029848+0000","last_clean":"2023-08-11T15:51:08.029848+0000","last_became_active":"2023-08-11T15:51:00.744340+0000","last_became_peered":"2023-08-11T15:51:00.744340+0000","last_unstale":"2023-08-11T15:51:08.029848+0000","last_undegraded":"2023-08-11T15:51:08.029848+0000","last_fullsized":"2023-08-11T15:51:08.029848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:56:37.817338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278162+0000","last_change":"2023-08-11T15:51:01.749456+0000","last_active":"2023-08-11T15:51:07.278162+0000","last_peered":"2023-08-11T15:51:07.278162+0000","last_clean":"2023-08-11T15:51:07.278162+0000","last_became_active":"2023-08-11T15:51:01.749042+0000","last_became_peered":"2023-08-11T15:51:01.749042+0000","last_unstale":"2023-08-11T15:51:07.278162+0000","last_undegraded":"2023-08-11T15:51:07.278162+0000","last_fullsized":"2023-08-11T15:51:07.278162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:03.472522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644834+0000","last_change":"2023-08-11T15:51:05.849122+0000","last_active":"2023-08-11T15:51:08.644834+0000","last_peered":"2023-08-11T15:51:08.644834+0000","last_clean":"2023-08-11T15:51:08.644834+0000","last_became_active":"2023-08-11T15:51:05.848698+0000","last_became_peered":"2023-08-11T15:51:05.848698+0000","last_unstale":"2023-08-11T15:51:08.644834+0000","last_undegraded":"2023-08-11T15:51:08.644834+0000","last_fullsized":"2023-08-11T15:51:08.644834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:46:03.919402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449963+0000","last_change":"2023-08-11T15:51:00.743735+0000","last_active":"2023-08-11T15:51:07.449963+0000","last_peered":"2023-08-11T15:51:07.449963+0000","last_clean":"2023-08-11T15:51:07.449963+0000","last_became_active":"2023-08-11T15:51:00.743412+0000","last_became_peered":"2023-08-11T15:51:00.743412+0000","last_unstale":"2023-08-11T15:51:07.449963+0000","last_undegraded":"2023-08-11T15:51:07.449963+0000","last_fullsized":"2023-08-11T15:51:07.449963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:07.487770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841592+0000","last_change":"2023-08-11T15:51:01.740683+0000","last_active":"2023-08-11T15:51:06.841592+0000","last_peered":"2023-08-11T15:51:06.841592+0000","last_clean":"2023-08-11T15:51:06.841592+0000","last_became_active":"2023-08-11T15:51:01.740230+0000","last_became_peered":"2023-08-11T15:51:01.740230+0000","last_unstale":"2023-08-11T15:51:06.841592+0000","last_undegraded":"2023-08-11T15:51:06.841592+0000","last_fullsized":"2023-08-11T15:51:06.841592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:57.966913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841129+0000","last_change":"2023-08-11T15:51:05.843204+0000","last_active":"2023-08-11T15:51:06.841129+0000","last_peered":"2023-08-11T15:51:06.841129+0000","last_clean":"2023-08-11T15:51:06.841129+0000","last_became_active":"2023-08-11T15:51:05.842830+0000","last_became_peered":"2023-08-11T15:51:05.842830+0000","last_unstale":"2023-08-11T15:51:06.841129+0000","last_undegraded":"2023-08-11T15:51:06.841129+0000","last_fullsized":"2023-08-11T15:51:06.841129+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:13.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277138+0000","last_change":"2023-08-11T15:51:00.735703+0000","last_active":"2023-08-11T15:51:07.277138+0000","last_peered":"2023-08-11T15:51:07.277138+0000","last_clean":"2023-08-11T15:51:07.277138+0000","last_became_active":"2023-08-11T15:51:00.735384+0000","last_became_peered":"2023-08-11T15:51:00.735384+0000","last_unstale":"2023-08-11T15:51:07.277138+0000","last_undegraded":"2023-08-11T15:51:07.277138+0000","last_fullsized":"2023-08-11T15:51:07.277138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:59.887617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644911+0000","last_change":"2023-08-11T15:51:01.742945+0000","last_active":"2023-08-11T15:51:08.644911+0000","last_peered":"2023-08-11T15:51:08.644911+0000","last_clean":"2023-08-11T15:51:08.644911+0000","last_became_active":"2023-08-11T15:51:01.742477+0000","last_became_peered":"2023-08-11T15:51:01.742477+0000","last_unstale":"2023-08-11T15:51:08.644911+0000","last_undegraded":"2023-08-11T15:51:08.644911+0000","last_fullsized":"2023-08-11T15:51:08.644911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:17:49.108624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276720+0000","last_change":"2023-08-11T15:51:01.755598+0000","last_active":"2023-08-11T15:51:07.276720+0000","last_peered":"2023-08-11T15:51:07.276720+0000","last_clean":"2023-08-11T15:51:07.276720+0000","last_became_active":"2023-08-11T15:51:01.755329+0000","last_became_peered":"2023-08-11T15:51:01.755329+0000","last_unstale":"2023-08-11T15:51:07.276720+0000","last_undegraded":"2023-08-11T15:51:07.276720+0000","last_fullsized":"2023-08-11T15:51:07.276720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:09.551106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842608+0000","last_change":"2023-08-11T15:51:00.747156+0000","last_active":"2023-08-11T15:51:06.842608+0000","last_peered":"2023-08-11T15:51:06.842608+0000","last_clean":"2023-08-11T15:51:06.842608+0000","last_became_active":"2023-08-11T15:51:00.745449+0000","last_became_peered":"2023-08-11T15:51:00.745449+0000","last_unstale":"2023-08-11T15:51:06.842608+0000","last_undegraded":"2023-08-11T15:51:06.842608+0000","last_fullsized":"2023-08-11T15:51:06.842608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643560+0000","last_change":"2023-08-11T15:51:05.839469+0000","last_active":"2023-08-11T15:51:08.643560+0000","last_peered":"2023-08-11T15:51:08.643560+0000","last_clean":"2023-08-11T15:51:08.643560+0000","last_became_active":"2023-08-11T15:51:05.839035+0000","last_became_peered":"2023-08-11T15:51:05.839035+0000","last_unstale":"2023-08-11T15:51:08.643560+0000","last_undegraded":"2023-08-11T15:51:08.643560+0000","last_fullsized":"2023-08-11T15:51:08.643560+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:11.122033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276959+0000","last_change":"2023-08-11T15:51:01.748158+0000","last_active":"2023-08-11T15:51:07.276959+0000","last_peered":"2023-08-11T15:51:07.276959+0000","last_clean":"2023-08-11T15:51:07.276959+0000","last_became_active":"2023-08-11T15:51:01.747870+0000","last_became_peered":"2023-08-11T15:51:01.747870+0000","last_unstale":"2023-08-11T15:51:07.276959+0000","last_undegraded":"2023-08-11T15:51:07.276959+0000","last_fullsized":"2023-08-11T15:51:07.276959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:57.966913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845778+0000","last_change":"2023-08-11T15:51:00.739033+0000","last_active":"2023-08-11T15:51:06.845778+0000","last_peered":"2023-08-11T15:51:06.845778+0000","last_clean":"2023-08-11T15:51:06.845778+0000","last_became_active":"2023-08-11T15:51:00.738746+0000","last_became_peered":"2023-08-11T15:51:00.738746+0000","last_unstale":"2023-08-11T15:51:06.845778+0000","last_undegraded":"2023-08-11T15:51:06.845778+0000","last_fullsized":"2023-08-11T15:51:06.845778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841377+0000","last_change":"2023-08-11T15:51:05.853925+0000","last_active":"2023-08-11T15:51:06.841377+0000","last_peered":"2023-08-11T15:51:06.841377+0000","last_clean":"2023-08-11T15:51:06.841377+0000","last_became_active":"2023-08-11T15:51:05.853531+0000","last_became_peered":"2023-08-11T15:51:05.853531+0000","last_unstale":"2023-08-11T15:51:06.841377+0000","last_undegraded":"2023-08-11T15:51:06.841377+0000","last_fullsized":"2023-08-11T15:51:06.841377+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:13.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843271+0000","last_change":"2023-08-11T15:51:00.733126+0000","last_active":"2023-08-11T15:51:06.843271+0000","last_peered":"2023-08-11T15:51:06.843271+0000","last_clean":"2023-08-11T15:51:06.843271+0000","last_became_active":"2023-08-11T15:51:00.731695+0000","last_became_peered":"2023-08-11T15:51:00.731695+0000","last_unstale":"2023-08-11T15:51:06.843271+0000","last_undegraded":"2023-08-11T15:51:06.843271+0000","last_fullsized":"2023-08-11T15:51:06.843271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:07.914582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449817+0000","last_change":"2023-08-11T15:51:01.750498+0000","last_active":"2023-08-11T15:51:07.449817+0000","last_peered":"2023-08-11T15:51:07.449817+0000","last_clean":"2023-08-11T15:51:07.449817+0000","last_became_active":"2023-08-11T15:51:01.750204+0000","last_became_peered":"2023-08-11T15:51:01.750204+0000","last_unstale":"2023-08-11T15:51:07.449817+0000","last_undegraded":"2023-08-11T15:51:07.449817+0000","last_fullsized":"2023-08-11T15:51:07.449817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:30:40.844864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277300+0000","last_change":"2023-08-11T15:51:05.849252+0000","last_active":"2023-08-11T15:51:07.277300+0000","last_peered":"2023-08-11T15:51:07.277300+0000","last_clean":"2023-08-11T15:51:07.277300+0000","last_became_active":"2023-08-11T15:51:05.848107+0000","last_became_peered":"2023-08-11T15:51:05.848107+0000","last_unstale":"2023-08-11T15:51:07.277300+0000","last_undegraded":"2023-08-11T15:51:07.277300+0000","last_fullsized":"2023-08-11T15:51:07.277300+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:13.022552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:22.839046+0000","last_change":"2023-08-11T15:51:00.743771+0000","last_active":"2023-08-11T15:51:22.839046+0000","last_peered":"2023-08-11T15:51:22.839046+0000","last_clean":"2023-08-11T15:51:22.839046+0000","last_became_active":"2023-08-11T15:51:00.741410+0000","last_became_peered":"2023-08-11T15:51:00.741410+0000","last_unstale":"2023-08-11T15:51:22.839046+0000","last_undegraded":"2023-08-11T15:51:22.839046+0000","last_fullsized":"2023-08-11T15:51:22.839046+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:46:01.024956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029670+0000","last_change":"2023-08-11T15:51:01.742960+0000","last_active":"2023-08-11T15:51:08.029670+0000","last_peered":"2023-08-11T15:51:08.029670+0000","last_clean":"2023-08-11T15:51:08.029670+0000","last_became_active":"2023-08-11T15:51:01.742674+0000","last_became_peered":"2023-08-11T15:51:01.742674+0000","last_unstale":"2023-08-11T15:51:08.029670+0000","last_undegraded":"2023-08-11T15:51:08.029670+0000","last_fullsized":"2023-08-11T15:51:08.029670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:16:42.978408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450187+0000","last_change":"2023-08-11T15:51:05.850946+0000","last_active":"2023-08-11T15:51:07.450187+0000","last_peered":"2023-08-11T15:51:07.450187+0000","last_clean":"2023-08-11T15:51:07.450187+0000","last_became_active":"2023-08-11T15:51:05.850520+0000","last_became_peered":"2023-08-11T15:51:05.850520+0000","last_unstale":"2023-08-11T15:51:07.450187+0000","last_undegraded":"2023-08-11T15:51:07.450187+0000","last_fullsized":"2023-08-11T15:51:07.450187+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:19:49.259478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029612+0000","last_change":"2023-08-11T15:51:01.733689+0000","last_active":"2023-08-11T15:51:08.029612+0000","last_peered":"2023-08-11T15:51:08.029612+0000","last_clean":"2023-08-11T15:51:08.029612+0000","last_became_active":"2023-08-11T15:51:01.732609+0000","last_became_peered":"2023-08-11T15:51:01.732609+0000","last_unstale":"2023-08-11T15:51:08.029612+0000","last_undegraded":"2023-08-11T15:51:08.029612+0000","last_fullsized":"2023-08-11T15:51:08.029612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:37:47.670825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.645014+0000","last_change":"2023-08-11T15:50:54.667601+0000","last_active":"2023-08-11T15:51:08.645014+0000","last_peered":"2023-08-11T15:51:08.645014+0000","last_clean":"2023-08-11T15:51:08.645014+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:51:08.645014+0000","last_undegraded":"2023-08-11T15:51:08.645014+0000","last_fullsized":"2023-08-11T15:51:08.645014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:26.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842067+0000","last_change":"2023-08-11T15:51:00.744472+0000","last_active":"2023-08-11T15:51:06.842067+0000","last_peered":"2023-08-11T15:51:06.842067+0000","last_clean":"2023-08-11T15:51:06.842067+0000","last_became_active":"2023-08-11T15:51:00.744162+0000","last_became_peered":"2023-08-11T15:51:00.744162+0000","last_unstale":"2023-08-11T15:51:06.842067+0000","last_undegraded":"2023-08-11T15:51:06.842067+0000","last_fullsized":"2023-08-11T15:51:06.842067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:44.445917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276689+0000","last_change":"2023-08-11T15:51:05.840924+0000","last_active":"2023-08-11T15:51:07.276689+0000","last_peered":"2023-08-11T15:51:07.276689+0000","last_clean":"2023-08-11T15:51:07.276689+0000","last_became_active":"2023-08-11T15:51:05.840447+0000","last_became_peered":"2023-08-11T15:51:05.840447+0000","last_unstale":"2023-08-11T15:51:07.276689+0000","last_undegraded":"2023-08-11T15:51:07.276689+0000","last_fullsized":"2023-08-11T15:51:07.276689+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:15.105060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277414+0000","last_change":"2023-08-11T15:51:01.747032+0000","last_active":"2023-08-11T15:51:07.277414+0000","last_peered":"2023-08-11T15:51:07.277414+0000","last_clean":"2023-08-11T15:51:07.277414+0000","last_became_active":"2023-08-11T15:51:01.746677+0000","last_became_peered":"2023-08-11T15:51:01.746677+0000","last_unstale":"2023-08-11T15:51:07.277414+0000","last_undegraded":"2023-08-11T15:51:07.277414+0000","last_fullsized":"2023-08-11T15:51:07.277414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:45.458114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450746+0000","last_change":"2023-08-11T15:50:54.662968+0000","last_active":"2023-08-11T15:51:07.450746+0000","last_peered":"2023-08-11T15:51:07.450746+0000","last_clean":"2023-08-11T15:51:07.450746+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:51:07.450746+0000","last_undegraded":"2023-08-11T15:51:07.450746+0000","last_fullsized":"2023-08-11T15:51:07.450746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:21.192999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278963+0000","last_change":"2023-08-11T15:51:00.728408+0000","last_active":"2023-08-11T15:51:07.278963+0000","last_peered":"2023-08-11T15:51:07.278963+0000","last_clean":"2023-08-11T15:51:07.278963+0000","last_became_active":"2023-08-11T15:51:00.721838+0000","last_became_peered":"2023-08-11T15:51:00.721838+0000","last_unstale":"2023-08-11T15:51:07.278963+0000","last_undegraded":"2023-08-11T15:51:07.278963+0000","last_fullsized":"2023-08-11T15:51:07.278963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:10.364383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028778+0000","last_change":"2023-08-11T15:51:05.850931+0000","last_active":"2023-08-11T15:51:08.028778+0000","last_peered":"2023-08-11T15:51:08.028778+0000","last_clean":"2023-08-11T15:51:08.028778+0000","last_became_active":"2023-08-11T15:51:05.850227+0000","last_became_peered":"2023-08-11T15:51:05.850227+0000","last_unstale":"2023-08-11T15:51:08.028778+0000","last_undegraded":"2023-08-11T15:51:08.028778+0000","last_fullsized":"2023-08-11T15:51:08.028778+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.029041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840867+0000","last_change":"2023-08-11T15:51:01.739504+0000","last_active":"2023-08-11T15:51:06.840867+0000","last_peered":"2023-08-11T15:51:06.840867+0000","last_clean":"2023-08-11T15:51:06.840867+0000","last_became_active":"2023-08-11T15:51:01.739172+0000","last_became_peered":"2023-08-11T15:51:01.739172+0000","last_unstale":"2023-08-11T15:51:06.840867+0000","last_undegraded":"2023-08-11T15:51:06.840867+0000","last_fullsized":"2023-08-11T15:51:06.840867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:12:34.705521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844166+0000","last_change":"2023-08-11T15:50:55.272783+0000","last_active":"2023-08-11T15:51:06.844166+0000","last_peered":"2023-08-11T15:51:06.844166+0000","last_clean":"2023-08-11T15:51:06.844166+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:51:06.844166+0000","last_undegraded":"2023-08-11T15:51:06.844166+0000","last_fullsized":"2023-08-11T15:51:06.844166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:26.593873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450158+0000","last_change":"2023-08-11T15:51:00.737083+0000","last_active":"2023-08-11T15:51:07.450158+0000","last_peered":"2023-08-11T15:51:07.450158+0000","last_clean":"2023-08-11T15:51:07.450158+0000","last_became_active":"2023-08-11T15:51:00.736799+0000","last_became_peered":"2023-08-11T15:51:00.736799+0000","last_unstale":"2023-08-11T15:51:07.450158+0000","last_undegraded":"2023-08-11T15:51:07.450158+0000","last_fullsized":"2023-08-11T15:51:07.450158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:03:14.195345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643834+0000","last_change":"2023-08-11T15:51:05.842861+0000","last_active":"2023-08-11T15:51:08.643834+0000","last_peered":"2023-08-11T15:51:08.643834+0000","last_clean":"2023-08-11T15:51:08.643834+0000","last_became_active":"2023-08-11T15:51:05.842445+0000","last_became_peered":"2023-08-11T15:51:05.842445+0000","last_unstale":"2023-08-11T15:51:08.643834+0000","last_undegraded":"2023-08-11T15:51:08.643834+0000","last_fullsized":"2023-08-11T15:51:08.643834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:41.996042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449997+0000","last_change":"2023-08-11T15:51:01.746944+0000","last_active":"2023-08-11T15:51:07.449997+0000","last_peered":"2023-08-11T15:51:07.449997+0000","last_clean":"2023-08-11T15:51:07.449997+0000","last_became_active":"2023-08-11T15:51:01.745106+0000","last_became_peered":"2023-08-11T15:51:01.745106+0000","last_unstale":"2023-08-11T15:51:07.449997+0000","last_undegraded":"2023-08-11T15:51:07.449997+0000","last_fullsized":"2023-08-11T15:51:07.449997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:38:45.900704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450317+0000","last_change":"2023-08-11T15:50:55.270974+0000","last_active":"2023-08-11T15:51:07.450317+0000","last_peered":"2023-08-11T15:51:07.450317+0000","last_clean":"2023-08-11T15:51:07.450317+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:51:07.450317+0000","last_undegraded":"2023-08-11T15:51:07.450317+0000","last_fullsized":"2023-08-11T15:51:07.450317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:49:35.912143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845445+0000","last_change":"2023-08-11T15:51:00.736540+0000","last_active":"2023-08-11T15:51:06.845445+0000","last_peered":"2023-08-11T15:51:06.845445+0000","last_clean":"2023-08-11T15:51:06.845445+0000","last_became_active":"2023-08-11T15:51:00.736211+0000","last_became_peered":"2023-08-11T15:51:00.736211+0000","last_unstale":"2023-08-11T15:51:06.845445+0000","last_undegraded":"2023-08-11T15:51:06.845445+0000","last_fullsized":"2023-08-11T15:51:06.845445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.749276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840401+0000","last_change":"2023-08-11T15:51:05.834845+0000","last_active":"2023-08-11T15:51:06.840401+0000","last_peered":"2023-08-11T15:51:06.840401+0000","last_clean":"2023-08-11T15:51:06.840401+0000","last_became_active":"2023-08-11T15:51:05.834329+0000","last_became_peered":"2023-08-11T15:51:05.834329+0000","last_unstale":"2023-08-11T15:51:06.840401+0000","last_undegraded":"2023-08-11T15:51:06.840401+0000","last_fullsized":"2023-08-11T15:51:06.840401+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:27:54.750554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450003+0000","last_change":"2023-08-11T15:51:01.744716+0000","last_active":"2023-08-11T15:51:07.450003+0000","last_peered":"2023-08-11T15:51:07.450003+0000","last_clean":"2023-08-11T15:51:07.450003+0000","last_became_active":"2023-08-11T15:51:01.744413+0000","last_became_peered":"2023-08-11T15:51:01.744413+0000","last_unstale":"2023-08-11T15:51:07.450003+0000","last_undegraded":"2023-08-11T15:51:07.450003+0000","last_fullsized":"2023-08-11T15:51:07.450003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:43:31.654955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030156+0000","last_change":"2023-08-11T15:51:00.744357+0000","last_active":"2023-08-11T15:51:08.030156+0000","last_peered":"2023-08-11T15:51:08.030156+0000","last_clean":"2023-08-11T15:51:08.030156+0000","last_became_active":"2023-08-11T15:51:00.744005+0000","last_became_peered":"2023-08-11T15:51:00.744005+0000","last_unstale":"2023-08-11T15:51:08.030156+0000","last_undegraded":"2023-08-11T15:51:08.030156+0000","last_fullsized":"2023-08-11T15:51:08.030156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:40:52.791014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842501+0000","last_change":"2023-08-11T15:50:54.660862+0000","last_active":"2023-08-11T15:51:06.842501+0000","last_peered":"2023-08-11T15:51:06.842501+0000","last_clean":"2023-08-11T15:51:06.842501+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:51:06.842501+0000","last_undegraded":"2023-08-11T15:51:06.842501+0000","last_fullsized":"2023-08-11T15:51:06.842501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:01.459444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840964+0000","last_change":"2023-08-11T15:51:05.834675+0000","last_active":"2023-08-11T15:51:06.840964+0000","last_peered":"2023-08-11T15:51:06.840964+0000","last_clean":"2023-08-11T15:51:06.840964+0000","last_became_active":"2023-08-11T15:51:05.833754+0000","last_became_peered":"2023-08-11T15:51:05.833754+0000","last_unstale":"2023-08-11T15:51:06.840964+0000","last_undegraded":"2023-08-11T15:51:06.840964+0000","last_fullsized":"2023-08-11T15:51:06.840964+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:02:13.485965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450824+0000","last_change":"2023-08-11T15:51:01.747929+0000","last_active":"2023-08-11T15:51:07.450824+0000","last_peered":"2023-08-11T15:51:07.450824+0000","last_clean":"2023-08-11T15:51:07.450824+0000","last_became_active":"2023-08-11T15:51:01.747489+0000","last_became_peered":"2023-08-11T15:51:01.747489+0000","last_unstale":"2023-08-11T15:51:07.450824+0000","last_undegraded":"2023-08-11T15:51:07.450824+0000","last_fullsized":"2023-08-11T15:51:07.450824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:16:47.148363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449531+0000","last_change":"2023-08-11T15:51:00.735611+0000","last_active":"2023-08-11T15:51:07.449531+0000","last_peered":"2023-08-11T15:51:07.449531+0000","last_clean":"2023-08-11T15:51:07.449531+0000","last_became_active":"2023-08-11T15:51:00.735334+0000","last_became_peered":"2023-08-11T15:51:00.735334+0000","last_unstale":"2023-08-11T15:51:07.449531+0000","last_undegraded":"2023-08-11T15:51:07.449531+0000","last_fullsized":"2023-08-11T15:51:07.449531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:21:47.785823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574361+0000","last_change":"2023-08-11T15:50:54.660575+0000","last_active":"2023-08-11T15:51:08.574361+0000","last_peered":"2023-08-11T15:51:08.574361+0000","last_clean":"2023-08-11T15:51:08.574361+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:51:08.574361+0000","last_undegraded":"2023-08-11T15:51:08.574361+0000","last_fullsized":"2023-08-11T15:51:08.574361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:44:19.275242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574984+0000","last_change":"2023-08-11T15:51:05.851204+0000","last_active":"2023-08-11T15:51:08.574984+0000","last_peered":"2023-08-11T15:51:08.574984+0000","last_clean":"2023-08-11T15:51:08.574984+0000","last_became_active":"2023-08-11T15:51:05.850707+0000","last_became_peered":"2023-08-11T15:51:05.850707+0000","last_unstale":"2023-08-11T15:51:08.574984+0000","last_undegraded":"2023-08-11T15:51:08.574984+0000","last_fullsized":"2023-08-11T15:51:08.574984+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:46:46.279070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841686+0000","last_change":"2023-08-11T15:51:01.754726+0000","last_active":"2023-08-11T15:51:06.841686+0000","last_peered":"2023-08-11T15:51:06.841686+0000","last_clean":"2023-08-11T15:51:06.841686+0000","last_became_active":"2023-08-11T15:51:01.754139+0000","last_became_peered":"2023-08-11T15:51:01.754139+0000","last_unstale":"2023-08-11T15:51:06.841686+0000","last_undegraded":"2023-08-11T15:51:06.841686+0000","last_fullsized":"2023-08-11T15:51:06.841686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839697+0000","last_change":"2023-08-11T15:51:00.753588+0000","last_active":"2023-08-11T15:51:06.839697+0000","last_peered":"2023-08-11T15:51:06.839697+0000","last_clean":"2023-08-11T15:51:06.839697+0000","last_became_active":"2023-08-11T15:51:00.753328+0000","last_became_peered":"2023-08-11T15:51:00.753328+0000","last_unstale":"2023-08-11T15:51:06.839697+0000","last_undegraded":"2023-08-11T15:51:06.839697+0000","last_fullsized":"2023-08-11T15:51:06.839697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:32.917550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842875+0000","last_change":"2023-08-11T15:50:54.669675+0000","last_active":"2023-08-11T15:51:06.842875+0000","last_peered":"2023-08-11T15:51:06.842875+0000","last_clean":"2023-08-11T15:51:06.842875+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:51:06.842875+0000","last_undegraded":"2023-08-11T15:51:06.842875+0000","last_fullsized":"2023-08-11T15:51:06.842875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:50.905375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845192+0000","last_change":"2023-08-11T15:51:05.839222+0000","last_active":"2023-08-11T15:51:06.845192+0000","last_peered":"2023-08-11T15:51:06.845192+0000","last_clean":"2023-08-11T15:51:06.845192+0000","last_became_active":"2023-08-11T15:51:05.838710+0000","last_became_peered":"2023-08-11T15:51:05.838710+0000","last_unstale":"2023-08-11T15:51:06.845192+0000","last_undegraded":"2023-08-11T15:51:06.845192+0000","last_fullsized":"2023-08-11T15:51:06.845192+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:07.568296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644528+0000","last_change":"2023-08-11T15:51:01.744538+0000","last_active":"2023-08-11T15:51:08.644528+0000","last_peered":"2023-08-11T15:51:08.644528+0000","last_clean":"2023-08-11T15:51:08.644528+0000","last_became_active":"2023-08-11T15:51:01.744177+0000","last_became_peered":"2023-08-11T15:51:01.744177+0000","last_unstale":"2023-08-11T15:51:08.644528+0000","last_undegraded":"2023-08-11T15:51:08.644528+0000","last_fullsized":"2023-08-11T15:51:08.644528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:36.996581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277663+0000","last_change":"2023-08-11T15:51:00.736542+0000","last_active":"2023-08-11T15:51:07.277663+0000","last_peered":"2023-08-11T15:51:07.277663+0000","last_clean":"2023-08-11T15:51:07.277663+0000","last_became_active":"2023-08-11T15:51:00.736156+0000","last_became_peered":"2023-08-11T15:51:00.736156+0000","last_unstale":"2023-08-11T15:51:07.277663+0000","last_undegraded":"2023-08-11T15:51:07.277663+0000","last_fullsized":"2023-08-11T15:51:07.277663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843800+0000","last_change":"2023-08-11T15:50:54.663730+0000","last_active":"2023-08-11T15:51:06.843800+0000","last_peered":"2023-08-11T15:51:06.843800+0000","last_clean":"2023-08-11T15:51:06.843800+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:51:06.843800+0000","last_undegraded":"2023-08-11T15:51:06.843800+0000","last_fullsized":"2023-08-11T15:51:06.843800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:36.826532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843222+0000","last_change":"2023-08-11T15:50:48.426393+0000","last_active":"2023-08-11T15:51:06.843222+0000","last_peered":"2023-08-11T15:51:06.843222+0000","last_clean":"2023-08-11T15:51:06.843222+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:51:06.843222+0000","last_undegraded":"2023-08-11T15:51:06.843222+0000","last_fullsized":"2023-08-11T15:51:06.843222+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_clean_scrub_stamp":"2023-08-11T15:50:47.133560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:47.340557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843724+0000","last_change":"2023-08-11T15:51:05.850670+0000","last_active":"2023-08-11T15:51:06.843724+0000","last_peered":"2023-08-11T15:51:06.843724+0000","last_clean":"2023-08-11T15:51:06.843724+0000","last_became_active":"2023-08-11T15:51:05.850026+0000","last_became_peered":"2023-08-11T15:51:05.850026+0000","last_unstale":"2023-08-11T15:51:06.843724+0000","last_undegraded":"2023-08-11T15:51:06.843724+0000","last_fullsized":"2023-08-11T15:51:06.843724+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:08.068219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450801+0000","last_change":"2023-08-11T15:51:01.748517+0000","last_active":"2023-08-11T15:51:07.450801+0000","last_peered":"2023-08-11T15:51:07.450801+0000","last_clean":"2023-08-11T15:51:07.450801+0000","last_became_active":"2023-08-11T15:51:01.748222+0000","last_became_peered":"2023-08-11T15:51:01.748222+0000","last_unstale":"2023-08-11T15:51:07.450801+0000","last_undegraded":"2023-08-11T15:51:07.450801+0000","last_fullsized":"2023-08-11T15:51:07.450801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:24:23.784093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277502+0000","last_change":"2023-08-11T15:51:00.734516+0000","last_active":"2023-08-11T15:51:07.277502+0000","last_peered":"2023-08-11T15:51:07.277502+0000","last_clean":"2023-08-11T15:51:07.277502+0000","last_became_active":"2023-08-11T15:51:00.734143+0000","last_became_peered":"2023-08-11T15:51:00.734143+0000","last_unstale":"2023-08-11T15:51:07.277502+0000","last_undegraded":"2023-08-11T15:51:07.277502+0000","last_fullsized":"2023-08-11T15:51:07.277502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:02.960249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030585+0000","last_change":"2023-08-11T15:51:05.848396+0000","last_active":"2023-08-11T15:51:08.030585+0000","last_peered":"2023-08-11T15:51:08.030585+0000","last_clean":"2023-08-11T15:51:08.030585+0000","last_became_active":"2023-08-11T15:51:05.846255+0000","last_became_peered":"2023-08-11T15:51:05.846255+0000","last_unstale":"2023-08-11T15:51:08.030585+0000","last_undegraded":"2023-08-11T15:51:08.030585+0000","last_fullsized":"2023-08-11T15:51:08.030585+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:11:39.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.031055+0000","last_change":"2023-08-11T15:51:01.745387+0000","last_active":"2023-08-11T15:51:08.031055+0000","last_peered":"2023-08-11T15:51:08.031055+0000","last_clean":"2023-08-11T15:51:08.031055+0000","last_became_active":"2023-08-11T15:51:01.745076+0000","last_became_peered":"2023-08-11T15:51:01.745076+0000","last_unstale":"2023-08-11T15:51:08.031055+0000","last_undegraded":"2023-08-11T15:51:08.031055+0000","last_fullsized":"2023-08-11T15:51:08.031055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:46:26.462256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030159+0000","last_change":"2023-08-11T15:51:00.749075+0000","last_active":"2023-08-11T15:51:08.030159+0000","last_peered":"2023-08-11T15:51:08.030159+0000","last_clean":"2023-08-11T15:51:08.030159+0000","last_became_active":"2023-08-11T15:51:00.748709+0000","last_became_peered":"2023-08-11T15:51:00.748709+0000","last_unstale":"2023-08-11T15:51:08.030159+0000","last_undegraded":"2023-08-11T15:51:08.030159+0000","last_fullsized":"2023-08-11T15:51:08.030159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:18:35.477914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844801+0000","last_change":"2023-08-11T15:51:05.843837+0000","last_active":"2023-08-11T15:51:06.844801+0000","last_peered":"2023-08-11T15:51:06.844801+0000","last_clean":"2023-08-11T15:51:06.844801+0000","last_became_active":"2023-08-11T15:51:05.843261+0000","last_became_peered":"2023-08-11T15:51:05.843261+0000","last_unstale":"2023-08-11T15:51:06.844801+0000","last_undegraded":"2023-08-11T15:51:06.844801+0000","last_fullsized":"2023-08-11T15:51:06.844801+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:08.068219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030584+0000","last_change":"2023-08-11T15:51:01.735881+0000","last_active":"2023-08-11T15:51:08.030584+0000","last_peered":"2023-08-11T15:51:08.030584+0000","last_clean":"2023-08-11T15:51:08.030584+0000","last_became_active":"2023-08-11T15:51:01.735297+0000","last_became_peered":"2023-08-11T15:51:01.735297+0000","last_unstale":"2023-08-11T15:51:08.030584+0000","last_undegraded":"2023-08-11T15:51:08.030584+0000","last_fullsized":"2023-08-11T15:51:08.030584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:06:05.316781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644461+0000","last_change":"2023-08-11T15:51:00.754786+0000","last_active":"2023-08-11T15:51:08.644461+0000","last_peered":"2023-08-11T15:51:08.644461+0000","last_clean":"2023-08-11T15:51:08.644461+0000","last_became_active":"2023-08-11T15:51:00.754437+0000","last_became_peered":"2023-08-11T15:51:00.754437+0000","last_unstale":"2023-08-11T15:51:08.644461+0000","last_undegraded":"2023-08-11T15:51:08.644461+0000","last_fullsized":"2023-08-11T15:51:08.644461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:41:49.524739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449772+0000","last_change":"2023-08-11T15:51:05.833713+0000","last_active":"2023-08-11T15:51:07.449772+0000","last_peered":"2023-08-11T15:51:07.449772+0000","last_clean":"2023-08-11T15:51:07.449772+0000","last_became_active":"2023-08-11T15:51:05.833089+0000","last_became_peered":"2023-08-11T15:51:05.833089+0000","last_unstale":"2023-08-11T15:51:07.449772+0000","last_undegraded":"2023-08-11T15:51:07.449772+0000","last_fullsized":"2023-08-11T15:51:07.449772+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:17:01.770576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030037+0000","last_change":"2023-08-11T15:51:01.748944+0000","last_active":"2023-08-11T15:51:08.030037+0000","last_peered":"2023-08-11T15:51:08.030037+0000","last_clean":"2023-08-11T15:51:08.030037+0000","last_became_active":"2023-08-11T15:51:01.748661+0000","last_became_peered":"2023-08-11T15:51:01.748661+0000","last_unstale":"2023-08-11T15:51:08.030037+0000","last_undegraded":"2023-08-11T15:51:08.030037+0000","last_fullsized":"2023-08-11T15:51:08.030037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:02:46.657720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278376+0000","last_change":"2023-08-11T15:51:00.743809+0000","last_active":"2023-08-11T15:51:07.278376+0000","last_peered":"2023-08-11T15:51:07.278376+0000","last_clean":"2023-08-11T15:51:07.278376+0000","last_became_active":"2023-08-11T15:51:00.743475+0000","last_became_peered":"2023-08-11T15:51:00.743475+0000","last_unstale":"2023-08-11T15:51:07.278376+0000","last_undegraded":"2023-08-11T15:51:07.278376+0000","last_fullsized":"2023-08-11T15:51:07.278376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574614+0000","last_change":"2023-08-11T15:51:05.845552+0000","last_active":"2023-08-11T15:51:08.574614+0000","last_peered":"2023-08-11T15:51:08.574614+0000","last_clean":"2023-08-11T15:51:08.574614+0000","last_became_active":"2023-08-11T15:51:05.845089+0000","last_became_peered":"2023-08-11T15:51:05.845089+0000","last_unstale":"2023-08-11T15:51:08.574614+0000","last_undegraded":"2023-08-11T15:51:08.574614+0000","last_fullsized":"2023-08-11T15:51:08.574614+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:14:44.850189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840418+0000","last_change":"2023-08-11T15:51:01.754448+0000","last_active":"2023-08-11T15:51:06.840418+0000","last_peered":"2023-08-11T15:51:06.840418+0000","last_clean":"2023-08-11T15:51:06.840418+0000","last_became_active":"2023-08-11T15:51:01.754103+0000","last_became_peered":"2023-08-11T15:51:01.754103+0000","last_unstale":"2023-08-11T15:51:06.840418+0000","last_undegraded":"2023-08-11T15:51:06.840418+0000","last_fullsized":"2023-08-11T15:51:06.840418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:48:19.707007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844315+0000","last_change":"2023-08-11T15:51:00.740399+0000","last_active":"2023-08-11T15:51:06.844315+0000","last_peered":"2023-08-11T15:51:06.844315+0000","last_clean":"2023-08-11T15:51:06.844315+0000","last_became_active":"2023-08-11T15:51:00.740100+0000","last_became_peered":"2023-08-11T15:51:00.740100+0000","last_unstale":"2023-08-11T15:51:06.844315+0000","last_undegraded":"2023-08-11T15:51:06.844315+0000","last_fullsized":"2023-08-11T15:51:06.844315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:07.914582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574234+0000","last_change":"2023-08-11T15:51:05.857518+0000","last_active":"2023-08-11T15:51:08.574234+0000","last_peered":"2023-08-11T15:51:08.574234+0000","last_clean":"2023-08-11T15:51:08.574234+0000","last_became_active":"2023-08-11T15:51:05.857113+0000","last_became_peered":"2023-08-11T15:51:05.857113+0000","last_unstale":"2023-08-11T15:51:08.574234+0000","last_undegraded":"2023-08-11T15:51:08.574234+0000","last_fullsized":"2023-08-11T15:51:08.574234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:27:22.211685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643846+0000","last_change":"2023-08-11T15:51:01.754343+0000","last_active":"2023-08-11T15:51:08.643846+0000","last_peered":"2023-08-11T15:51:08.643846+0000","last_clean":"2023-08-11T15:51:08.643846+0000","last_became_active":"2023-08-11T15:51:01.754068+0000","last_became_peered":"2023-08-11T15:51:01.754068+0000","last_unstale":"2023-08-11T15:51:08.643846+0000","last_undegraded":"2023-08-11T15:51:08.643846+0000","last_fullsized":"2023-08-11T15:51:08.643846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:17.417849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843859+0000","last_change":"2023-08-11T15:51:00.739404+0000","last_active":"2023-08-11T15:51:06.843859+0000","last_peered":"2023-08-11T15:51:06.843859+0000","last_clean":"2023-08-11T15:51:06.843859+0000","last_became_active":"2023-08-11T15:51:00.738824+0000","last_became_peered":"2023-08-11T15:51:00.738824+0000","last_unstale":"2023-08-11T15:51:06.843859+0000","last_undegraded":"2023-08-11T15:51:06.843859+0000","last_fullsized":"2023-08-11T15:51:06.843859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574756+0000","last_change":"2023-08-11T15:51:05.852495+0000","last_active":"2023-08-11T15:51:08.574756+0000","last_peered":"2023-08-11T15:51:08.574756+0000","last_clean":"2023-08-11T15:51:08.574756+0000","last_became_active":"2023-08-11T15:51:05.852028+0000","last_became_peered":"2023-08-11T15:51:05.852028+0000","last_unstale":"2023-08-11T15:51:08.574756+0000","last_undegraded":"2023-08-11T15:51:08.574756+0000","last_fullsized":"2023-08-11T15:51:08.574756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:00:50.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030008+0000","last_change":"2023-08-11T15:51:01.746332+0000","last_active":"2023-08-11T15:51:08.030008+0000","last_peered":"2023-08-11T15:51:08.030008+0000","last_clean":"2023-08-11T15:51:08.030008+0000","last_became_active":"2023-08-11T15:51:01.746037+0000","last_became_peered":"2023-08-11T15:51:01.746037+0000","last_unstale":"2023-08-11T15:51:08.030008+0000","last_undegraded":"2023-08-11T15:51:08.030008+0000","last_fullsized":"2023-08-11T15:51:08.030008+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:01:53.294858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846006+0000","last_change":"2023-08-11T15:51:00.750241+0000","last_active":"2023-08-11T15:51:06.846006+0000","last_peered":"2023-08-11T15:51:06.846006+0000","last_clean":"2023-08-11T15:51:06.846006+0000","last_became_active":"2023-08-11T15:51:00.749526+0000","last_became_peered":"2023-08-11T15:51:00.749526+0000","last_unstale":"2023-08-11T15:51:06.846006+0000","last_undegraded":"2023-08-11T15:51:06.846006+0000","last_fullsized":"2023-08-11T15:51:06.846006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.749276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574297+0000","last_change":"2023-08-11T15:51:05.852589+0000","last_active":"2023-08-11T15:51:08.574297+0000","last_peered":"2023-08-11T15:51:08.574297+0000","last_clean":"2023-08-11T15:51:08.574297+0000","last_became_active":"2023-08-11T15:51:05.852123+0000","last_became_peered":"2023-08-11T15:51:05.852123+0000","last_unstale":"2023-08-11T15:51:08.574297+0000","last_undegraded":"2023-08-11T15:51:08.574297+0000","last_fullsized":"2023-08-11T15:51:08.574297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:52.516197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841591+0000","last_change":"2023-08-11T15:51:01.740461+0000","last_active":"2023-08-11T15:51:06.841591+0000","last_peered":"2023-08-11T15:51:06.841591+0000","last_clean":"2023-08-11T15:51:06.841591+0000","last_became_active":"2023-08-11T15:51:01.740152+0000","last_became_peered":"2023-08-11T15:51:01.740152+0000","last_unstale":"2023-08-11T15:51:06.841591+0000","last_undegraded":"2023-08-11T15:51:06.841591+0000","last_fullsized":"2023-08-11T15:51:06.841591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:09.551106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846676+0000","last_change":"2023-08-11T15:51:00.749804+0000","last_active":"2023-08-11T15:51:06.846676+0000","last_peered":"2023-08-11T15:51:06.846676+0000","last_clean":"2023-08-11T15:51:06.846676+0000","last_became_active":"2023-08-11T15:51:00.749397+0000","last_became_peered":"2023-08-11T15:51:00.749397+0000","last_unstale":"2023-08-11T15:51:06.846676+0000","last_undegraded":"2023-08-11T15:51:06.846676+0000","last_fullsized":"2023-08-11T15:51:06.846676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843683+0000","last_change":"2023-08-11T15:51:05.857250+0000","last_active":"2023-08-11T15:51:06.843683+0000","last_peered":"2023-08-11T15:51:06.843683+0000","last_clean":"2023-08-11T15:51:06.843683+0000","last_became_active":"2023-08-11T15:51:05.856843+0000","last_became_peered":"2023-08-11T15:51:05.856843+0000","last_unstale":"2023-08-11T15:51:06.843683+0000","last_undegraded":"2023-08-11T15:51:06.843683+0000","last_fullsized":"2023-08-11T15:51:06.843683+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:30:03.847116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846267+0000","last_change":"2023-08-11T15:51:05.849584+0000","last_active":"2023-08-11T15:51:06.846267+0000","last_peered":"2023-08-11T15:51:06.846267+0000","last_clean":"2023-08-11T15:51:06.846267+0000","last_became_active":"2023-08-11T15:51:05.849144+0000","last_became_peered":"2023-08-11T15:51:05.849144+0000","last_unstale":"2023-08-11T15:51:06.846267+0000","last_undegraded":"2023-08-11T15:51:06.846267+0000","last_fullsized":"2023-08-11T15:51:06.846267+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:19:51.734914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643988+0000","last_change":"2023-08-11T15:51:00.743098+0000","last_active":"2023-08-11T15:51:08.643988+0000","last_peered":"2023-08-11T15:51:08.643988+0000","last_clean":"2023-08-11T15:51:08.643988+0000","last_became_active":"2023-08-11T15:51:00.742606+0000","last_became_peered":"2023-08-11T15:51:00.742606+0000","last_unstale":"2023-08-11T15:51:08.643988+0000","last_undegraded":"2023-08-11T15:51:08.643988+0000","last_fullsized":"2023-08-11T15:51:08.643988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:38:17.546002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278685+0000","last_change":"2023-08-11T15:51:01.746170+0000","last_active":"2023-08-11T15:51:07.278685+0000","last_peered":"2023-08-11T15:51:07.278685+0000","last_clean":"2023-08-11T15:51:07.278685+0000","last_became_active":"2023-08-11T15:51:01.745672+0000","last_became_peered":"2023-08-11T15:51:01.745672+0000","last_unstale":"2023-08-11T15:51:07.278685+0000","last_undegraded":"2023-08-11T15:51:07.278685+0000","last_fullsized":"2023-08-11T15:51:07.278685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:20:27.436792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278398+0000","last_change":"2023-08-11T15:51:05.844332+0000","last_active":"2023-08-11T15:51:07.278398+0000","last_peered":"2023-08-11T15:51:07.278398+0000","last_clean":"2023-08-11T15:51:07.278398+0000","last_became_active":"2023-08-11T15:51:05.841336+0000","last_became_peered":"2023-08-11T15:51:05.841336+0000","last_unstale":"2023-08-11T15:51:07.278398+0000","last_undegraded":"2023-08-11T15:51:07.278398+0000","last_fullsized":"2023-08-11T15:51:07.278398+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:19:51.734914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844582+0000","last_change":"2023-08-11T15:51:00.750609+0000","last_active":"2023-08-11T15:51:06.844582+0000","last_peered":"2023-08-11T15:51:06.844582+0000","last_clean":"2023-08-11T15:51:06.844582+0000","last_became_active":"2023-08-11T15:51:00.750277+0000","last_became_peered":"2023-08-11T15:51:00.750277+0000","last_unstale":"2023-08-11T15:51:06.844582+0000","last_undegraded":"2023-08-11T15:51:06.844582+0000","last_fullsized":"2023-08-11T15:51:06.844582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:19.559486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841973+0000","last_change":"2023-08-11T15:51:01.750332+0000","last_active":"2023-08-11T15:51:06.841973+0000","last_peered":"2023-08-11T15:51:06.841973+0000","last_clean":"2023-08-11T15:51:06.841973+0000","last_became_active":"2023-08-11T15:51:01.749234+0000","last_became_peered":"2023-08-11T15:51:01.749234+0000","last_unstale":"2023-08-11T15:51:06.841973+0000","last_undegraded":"2023-08-11T15:51:06.841973+0000","last_fullsized":"2023-08-11T15:51:06.841973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:08.926778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839995+0000","last_change":"2023-08-11T15:51:00.745865+0000","last_active":"2023-08-11T15:51:06.839995+0000","last_peered":"2023-08-11T15:51:06.839995+0000","last_clean":"2023-08-11T15:51:06.839995+0000","last_became_active":"2023-08-11T15:51:00.745469+0000","last_became_peered":"2023-08-11T15:51:00.745469+0000","last_unstale":"2023-08-11T15:51:06.839995+0000","last_undegraded":"2023-08-11T15:51:06.839995+0000","last_fullsized":"2023-08-11T15:51:06.839995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:25:15.827962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845118+0000","last_change":"2023-08-11T15:51:01.740388+0000","last_active":"2023-08-11T15:51:06.845118+0000","last_peered":"2023-08-11T15:51:06.845118+0000","last_clean":"2023-08-11T15:51:06.845118+0000","last_became_active":"2023-08-11T15:51:01.739972+0000","last_became_peered":"2023-08-11T15:51:01.739972+0000","last_unstale":"2023-08-11T15:51:06.845118+0000","last_undegraded":"2023-08-11T15:51:06.845118+0000","last_fullsized":"2023-08-11T15:51:06.845118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:41.231851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277756+0000","last_change":"2023-08-11T15:51:05.847377+0000","last_active":"2023-08-11T15:51:07.277756+0000","last_peered":"2023-08-11T15:51:07.277756+0000","last_clean":"2023-08-11T15:51:07.277756+0000","last_became_active":"2023-08-11T15:51:05.843902+0000","last_became_peered":"2023-08-11T15:51:05.843902+0000","last_unstale":"2023-08-11T15:51:07.277756+0000","last_undegraded":"2023-08-11T15:51:07.277756+0000","last_fullsized":"2023-08-11T15:51:07.277756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:45.327626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843318+0000","last_change":"2023-08-11T15:51:00.753432+0000","last_active":"2023-08-11T15:51:06.843318+0000","last_peered":"2023-08-11T15:51:06.843318+0000","last_clean":"2023-08-11T15:51:06.843318+0000","last_became_active":"2023-08-11T15:51:00.753099+0000","last_became_peered":"2023-08-11T15:51:00.753099+0000","last_unstale":"2023-08-11T15:51:06.843318+0000","last_undegraded":"2023-08-11T15:51:06.843318+0000","last_fullsized":"2023-08-11T15:51:06.843318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:44.445917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278374+0000","last_change":"2023-08-11T15:51:01.754975+0000","last_active":"2023-08-11T15:51:07.278374+0000","last_peered":"2023-08-11T15:51:07.278374+0000","last_clean":"2023-08-11T15:51:07.278374+0000","last_became_active":"2023-08-11T15:51:01.754617+0000","last_became_peered":"2023-08-11T15:51:01.754617+0000","last_unstale":"2023-08-11T15:51:07.278374+0000","last_undegraded":"2023-08-11T15:51:07.278374+0000","last_fullsized":"2023-08-11T15:51:07.278374+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:20.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844645+0000","last_change":"2023-08-11T15:51:05.841963+0000","last_active":"2023-08-11T15:51:06.844645+0000","last_peered":"2023-08-11T15:51:06.844645+0000","last_clean":"2023-08-11T15:51:06.844645+0000","last_became_active":"2023-08-11T15:51:05.841100+0000","last_became_peered":"2023-08-11T15:51:05.841100+0000","last_unstale":"2023-08-11T15:51:06.844645+0000","last_undegraded":"2023-08-11T15:51:06.844645+0000","last_fullsized":"2023-08-11T15:51:06.844645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:49.553888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643596+0000","last_change":"2023-08-11T15:51:01.740691+0000","last_active":"2023-08-11T15:51:08.643596+0000","last_peered":"2023-08-11T15:51:08.643596+0000","last_clean":"2023-08-11T15:51:08.643596+0000","last_became_active":"2023-08-11T15:51:01.740411+0000","last_became_peered":"2023-08-11T15:51:01.740411+0000","last_unstale":"2023-08-11T15:51:08.643596+0000","last_undegraded":"2023-08-11T15:51:08.643596+0000","last_fullsized":"2023-08-11T15:51:08.643596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:14:24.513369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276978+0000","last_change":"2023-08-11T15:51:01.754967+0000","last_active":"2023-08-11T15:51:07.276978+0000","last_peered":"2023-08-11T15:51:07.276978+0000","last_clean":"2023-08-11T15:51:07.276978+0000","last_became_active":"2023-08-11T15:51:01.754670+0000","last_became_peered":"2023-08-11T15:51:01.754670+0000","last_unstale":"2023-08-11T15:51:07.276978+0000","last_undegraded":"2023-08-11T15:51:07.276978+0000","last_fullsized":"2023-08-11T15:51:07.276978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029518+0000","last_change":"2023-08-11T15:51:01.743911+0000","last_active":"2023-08-11T15:51:08.029518+0000","last_peered":"2023-08-11T15:51:08.029518+0000","last_clean":"2023-08-11T15:51:08.029518+0000","last_became_active":"2023-08-11T15:51:01.743608+0000","last_became_peered":"2023-08-11T15:51:01.743608+0000","last_unstale":"2023-08-11T15:51:08.029518+0000","last_undegraded":"2023-08-11T15:51:08.029518+0000","last_fullsized":"2023-08-11T15:51:08.029518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:45:30.132181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277768+0000","last_change":"2023-08-11T15:51:01.748054+0000","last_active":"2023-08-11T15:51:07.277768+0000","last_peered":"2023-08-11T15:51:07.277768+0000","last_clean":"2023-08-11T15:51:07.277768+0000","last_became_active":"2023-08-11T15:51:01.747751+0000","last_became_peered":"2023-08-11T15:51:01.747751+0000","last_unstale":"2023-08-11T15:51:07.277768+0000","last_undegraded":"2023-08-11T15:51:07.277768+0000","last_fullsized":"2023-08-11T15:51:07.277768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:33.655411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450215+0000","last_change":"2023-08-11T15:51:01.745398+0000","last_active":"2023-08-11T15:51:07.450215+0000","last_peered":"2023-08-11T15:51:07.450215+0000","last_clean":"2023-08-11T15:51:07.450215+0000","last_became_active":"2023-08-11T15:51:01.745013+0000","last_became_peered":"2023-08-11T15:51:01.745013+0000","last_unstale":"2023-08-11T15:51:07.450215+0000","last_undegraded":"2023-08-11T15:51:07.450215+0000","last_fullsized":"2023-08-11T15:51:07.450215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:37:45.341896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844793+0000","last_change":"2023-08-11T15:51:01.747243+0000","last_active":"2023-08-11T15:51:06.844793+0000","last_peered":"2023-08-11T15:51:06.844793+0000","last_clean":"2023-08-11T15:51:06.844793+0000","last_became_active":"2023-08-11T15:51:01.746905+0000","last_became_peered":"2023-08-11T15:51:01.746905+0000","last_unstale":"2023-08-11T15:51:06.844793+0000","last_undegraded":"2023-08-11T15:51:06.844793+0000","last_fullsized":"2023-08-11T15:51:06.844793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450600+0000","last_change":"2023-08-11T15:51:01.743655+0000","last_active":"2023-08-11T15:51:07.450600+0000","last_peered":"2023-08-11T15:51:07.450600+0000","last_clean":"2023-08-11T15:51:07.450600+0000","last_became_active":"2023-08-11T15:51:01.743308+0000","last_became_peered":"2023-08-11T15:51:01.743308+0000","last_unstale":"2023-08-11T15:51:07.450600+0000","last_undegraded":"2023-08-11T15:51:07.450600+0000","last_fullsized":"2023-08-11T15:51:07.450600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:08:59.509573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450686+0000","last_change":"2023-08-11T15:51:01.747270+0000","last_active":"2023-08-11T15:51:07.450686+0000","last_peered":"2023-08-11T15:51:07.450686+0000","last_clean":"2023-08-11T15:51:07.450686+0000","last_became_active":"2023-08-11T15:51:01.746924+0000","last_became_peered":"2023-08-11T15:51:01.746924+0000","last_unstale":"2023-08-11T15:51:07.450686+0000","last_undegraded":"2023-08-11T15:51:07.450686+0000","last_fullsized":"2023-08-11T15:51:07.450686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:07:01.100072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842527+0000","last_change":"2023-08-11T15:51:01.750707+0000","last_active":"2023-08-11T15:51:06.842527+0000","last_peered":"2023-08-11T15:51:06.842527+0000","last_clean":"2023-08-11T15:51:06.842527+0000","last_became_active":"2023-08-11T15:51:01.750243+0000","last_became_peered":"2023-08-11T15:51:01.750243+0000","last_unstale":"2023-08-11T15:51:06.842527+0000","last_undegraded":"2023-08-11T15:51:06.842527+0000","last_fullsized":"2023-08-11T15:51:06.842527+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276812+0000","last_change":"2023-08-11T15:51:01.744825+0000","last_active":"2023-08-11T15:51:07.276812+0000","last_peered":"2023-08-11T15:51:07.276812+0000","last_clean":"2023-08-11T15:51:07.276812+0000","last_became_active":"2023-08-11T15:51:01.744446+0000","last_became_peered":"2023-08-11T15:51:01.744446+0000","last_unstale":"2023-08-11T15:51:07.276812+0000","last_undegraded":"2023-08-11T15:51:07.276812+0000","last_fullsized":"2023-08-11T15:51:07.276812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:08.520981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839890+0000","last_change":"2023-08-11T15:51:01.754226+0000","last_active":"2023-08-11T15:51:06.839890+0000","last_peered":"2023-08-11T15:51:06.839890+0000","last_clean":"2023-08-11T15:51:06.839890+0000","last_became_active":"2023-08-11T15:51:01.753864+0000","last_became_peered":"2023-08-11T15:51:01.753864+0000","last_unstale":"2023-08-11T15:51:06.839890+0000","last_undegraded":"2023-08-11T15:51:06.839890+0000","last_fullsized":"2023-08-11T15:51:06.839890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:42:42.499555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843198+0000","last_change":"2023-08-11T15:51:01.751698+0000","last_active":"2023-08-11T15:51:06.843198+0000","last_peered":"2023-08-11T15:51:06.843198+0000","last_clean":"2023-08-11T15:51:06.843198+0000","last_became_active":"2023-08-11T15:51:01.751183+0000","last_became_peered":"2023-08-11T15:51:01.751183+0000","last_unstale":"2023-08-11T15:51:06.843198+0000","last_undegraded":"2023-08-11T15:51:06.843198+0000","last_fullsized":"2023-08-11T15:51:06.843198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029934+0000","last_change":"2023-08-11T15:51:01.743064+0000","last_active":"2023-08-11T15:51:08.029934+0000","last_peered":"2023-08-11T15:51:08.029934+0000","last_clean":"2023-08-11T15:51:08.029934+0000","last_became_active":"2023-08-11T15:51:01.742783+0000","last_became_peered":"2023-08-11T15:51:01.742783+0000","last_unstale":"2023-08-11T15:51:08.029934+0000","last_undegraded":"2023-08-11T15:51:08.029934+0000","last_fullsized":"2023-08-11T15:51:08.029934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:25:56.529094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574852+0000","last_change":"2023-08-11T15:51:01.746080+0000","last_active":"2023-08-11T15:51:08.574852+0000","last_peered":"2023-08-11T15:51:08.574852+0000","last_clean":"2023-08-11T15:51:08.574852+0000","last_became_active":"2023-08-11T15:51:01.745366+0000","last_became_peered":"2023-08-11T15:51:01.745366+0000","last_unstale":"2023-08-11T15:51:08.574852+0000","last_undegraded":"2023-08-11T15:51:08.574852+0000","last_fullsized":"2023-08-11T15:51:08.574852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:58:33.514033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030489+0000","last_change":"2023-08-11T15:51:01.735183+0000","last_active":"2023-08-11T15:51:08.030489+0000","last_peered":"2023-08-11T15:51:08.030489+0000","last_clean":"2023-08-11T15:51:08.030489+0000","last_became_active":"2023-08-11T15:51:01.734771+0000","last_became_peered":"2023-08-11T15:51:01.734771+0000","last_unstale":"2023-08-11T15:51:08.030489+0000","last_undegraded":"2023-08-11T15:51:08.030489+0000","last_fullsized":"2023-08-11T15:51:08.030489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:28:21.255388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842288+0000","last_change":"2023-08-11T15:51:01.753111+0000","last_active":"2023-08-11T15:51:06.842288+0000","last_peered":"2023-08-11T15:51:06.842288+0000","last_clean":"2023-08-11T15:51:06.842288+0000","last_became_active":"2023-08-11T15:51:01.752792+0000","last_became_peered":"2023-08-11T15:51:01.752792+0000","last_unstale":"2023-08-11T15:51:06.842288+0000","last_undegraded":"2023-08-11T15:51:06.842288+0000","last_fullsized":"2023-08-11T15:51:06.842288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028692+0000","last_change":"2023-08-11T15:51:01.737005+0000","last_active":"2023-08-11T15:51:08.028692+0000","last_peered":"2023-08-11T15:51:08.028692+0000","last_clean":"2023-08-11T15:51:08.028692+0000","last_became_active":"2023-08-11T15:51:01.736669+0000","last_became_peered":"2023-08-11T15:51:01.736669+0000","last_unstale":"2023-08-11T15:51:08.028692+0000","last_undegraded":"2023-08-11T15:51:08.028692+0000","last_fullsized":"2023-08-11T15:51:08.028692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:10:00.491572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844298+0000","last_change":"2023-08-11T15:51:01.755893+0000","last_active":"2023-08-11T15:51:06.844298+0000","last_peered":"2023-08-11T15:51:06.844298+0000","last_clean":"2023-08-11T15:51:06.844298+0000","last_became_active":"2023-08-11T15:51:01.755182+0000","last_became_peered":"2023-08-11T15:51:01.755182+0000","last_unstale":"2023-08-11T15:51:06.844298+0000","last_undegraded":"2023-08-11T15:51:06.844298+0000","last_fullsized":"2023-08-11T15:51:06.844298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:03.472522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842189+0000","last_change":"2023-08-11T15:51:01.739785+0000","last_active":"2023-08-11T15:51:06.842189+0000","last_peered":"2023-08-11T15:51:06.842189+0000","last_clean":"2023-08-11T15:51:06.842189+0000","last_became_active":"2023-08-11T15:51:01.739500+0000","last_became_peered":"2023-08-11T15:51:01.739500+0000","last_unstale":"2023-08-11T15:51:06.842189+0000","last_undegraded":"2023-08-11T15:51:06.842189+0000","last_fullsized":"2023-08-11T15:51:06.842189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574594+0000","last_change":"2023-08-11T15:51:01.732565+0000","last_active":"2023-08-11T15:51:08.574594+0000","last_peered":"2023-08-11T15:51:08.574594+0000","last_clean":"2023-08-11T15:51:08.574594+0000","last_became_active":"2023-08-11T15:51:01.732076+0000","last_became_peered":"2023-08-11T15:51:01.732076+0000","last_unstale":"2023-08-11T15:51:08.574594+0000","last_undegraded":"2023-08-11T15:51:08.574594+0000","last_fullsized":"2023-08-11T15:51:08.574594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:20.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575381+0000","last_change":"2023-08-11T15:51:01.736603+0000","last_active":"2023-08-11T15:51:08.575381+0000","last_peered":"2023-08-11T15:51:08.575381+0000","last_clean":"2023-08-11T15:51:08.575381+0000","last_became_active":"2023-08-11T15:51:01.735835+0000","last_became_peered":"2023-08-11T15:51:01.735835+0000","last_unstale":"2023-08-11T15:51:08.575381+0000","last_undegraded":"2023-08-11T15:51:08.575381+0000","last_fullsized":"2023-08-11T15:51:08.575381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:07.026259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030580+0000","last_change":"2023-08-11T15:51:01.752520+0000","last_active":"2023-08-11T15:51:08.030580+0000","last_peered":"2023-08-11T15:51:08.030580+0000","last_clean":"2023-08-11T15:51:08.030580+0000","last_became_active":"2023-08-11T15:51:01.752203+0000","last_became_peered":"2023-08-11T15:51:01.752203+0000","last_unstale":"2023-08-11T15:51:08.030580+0000","last_undegraded":"2023-08-11T15:51:08.030580+0000","last_fullsized":"2023-08-11T15:51:08.030580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:29:45.923242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845432+0000","last_change":"2023-08-11T15:51:01.737002+0000","last_active":"2023-08-11T15:51:06.845432+0000","last_peered":"2023-08-11T15:51:06.845432+0000","last_clean":"2023-08-11T15:51:06.845432+0000","last_became_active":"2023-08-11T15:51:01.736720+0000","last_became_peered":"2023-08-11T15:51:01.736720+0000","last_unstale":"2023-08-11T15:51:06.845432+0000","last_undegraded":"2023-08-11T15:51:06.845432+0000","last_fullsized":"2023-08-11T15:51:06.845432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:20.571682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841857+0000","last_change":"2023-08-11T15:51:01.752888+0000","last_active":"2023-08-11T15:51:06.841857+0000","last_peered":"2023-08-11T15:51:06.841857+0000","last_clean":"2023-08-11T15:51:06.841857+0000","last_became_active":"2023-08-11T15:51:01.752593+0000","last_became_peered":"2023-08-11T15:51:01.752593+0000","last_unstale":"2023-08-11T15:51:06.841857+0000","last_undegraded":"2023-08-11T15:51:06.841857+0000","last_fullsized":"2023-08-11T15:51:06.841857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:39:00.899813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841369+0000","last_change":"2023-08-11T15:51:01.741200+0000","last_active":"2023-08-11T15:51:06.841369+0000","last_peered":"2023-08-11T15:51:06.841369+0000","last_clean":"2023-08-11T15:51:06.841369+0000","last_became_active":"2023-08-11T15:51:01.738593+0000","last_became_peered":"2023-08-11T15:51:01.738593+0000","last_unstale":"2023-08-11T15:51:06.841369+0000","last_undegraded":"2023-08-11T15:51:06.841369+0000","last_fullsized":"2023-08-11T15:51:06.841369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:02:09.390191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844003+0000","last_change":"2023-08-11T15:51:01.756072+0000","last_active":"2023-08-11T15:51:06.844003+0000","last_peered":"2023-08-11T15:51:06.844003+0000","last_clean":"2023-08-11T15:51:06.844003+0000","last_became_active":"2023-08-11T15:51:01.755522+0000","last_became_peered":"2023-08-11T15:51:01.755522+0000","last_unstale":"2023-08-11T15:51:06.844003+0000","last_undegraded":"2023-08-11T15:51:06.844003+0000","last_fullsized":"2023-08-11T15:51:06.844003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:41.231851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029178+0000","last_change":"2023-08-11T15:51:01.737005+0000","last_active":"2023-08-11T15:51:08.029178+0000","last_peered":"2023-08-11T15:51:08.029178+0000","last_clean":"2023-08-11T15:51:08.029178+0000","last_became_active":"2023-08-11T15:51:01.736657+0000","last_became_peered":"2023-08-11T15:51:01.736657+0000","last_unstale":"2023-08-11T15:51:08.029178+0000","last_undegraded":"2023-08-11T15:51:08.029178+0000","last_fullsized":"2023-08-11T15:51:08.029178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:20:52.818361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841216+0000","last_change":"2023-08-11T15:51:01.749800+0000","last_active":"2023-08-11T15:51:06.841216+0000","last_peered":"2023-08-11T15:51:06.841216+0000","last_clean":"2023-08-11T15:51:06.841216+0000","last_became_active":"2023-08-11T15:51:01.749436+0000","last_became_peered":"2023-08-11T15:51:01.749436+0000","last_unstale":"2023-08-11T15:51:06.841216+0000","last_undegraded":"2023-08-11T15:51:06.841216+0000","last_fullsized":"2023-08-11T15:51:06.841216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644342+0000","last_change":"2023-08-11T15:51:01.739883+0000","last_active":"2023-08-11T15:51:08.644342+0000","last_peered":"2023-08-11T15:51:08.644342+0000","last_clean":"2023-08-11T15:51:08.644342+0000","last_became_active":"2023-08-11T15:51:01.739490+0000","last_became_peered":"2023-08-11T15:51:01.739490+0000","last_unstale":"2023-08-11T15:51:08.644342+0000","last_undegraded":"2023-08-11T15:51:08.644342+0000","last_fullsized":"2023-08-11T15:51:08.644342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:19.428228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030296+0000","last_change":"2023-08-11T15:51:01.737093+0000","last_active":"2023-08-11T15:51:08.030296+0000","last_peered":"2023-08-11T15:51:08.030296+0000","last_clean":"2023-08-11T15:51:08.030296+0000","last_became_active":"2023-08-11T15:51:01.736803+0000","last_became_peered":"2023-08-11T15:51:01.736803+0000","last_unstale":"2023-08-11T15:51:08.030296+0000","last_undegraded":"2023-08-11T15:51:08.030296+0000","last_fullsized":"2023-08-11T15:51:08.030296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:50:46.864696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844939+0000","last_change":"2023-08-11T15:51:01.753163+0000","last_active":"2023-08-11T15:51:06.844939+0000","last_peered":"2023-08-11T15:51:06.844939+0000","last_clean":"2023-08-11T15:51:06.844939+0000","last_became_active":"2023-08-11T15:51:01.752836+0000","last_became_peered":"2023-08-11T15:51:01.752836+0000","last_unstale":"2023-08-11T15:51:06.844939+0000","last_undegraded":"2023-08-11T15:51:06.844939+0000","last_fullsized":"2023-08-11T15:51:06.844939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:15.195226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575171+0000","last_change":"2023-08-11T15:51:01.739059+0000","last_active":"2023-08-11T15:51:08.575171+0000","last_peered":"2023-08-11T15:51:08.575171+0000","last_clean":"2023-08-11T15:51:08.575171+0000","last_became_active":"2023-08-11T15:51:01.738739+0000","last_became_peered":"2023-08-11T15:51:01.738739+0000","last_unstale":"2023-08-11T15:51:08.575171+0000","last_undegraded":"2023-08-11T15:51:08.575171+0000","last_fullsized":"2023-08-11T15:51:08.575171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:11:40.167154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":921600,"data_stored":717621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5840000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":123848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":319488,"data_stored":127093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]}]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":123286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]}]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":897024,"data_stored":714146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.901}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.869}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:51:33.198 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:51:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.468+0000 7f50acb60640 1 Processor -- start 2023-08-11T15:51:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.469+0000 7f50acb60640 1 -- start start 2023-08-11T15:51:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.470+0000 7f50acb60640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f50a80b7d40 0x7f50a80d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.470+0000 7f50acb60640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a80b5170 con 0x7f50a815c1f0 2023-08-11T15:51:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.470+0000 7f50acb60640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a80b81a0 con 0x7f50a80d09f0 2023-08-11T15:51:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.470+0000 7f50acb60640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a815d080 con 0x7f50a80b7d40 2023-08-11T15:51:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.470+0000 7f50a77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f50a815c1f0 0x7f50a815c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48552/0 (socket says 172.21.15.27:48552) 2023-08-11T15:51:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.470+0000 7f50a77fe640 1 -- 172.21.15.27:0/394352113 learned_addr learned my addr 172.21.15.27:0/394352113 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.470+0000 7f509effd640 1 --2- 172.21.15.27:0/394352113 >> v2:172.21.15.119:3300/0 conn(0x7f50a80b7d40 0x7f50a80d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001902256 0 0) 0x7f50a80b5170 con 0x7f50a815c1f0 2023-08-11T15:51:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5094003520 con 0x7f50a815c1f0 2023-08-11T15:51:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2646358561 0 0) 0x7f5094003520 con 0x7f50a815c1f0 2023-08-11T15:51:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 >> v2:172.21.15.119:3300/0 conn(0x7f50a80b7d40 msgr2=0x7f50a80d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 --2- 172.21.15.27:0/394352113 >> v2:172.21.15.119:3300/0 conn(0x7f50a80b7d40 0x7f50a80d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 >> v1:172.21.15.119:6789/0 conn(0x7f50a80d09f0 legacy=0x7f50a8160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a815d360 con 0x7f50a815c1f0 2023-08-11T15:51:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f5098001670 con 0x7f50a815c1f0 2023-08-11T15:51:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 >> v1:172.21.15.27:6789/0 conn(0x7f50a815c1f0 legacy=0x7f50a815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094003970 0x7f5094003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f50940047c0 0x7f5094004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5094005180 0x7f509400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a815d080 con 0x7f5094005180 2023-08-11T15:51:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509400c030 con 0x7f5094003970 2023-08-11T15:51:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.471+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509400c1f0 con 0x7f50940047c0 2023-08-11T15:51:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.472+0000 7f50a7fff640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5094005180 0x7f509400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.472+0000 7f50a77fe640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094003970 0x7f5094003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.472+0000 7f509effd640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f50940047c0 0x7f5094004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.472+0000 7f50a7fff640 1 -- 172.21.15.27:0/394352113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f50940047c0 msgr2=0x7f5094004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.472+0000 7f50a7fff640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f50940047c0 0x7f5094004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.472+0000 7f50a7fff640 1 -- 172.21.15.27:0/394352113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094003970 msgr2=0x7f5094003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.472+0000 7f50a7fff640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094003970 0x7f5094003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.472+0000 7f50a7fff640 1 -- 172.21.15.27:0/394352113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094003520 con 0x7f5094005180 2023-08-11T15:51:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.473+0000 7f50a7fff640 1 -- 172.21.15.27:0/394352113 reap_dead start 2023-08-11T15:51:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.473+0000 7f50a7fff640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5094005180 0x7f509400b980 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f50a0001750 tx=0x7f50940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0b02661d6c4545 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.473+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a00052d0 con 0x7f5094005180 2023-08-11T15:51:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.473+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a002e050 con 0x7f5094005180 2023-08-11T15:51:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.473+0000 7f50a6ffd640 1 -- 172.21.15.27:0/394352113 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0003830 con 0x7f5094005180 2023-08-11T15:51:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.474+0000 7f50acb60640 1 -- 172.21.15.27:0/394352113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5094005180 msgr2=0x7f509400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.474+0000 7f50acb60640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5094005180 0x7f509400b980 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f50a0001750 tx=0x7f50940057f0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.474+0000 7f50acb60640 1 -- 172.21.15.27:0/394352113 shutdown_connections 2023-08-11T15:51:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.474+0000 7f50acb60640 1 --2- 172.21.15.27:0/394352113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5094005180 0x7f509400b980 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.474+0000 7f50acb60640 1 -- 172.21.15.27:0/394352113 >> 172.21.15.27:0/394352113 conn(0x7f50a806a250 msgr2=0x7f50a806a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.474+0000 7f50acb60640 1 -- 172.21.15.27:0/394352113 shutdown_connections 2023-08-11T15:51:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.474+0000 7f50acb60640 1 -- 172.21.15.27:0/394352113 wait complete. 2023-08-11T15:51:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.476+0000 7f50acb60640 1 Processor -- start 2023-08-11T15:51:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.476+0000 7f50acb60640 1 -- start start 2023-08-11T15:51:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f50acb60640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5094005180 0x7f50a814ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f50acb60640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a814e970 0x7f50a814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f50acb60640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a80cb250 0x7f50a80cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f50acb60640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a814d210 con 0x7f50a814e970 2023-08-11T15:51:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f50acb60640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a80c5a40 con 0x7f50a80cb250 2023-08-11T15:51:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f50acb60640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a80c5d20 con 0x7f5094005180 2023-08-11T15:51:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f509effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a814e970 0x7f50a814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f509effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a814e970 0x7f50a814edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46540/0 (socket says 172.21.15.27:46540) 2023-08-11T15:51:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f509effd640 1 -- 172.21.15.27:0/1291087046 learned_addr learned my addr 172.21.15.27:0/1291087046 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f50a77fe640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5094005180 0x7f50a814ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.477+0000 7f50a7fff640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a80cb250 0x7f50a80cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f509effd640 1 -- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5094005180 msgr2=0x7f50a814ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f509effd640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5094005180 0x7f50a814ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f509effd640 1 -- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a80cb250 msgr2=0x7f50a80cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f509effd640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a80cb250 0x7f50a80cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f509effd640 1 -- 172.21.15.27:0/1291087046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a80c6000 con 0x7f50a814e970 2023-08-11T15:51:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f50a77fe640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5094005180 0x7f50a814ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f509effd640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a814e970 0x7f50a814edf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5090009fb0 tx=0x7f50900115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f50a4ff9640 1 -- 172.21.15.27:0/1291087046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000e030 con 0x7f50a814e970 2023-08-11T15:51:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f50a4ff9640 1 -- 172.21.15.27:0/1291087046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5090016470 con 0x7f50a814e970 2023-08-11T15:51:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f50a4ff9640 1 -- 172.21.15.27:0/1291087046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090016820 con 0x7f50a814e970 2023-08-11T15:51:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f50acb60640 1 -- 172.21.15.27:0/1291087046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a80c62e0 con 0x7f50a814e970 2023-08-11T15:51:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.478+0000 7f50acb60640 1 -- 172.21.15.27:0/1291087046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a80c68a0 con 0x7f50a814e970 2023-08-11T15:51:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.479+0000 7f50acb60640 1 -- 172.21.15.27:0/1291087046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a80671d0 con 0x7f50a814e970 2023-08-11T15:51:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.480+0000 7f50a4ff9640 1 -- 172.21.15.27:0/1291087046 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5090016980 con 0x7f50a814e970 2023-08-11T15:51:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.481+0000 7f50a4ff9640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f508806a050 0x7f508806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.481+0000 7f50a4ff9640 1 -- 172.21.15.27:0/1291087046 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f509000f070 con 0x7f50a814e970 2023-08-11T15:51:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.482+0000 7f50a77fe640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f508806a050 0x7f508806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.483+0000 7f50a77fe640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f508806a050 0x7f508806c510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5098001eb0 tx=0x7f5098000f50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.486+0000 7f50a4ff9640 1 -- 172.21.15.27:0/1291087046 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f5090054c20 con 0x7f50a814e970 2023-08-11T15:51:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.675+0000 7f50acb60640 1 -- 172.21.15.27:0/1291087046 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50a815fff0 con 0x7f508806a050 2023-08-11T15:51:33.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.702+0000 7f50a4ff9640 1 -- 172.21.15.27:0/1291087046 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367163 (secure 0 0 0) 0x7f5098005e60 con 0x7f508806a050 2023-08-11T15:51:33.705 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:51:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.709+0000 7f50acb60640 1 -- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f508806a050 msgr2=0x7f508806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.709+0000 7f50acb60640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f508806a050 0x7f508806c510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5098001eb0 tx=0x7f5098000f50 comp rx=0 tx=0).stop 2023-08-11T15:51:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.709+0000 7f50acb60640 1 -- 172.21.15.27:0/1291087046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a814e970 msgr2=0x7f50a814edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.709+0000 7f50acb60640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a814e970 0x7f50a814edf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5090009fb0 tx=0x7f50900115a0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.710+0000 7f50acb60640 1 -- 172.21.15.27:0/1291087046 shutdown_connections 2023-08-11T15:51:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.710+0000 7f50acb60640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f508806a050 0x7f508806c510 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.710+0000 7f50acb60640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a80cb250 0x7f50a80cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.710+0000 7f50acb60640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a814e970 0x7f50a814edf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.710+0000 7f50acb60640 1 --2- 172.21.15.27:0/1291087046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5094005180 0x7f50a814ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.710+0000 7f50acb60640 1 -- 172.21.15.27:0/1291087046 >> 172.21.15.27:0/1291087046 conn(0x7f50a806a250 msgr2=0x7f50a8158580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.710+0000 7f50acb60640 1 -- 172.21.15.27:0/1291087046 shutdown_connections 2023-08-11T15:51:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:33.710+0000 7f50acb60640 1 -- 172.21.15.27:0/1291087046 wait complete. 2023-08-11T15:51:33.733 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-08-11T15:51:32.722468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232028,"kb_used_data":3560,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754742692,"statfs":{"total":773094113280,"available":772856516608,"internally_reserved":0,"allocated":3645440,"data_stored":2175582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005576"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449971+0000","last_change":"2023-08-11T15:51:05.843998+0000","last_active":"2023-08-11T15:51:07.449971+0000","last_peered":"2023-08-11T15:51:07.449971+0000","last_clean":"2023-08-11T15:51:07.449971+0000","last_became_active":"2023-08-11T15:51:05.843613+0000","last_became_peered":"2023-08-11T15:51:05.843613+0000","last_unstale":"2023-08-11T15:51:07.449971+0000","last_undegraded":"2023-08-11T15:51:07.449971+0000","last_fullsized":"2023-08-11T15:51:07.449971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:35:45.480000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842354+0000","last_change":"2023-08-11T15:51:00.746010+0000","last_active":"2023-08-11T15:51:06.842354+0000","last_peered":"2023-08-11T15:51:06.842354+0000","last_clean":"2023-08-11T15:51:06.842354+0000","last_became_active":"2023-08-11T15:51:00.745732+0000","last_became_peered":"2023-08-11T15:51:00.745732+0000","last_unstale":"2023-08-11T15:51:06.842354+0000","last_undegraded":"2023-08-11T15:51:06.842354+0000","last_fullsized":"2023-08-11T15:51:06.842354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:02.960249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028896+0000","last_change":"2023-08-11T15:51:01.746812+0000","last_active":"2023-08-11T15:51:08.028896+0000","last_peered":"2023-08-11T15:51:08.028896+0000","last_clean":"2023-08-11T15:51:08.028896+0000","last_became_active":"2023-08-11T15:51:01.745729+0000","last_became_peered":"2023-08-11T15:51:01.745729+0000","last_unstale":"2023-08-11T15:51:08.028896+0000","last_undegraded":"2023-08-11T15:51:08.028896+0000","last_fullsized":"2023-08-11T15:51:08.028896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:46.849896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028532+0000","last_change":"2023-08-11T15:51:05.846118+0000","last_active":"2023-08-11T15:51:08.028532+0000","last_peered":"2023-08-11T15:51:08.028532+0000","last_clean":"2023-08-11T15:51:08.028532+0000","last_became_active":"2023-08-11T15:51:05.845703+0000","last_became_peered":"2023-08-11T15:51:05.845703+0000","last_unstale":"2023-08-11T15:51:08.028532+0000","last_undegraded":"2023-08-11T15:51:08.028532+0000","last_fullsized":"2023-08-11T15:51:08.028532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:15.472354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278289+0000","last_change":"2023-08-11T15:51:00.732689+0000","last_active":"2023-08-11T15:51:07.278289+0000","last_peered":"2023-08-11T15:51:07.278289+0000","last_clean":"2023-08-11T15:51:07.278289+0000","last_became_active":"2023-08-11T15:51:00.732185+0000","last_became_peered":"2023-08-11T15:51:00.732185+0000","last_unstale":"2023-08-11T15:51:07.278289+0000","last_undegraded":"2023-08-11T15:51:07.278289+0000","last_fullsized":"2023-08-11T15:51:07.278289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:19.559486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277505+0000","last_change":"2023-08-11T15:51:01.740441+0000","last_active":"2023-08-11T15:51:07.277505+0000","last_peered":"2023-08-11T15:51:07.277505+0000","last_clean":"2023-08-11T15:51:07.277505+0000","last_became_active":"2023-08-11T15:51:01.740104+0000","last_became_peered":"2023-08-11T15:51:01.740104+0000","last_unstale":"2023-08-11T15:51:07.277505+0000","last_undegraded":"2023-08-11T15:51:07.277505+0000","last_fullsized":"2023-08-11T15:51:07.277505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842536+0000","last_change":"2023-08-11T15:51:05.850642+0000","last_active":"2023-08-11T15:51:06.842536+0000","last_peered":"2023-08-11T15:51:06.842536+0000","last_clean":"2023-08-11T15:51:06.842536+0000","last_became_active":"2023-08-11T15:51:05.850276+0000","last_became_peered":"2023-08-11T15:51:05.850276+0000","last_unstale":"2023-08-11T15:51:06.842536+0000","last_undegraded":"2023-08-11T15:51:06.842536+0000","last_fullsized":"2023-08-11T15:51:06.842536+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:06:02.062687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845184+0000","last_change":"2023-08-11T15:51:00.753395+0000","last_active":"2023-08-11T15:51:06.845184+0000","last_peered":"2023-08-11T15:51:06.845184+0000","last_clean":"2023-08-11T15:51:06.845184+0000","last_became_active":"2023-08-11T15:51:00.753094+0000","last_became_peered":"2023-08-11T15:51:00.753094+0000","last_unstale":"2023-08-11T15:51:06.845184+0000","last_undegraded":"2023-08-11T15:51:06.845184+0000","last_fullsized":"2023-08-11T15:51:06.845184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:32.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029427+0000","last_change":"2023-08-11T15:51:01.748176+0000","last_active":"2023-08-11T15:51:08.029427+0000","last_peered":"2023-08-11T15:51:08.029427+0000","last_clean":"2023-08-11T15:51:08.029427+0000","last_became_active":"2023-08-11T15:51:01.747880+0000","last_became_peered":"2023-08-11T15:51:01.747880+0000","last_unstale":"2023-08-11T15:51:08.029427+0000","last_undegraded":"2023-08-11T15:51:08.029427+0000","last_fullsized":"2023-08-11T15:51:08.029427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:40:28.656635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842819+0000","last_change":"2023-08-11T15:51:05.854549+0000","last_active":"2023-08-11T15:51:06.842819+0000","last_peered":"2023-08-11T15:51:06.842819+0000","last_clean":"2023-08-11T15:51:06.842819+0000","last_became_active":"2023-08-11T15:51:05.853942+0000","last_became_peered":"2023-08-11T15:51:05.853942+0000","last_unstale":"2023-08-11T15:51:06.842819+0000","last_undegraded":"2023-08-11T15:51:06.842819+0000","last_fullsized":"2023-08-11T15:51:06.842819+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:12.616756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844270+0000","last_change":"2023-08-11T15:51:00.754010+0000","last_active":"2023-08-11T15:51:06.844270+0000","last_peered":"2023-08-11T15:51:06.844270+0000","last_clean":"2023-08-11T15:51:06.844270+0000","last_became_active":"2023-08-11T15:51:00.753640+0000","last_became_peered":"2023-08-11T15:51:00.753640+0000","last_unstale":"2023-08-11T15:51:06.844270+0000","last_undegraded":"2023-08-11T15:51:06.844270+0000","last_fullsized":"2023-08-11T15:51:06.844270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:59.887617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029380+0000","last_change":"2023-08-11T15:51:01.748240+0000","last_active":"2023-08-11T15:51:08.029380+0000","last_peered":"2023-08-11T15:51:08.029380+0000","last_clean":"2023-08-11T15:51:08.029380+0000","last_became_active":"2023-08-11T15:51:01.747946+0000","last_became_peered":"2023-08-11T15:51:01.747946+0000","last_unstale":"2023-08-11T15:51:08.029380+0000","last_undegraded":"2023-08-11T15:51:08.029380+0000","last_fullsized":"2023-08-11T15:51:08.029380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:08:40.170218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450286+0000","last_change":"2023-08-11T15:51:05.843794+0000","last_active":"2023-08-11T15:51:07.450286+0000","last_peered":"2023-08-11T15:51:07.450286+0000","last_clean":"2023-08-11T15:51:07.450286+0000","last_became_active":"2023-08-11T15:51:05.843006+0000","last_became_peered":"2023-08-11T15:51:05.843006+0000","last_unstale":"2023-08-11T15:51:07.450286+0000","last_undegraded":"2023-08-11T15:51:07.450286+0000","last_fullsized":"2023-08-11T15:51:07.450286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:53:03.920014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841384+0000","last_change":"2023-08-11T15:51:00.753341+0000","last_active":"2023-08-11T15:51:06.841384+0000","last_peered":"2023-08-11T15:51:06.841384+0000","last_clean":"2023-08-11T15:51:06.841384+0000","last_became_active":"2023-08-11T15:51:00.753070+0000","last_became_peered":"2023-08-11T15:51:00.753070+0000","last_unstale":"2023-08-11T15:51:06.841384+0000","last_undegraded":"2023-08-11T15:51:06.841384+0000","last_fullsized":"2023-08-11T15:51:06.841384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:07.508785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575377+0000","last_change":"2023-08-11T15:51:01.745198+0000","last_active":"2023-08-11T15:51:08.575377+0000","last_peered":"2023-08-11T15:51:08.575377+0000","last_clean":"2023-08-11T15:51:08.575377+0000","last_became_active":"2023-08-11T15:51:01.744845+0000","last_became_peered":"2023-08-11T15:51:01.744845+0000","last_unstale":"2023-08-11T15:51:08.575377+0000","last_undegraded":"2023-08-11T15:51:08.575377+0000","last_fullsized":"2023-08-11T15:51:08.575377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:07:54.433272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029435+0000","last_change":"2023-08-11T15:51:05.846476+0000","last_active":"2023-08-11T15:51:08.029435+0000","last_peered":"2023-08-11T15:51:08.029435+0000","last_clean":"2023-08-11T15:51:08.029435+0000","last_became_active":"2023-08-11T15:51:05.846010+0000","last_became_peered":"2023-08-11T15:51:05.846010+0000","last_unstale":"2023-08-11T15:51:08.029435+0000","last_undegraded":"2023-08-11T15:51:08.029435+0000","last_fullsized":"2023-08-11T15:51:08.029435+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:55:13.061478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029848+0000","last_change":"2023-08-11T15:51:00.748391+0000","last_active":"2023-08-11T15:51:08.029848+0000","last_peered":"2023-08-11T15:51:08.029848+0000","last_clean":"2023-08-11T15:51:08.029848+0000","last_became_active":"2023-08-11T15:51:00.744340+0000","last_became_peered":"2023-08-11T15:51:00.744340+0000","last_unstale":"2023-08-11T15:51:08.029848+0000","last_undegraded":"2023-08-11T15:51:08.029848+0000","last_fullsized":"2023-08-11T15:51:08.029848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:56:37.817338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278162+0000","last_change":"2023-08-11T15:51:01.749456+0000","last_active":"2023-08-11T15:51:07.278162+0000","last_peered":"2023-08-11T15:51:07.278162+0000","last_clean":"2023-08-11T15:51:07.278162+0000","last_became_active":"2023-08-11T15:51:01.749042+0000","last_became_peered":"2023-08-11T15:51:01.749042+0000","last_unstale":"2023-08-11T15:51:07.278162+0000","last_undegraded":"2023-08-11T15:51:07.278162+0000","last_fullsized":"2023-08-11T15:51:07.278162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:03.472522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644834+0000","last_change":"2023-08-11T15:51:05.849122+0000","last_active":"2023-08-11T15:51:08.644834+0000","last_peered":"2023-08-11T15:51:08.644834+0000","last_clean":"2023-08-11T15:51:08.644834+0000","last_became_active":"2023-08-11T15:51:05.848698+0000","last_became_peered":"2023-08-11T15:51:05.848698+0000","last_unstale":"2023-08-11T15:51:08.644834+0000","last_undegraded":"2023-08-11T15:51:08.644834+0000","last_fullsized":"2023-08-11T15:51:08.644834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:46:03.919402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449963+0000","last_change":"2023-08-11T15:51:00.743735+0000","last_active":"2023-08-11T15:51:07.449963+0000","last_peered":"2023-08-11T15:51:07.449963+0000","last_clean":"2023-08-11T15:51:07.449963+0000","last_became_active":"2023-08-11T15:51:00.743412+0000","last_became_peered":"2023-08-11T15:51:00.743412+0000","last_unstale":"2023-08-11T15:51:07.449963+0000","last_undegraded":"2023-08-11T15:51:07.449963+0000","last_fullsized":"2023-08-11T15:51:07.449963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:07.487770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841592+0000","last_change":"2023-08-11T15:51:01.740683+0000","last_active":"2023-08-11T15:51:06.841592+0000","last_peered":"2023-08-11T15:51:06.841592+0000","last_clean":"2023-08-11T15:51:06.841592+0000","last_became_active":"2023-08-11T15:51:01.740230+0000","last_became_peered":"2023-08-11T15:51:01.740230+0000","last_unstale":"2023-08-11T15:51:06.841592+0000","last_undegraded":"2023-08-11T15:51:06.841592+0000","last_fullsized":"2023-08-11T15:51:06.841592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:57.966913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841129+0000","last_change":"2023-08-11T15:51:05.843204+0000","last_active":"2023-08-11T15:51:06.841129+0000","last_peered":"2023-08-11T15:51:06.841129+0000","last_clean":"2023-08-11T15:51:06.841129+0000","last_became_active":"2023-08-11T15:51:05.842830+0000","last_became_peered":"2023-08-11T15:51:05.842830+0000","last_unstale":"2023-08-11T15:51:06.841129+0000","last_undegraded":"2023-08-11T15:51:06.841129+0000","last_fullsized":"2023-08-11T15:51:06.841129+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:13.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277138+0000","last_change":"2023-08-11T15:51:00.735703+0000","last_active":"2023-08-11T15:51:07.277138+0000","last_peered":"2023-08-11T15:51:07.277138+0000","last_clean":"2023-08-11T15:51:07.277138+0000","last_became_active":"2023-08-11T15:51:00.735384+0000","last_became_peered":"2023-08-11T15:51:00.735384+0000","last_unstale":"2023-08-11T15:51:07.277138+0000","last_undegraded":"2023-08-11T15:51:07.277138+0000","last_fullsized":"2023-08-11T15:51:07.277138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:59.887617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644911+0000","last_change":"2023-08-11T15:51:01.742945+0000","last_active":"2023-08-11T15:51:08.644911+0000","last_peered":"2023-08-11T15:51:08.644911+0000","last_clean":"2023-08-11T15:51:08.644911+0000","last_became_active":"2023-08-11T15:51:01.742477+0000","last_became_peered":"2023-08-11T15:51:01.742477+0000","last_unstale":"2023-08-11T15:51:08.644911+0000","last_undegraded":"2023-08-11T15:51:08.644911+0000","last_fullsized":"2023-08-11T15:51:08.644911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:17:49.108624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276720+0000","last_change":"2023-08-11T15:51:01.755598+0000","last_active":"2023-08-11T15:51:07.276720+0000","last_peered":"2023-08-11T15:51:07.276720+0000","last_clean":"2023-08-11T15:51:07.276720+0000","last_became_active":"2023-08-11T15:51:01.755329+0000","last_became_peered":"2023-08-11T15:51:01.755329+0000","last_unstale":"2023-08-11T15:51:07.276720+0000","last_undegraded":"2023-08-11T15:51:07.276720+0000","last_fullsized":"2023-08-11T15:51:07.276720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:09.551106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842608+0000","last_change":"2023-08-11T15:51:00.747156+0000","last_active":"2023-08-11T15:51:06.842608+0000","last_peered":"2023-08-11T15:51:06.842608+0000","last_clean":"2023-08-11T15:51:06.842608+0000","last_became_active":"2023-08-11T15:51:00.745449+0000","last_became_peered":"2023-08-11T15:51:00.745449+0000","last_unstale":"2023-08-11T15:51:06.842608+0000","last_undegraded":"2023-08-11T15:51:06.842608+0000","last_fullsized":"2023-08-11T15:51:06.842608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643560+0000","last_change":"2023-08-11T15:51:05.839469+0000","last_active":"2023-08-11T15:51:08.643560+0000","last_peered":"2023-08-11T15:51:08.643560+0000","last_clean":"2023-08-11T15:51:08.643560+0000","last_became_active":"2023-08-11T15:51:05.839035+0000","last_became_peered":"2023-08-11T15:51:05.839035+0000","last_unstale":"2023-08-11T15:51:08.643560+0000","last_undegraded":"2023-08-11T15:51:08.643560+0000","last_fullsized":"2023-08-11T15:51:08.643560+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:11.122033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276959+0000","last_change":"2023-08-11T15:51:01.748158+0000","last_active":"2023-08-11T15:51:07.276959+0000","last_peered":"2023-08-11T15:51:07.276959+0000","last_clean":"2023-08-11T15:51:07.276959+0000","last_became_active":"2023-08-11T15:51:01.747870+0000","last_became_peered":"2023-08-11T15:51:01.747870+0000","last_unstale":"2023-08-11T15:51:07.276959+0000","last_undegraded":"2023-08-11T15:51:07.276959+0000","last_fullsized":"2023-08-11T15:51:07.276959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:57.966913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845778+0000","last_change":"2023-08-11T15:51:00.739033+0000","last_active":"2023-08-11T15:51:06.845778+0000","last_peered":"2023-08-11T15:51:06.845778+0000","last_clean":"2023-08-11T15:51:06.845778+0000","last_became_active":"2023-08-11T15:51:00.738746+0000","last_became_peered":"2023-08-11T15:51:00.738746+0000","last_unstale":"2023-08-11T15:51:06.845778+0000","last_undegraded":"2023-08-11T15:51:06.845778+0000","last_fullsized":"2023-08-11T15:51:06.845778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841377+0000","last_change":"2023-08-11T15:51:05.853925+0000","last_active":"2023-08-11T15:51:06.841377+0000","last_peered":"2023-08-11T15:51:06.841377+0000","last_clean":"2023-08-11T15:51:06.841377+0000","last_became_active":"2023-08-11T15:51:05.853531+0000","last_became_peered":"2023-08-11T15:51:05.853531+0000","last_unstale":"2023-08-11T15:51:06.841377+0000","last_undegraded":"2023-08-11T15:51:06.841377+0000","last_fullsized":"2023-08-11T15:51:06.841377+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:13.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843271+0000","last_change":"2023-08-11T15:51:00.733126+0000","last_active":"2023-08-11T15:51:06.843271+0000","last_peered":"2023-08-11T15:51:06.843271+0000","last_clean":"2023-08-11T15:51:06.843271+0000","last_became_active":"2023-08-11T15:51:00.731695+0000","last_became_peered":"2023-08-11T15:51:00.731695+0000","last_unstale":"2023-08-11T15:51:06.843271+0000","last_undegraded":"2023-08-11T15:51:06.843271+0000","last_fullsized":"2023-08-11T15:51:06.843271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:07.914582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449817+0000","last_change":"2023-08-11T15:51:01.750498+0000","last_active":"2023-08-11T15:51:07.449817+0000","last_peered":"2023-08-11T15:51:07.449817+0000","last_clean":"2023-08-11T15:51:07.449817+0000","last_became_active":"2023-08-11T15:51:01.750204+0000","last_became_peered":"2023-08-11T15:51:01.750204+0000","last_unstale":"2023-08-11T15:51:07.449817+0000","last_undegraded":"2023-08-11T15:51:07.449817+0000","last_fullsized":"2023-08-11T15:51:07.449817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:30:40.844864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277300+0000","last_change":"2023-08-11T15:51:05.849252+0000","last_active":"2023-08-11T15:51:07.277300+0000","last_peered":"2023-08-11T15:51:07.277300+0000","last_clean":"2023-08-11T15:51:07.277300+0000","last_became_active":"2023-08-11T15:51:05.848107+0000","last_became_peered":"2023-08-11T15:51:05.848107+0000","last_unstale":"2023-08-11T15:51:07.277300+0000","last_undegraded":"2023-08-11T15:51:07.277300+0000","last_fullsized":"2023-08-11T15:51:07.277300+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:13.022552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:22.839046+0000","last_change":"2023-08-11T15:51:00.743771+0000","last_active":"2023-08-11T15:51:22.839046+0000","last_peered":"2023-08-11T15:51:22.839046+0000","last_clean":"2023-08-11T15:51:22.839046+0000","last_became_active":"2023-08-11T15:51:00.741410+0000","last_became_peered":"2023-08-11T15:51:00.741410+0000","last_unstale":"2023-08-11T15:51:22.839046+0000","last_undegraded":"2023-08-11T15:51:22.839046+0000","last_fullsized":"2023-08-11T15:51:22.839046+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:46:01.024956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029670+0000","last_change":"2023-08-11T15:51:01.742960+0000","last_active":"2023-08-11T15:51:08.029670+0000","last_peered":"2023-08-11T15:51:08.029670+0000","last_clean":"2023-08-11T15:51:08.029670+0000","last_became_active":"2023-08-11T15:51:01.742674+0000","last_became_peered":"2023-08-11T15:51:01.742674+0000","last_unstale":"2023-08-11T15:51:08.029670+0000","last_undegraded":"2023-08-11T15:51:08.029670+0000","last_fullsized":"2023-08-11T15:51:08.029670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:16:42.978408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450187+0000","last_change":"2023-08-11T15:51:05.850946+0000","last_active":"2023-08-11T15:51:07.450187+0000","last_peered":"2023-08-11T15:51:07.450187+0000","last_clean":"2023-08-11T15:51:07.450187+0000","last_became_active":"2023-08-11T15:51:05.850520+0000","last_became_peered":"2023-08-11T15:51:05.850520+0000","last_unstale":"2023-08-11T15:51:07.450187+0000","last_undegraded":"2023-08-11T15:51:07.450187+0000","last_fullsized":"2023-08-11T15:51:07.450187+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:19:49.259478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029612+0000","last_change":"2023-08-11T15:51:01.733689+0000","last_active":"2023-08-11T15:51:08.029612+0000","last_peered":"2023-08-11T15:51:08.029612+0000","last_clean":"2023-08-11T15:51:08.029612+0000","last_became_active":"2023-08-11T15:51:01.732609+0000","last_became_peered":"2023-08-11T15:51:01.732609+0000","last_unstale":"2023-08-11T15:51:08.029612+0000","last_undegraded":"2023-08-11T15:51:08.029612+0000","last_fullsized":"2023-08-11T15:51:08.029612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:37:47.670825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.645014+0000","last_change":"2023-08-11T15:50:54.667601+0000","last_active":"2023-08-11T15:51:08.645014+0000","last_peered":"2023-08-11T15:51:08.645014+0000","last_clean":"2023-08-11T15:51:08.645014+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:51:08.645014+0000","last_undegraded":"2023-08-11T15:51:08.645014+0000","last_fullsized":"2023-08-11T15:51:08.645014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:26.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842067+0000","last_change":"2023-08-11T15:51:00.744472+0000","last_active":"2023-08-11T15:51:06.842067+0000","last_peered":"2023-08-11T15:51:06.842067+0000","last_clean":"2023-08-11T15:51:06.842067+0000","last_became_active":"2023-08-11T15:51:00.744162+0000","last_became_peered":"2023-08-11T15:51:00.744162+0000","last_unstale":"2023-08-11T15:51:06.842067+0000","last_undegraded":"2023-08-11T15:51:06.842067+0000","last_fullsized":"2023-08-11T15:51:06.842067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:44.445917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276689+0000","last_change":"2023-08-11T15:51:05.840924+0000","last_active":"2023-08-11T15:51:07.276689+0000","last_peered":"2023-08-11T15:51:07.276689+0000","last_clean":"2023-08-11T15:51:07.276689+0000","last_became_active":"2023-08-11T15:51:05.840447+0000","last_became_peered":"2023-08-11T15:51:05.840447+0000","last_unstale":"2023-08-11T15:51:07.276689+0000","last_undegraded":"2023-08-11T15:51:07.276689+0000","last_fullsized":"2023-08-11T15:51:07.276689+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:15.105060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277414+0000","last_change":"2023-08-11T15:51:01.747032+0000","last_active":"2023-08-11T15:51:07.277414+0000","last_peered":"2023-08-11T15:51:07.277414+0000","last_clean":"2023-08-11T15:51:07.277414+0000","last_became_active":"2023-08-11T15:51:01.746677+0000","last_became_peered":"2023-08-11T15:51:01.746677+0000","last_unstale":"2023-08-11T15:51:07.277414+0000","last_undegraded":"2023-08-11T15:51:07.277414+0000","last_fullsized":"2023-08-11T15:51:07.277414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:45.458114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450746+0000","last_change":"2023-08-11T15:50:54.662968+0000","last_active":"2023-08-11T15:51:07.450746+0000","last_peered":"2023-08-11T15:51:07.450746+0000","last_clean":"2023-08-11T15:51:07.450746+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:51:07.450746+0000","last_undegraded":"2023-08-11T15:51:07.450746+0000","last_fullsized":"2023-08-11T15:51:07.450746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:21.192999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278963+0000","last_change":"2023-08-11T15:51:00.728408+0000","last_active":"2023-08-11T15:51:07.278963+0000","last_peered":"2023-08-11T15:51:07.278963+0000","last_clean":"2023-08-11T15:51:07.278963+0000","last_became_active":"2023-08-11T15:51:00.721838+0000","last_became_peered":"2023-08-11T15:51:00.721838+0000","last_unstale":"2023-08-11T15:51:07.278963+0000","last_undegraded":"2023-08-11T15:51:07.278963+0000","last_fullsized":"2023-08-11T15:51:07.278963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:10.364383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028778+0000","last_change":"2023-08-11T15:51:05.850931+0000","last_active":"2023-08-11T15:51:08.028778+0000","last_peered":"2023-08-11T15:51:08.028778+0000","last_clean":"2023-08-11T15:51:08.028778+0000","last_became_active":"2023-08-11T15:51:05.850227+0000","last_became_peered":"2023-08-11T15:51:05.850227+0000","last_unstale":"2023-08-11T15:51:08.028778+0000","last_undegraded":"2023-08-11T15:51:08.028778+0000","last_fullsized":"2023-08-11T15:51:08.028778+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.029041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840867+0000","last_change":"2023-08-11T15:51:01.739504+0000","last_active":"2023-08-11T15:51:06.840867+0000","last_peered":"2023-08-11T15:51:06.840867+0000","last_clean":"2023-08-11T15:51:06.840867+0000","last_became_active":"2023-08-11T15:51:01.739172+0000","last_became_peered":"2023-08-11T15:51:01.739172+0000","last_unstale":"2023-08-11T15:51:06.840867+0000","last_undegraded":"2023-08-11T15:51:06.840867+0000","last_fullsized":"2023-08-11T15:51:06.840867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:12:34.705521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844166+0000","last_change":"2023-08-11T15:50:55.272783+0000","last_active":"2023-08-11T15:51:06.844166+0000","last_peered":"2023-08-11T15:51:06.844166+0000","last_clean":"2023-08-11T15:51:06.844166+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:51:06.844166+0000","last_undegraded":"2023-08-11T15:51:06.844166+0000","last_fullsized":"2023-08-11T15:51:06.844166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:26.593873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450158+0000","last_change":"2023-08-11T15:51:00.737083+0000","last_active":"2023-08-11T15:51:07.450158+0000","last_peered":"2023-08-11T15:51:07.450158+0000","last_clean":"2023-08-11T15:51:07.450158+0000","last_became_active":"2023-08-11T15:51:00.736799+0000","last_became_peered":"2023-08-11T15:51:00.736799+0000","last_unstale":"2023-08-11T15:51:07.450158+0000","last_undegraded":"2023-08-11T15:51:07.450158+0000","last_fullsized":"2023-08-11T15:51:07.450158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:03:14.195345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643834+0000","last_change":"2023-08-11T15:51:05.842861+0000","last_active":"2023-08-11T15:51:08.643834+0000","last_peered":"2023-08-11T15:51:08.643834+0000","last_clean":"2023-08-11T15:51:08.643834+0000","last_became_active":"2023-08-11T15:51:05.842445+0000","last_became_peered":"2023-08-11T15:51:05.842445+0000","last_unstale":"2023-08-11T15:51:08.643834+0000","last_undegraded":"2023-08-11T15:51:08.643834+0000","last_fullsized":"2023-08-11T15:51:08.643834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:41.996042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449997+0000","last_change":"2023-08-11T15:51:01.746944+0000","last_active":"2023-08-11T15:51:07.449997+0000","last_peered":"2023-08-11T15:51:07.449997+0000","last_clean":"2023-08-11T15:51:07.449997+0000","last_became_active":"2023-08-11T15:51:01.745106+0000","last_became_peered":"2023-08-11T15:51:01.745106+0000","last_unstale":"2023-08-11T15:51:07.449997+0000","last_undegraded":"2023-08-11T15:51:07.449997+0000","last_fullsized":"2023-08-11T15:51:07.449997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:38:45.900704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450317+0000","last_change":"2023-08-11T15:50:55.270974+0000","last_active":"2023-08-11T15:51:07.450317+0000","last_peered":"2023-08-11T15:51:07.450317+0000","last_clean":"2023-08-11T15:51:07.450317+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:51:07.450317+0000","last_undegraded":"2023-08-11T15:51:07.450317+0000","last_fullsized":"2023-08-11T15:51:07.450317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:49:35.912143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845445+0000","last_change":"2023-08-11T15:51:00.736540+0000","last_active":"2023-08-11T15:51:06.845445+0000","last_peered":"2023-08-11T15:51:06.845445+0000","last_clean":"2023-08-11T15:51:06.845445+0000","last_became_active":"2023-08-11T15:51:00.736211+0000","last_became_peered":"2023-08-11T15:51:00.736211+0000","last_unstale":"2023-08-11T15:51:06.845445+0000","last_undegraded":"2023-08-11T15:51:06.845445+0000","last_fullsized":"2023-08-11T15:51:06.845445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.749276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840401+0000","last_change":"2023-08-11T15:51:05.834845+0000","last_active":"2023-08-11T15:51:06.840401+0000","last_peered":"2023-08-11T15:51:06.840401+0000","last_clean":"2023-08-11T15:51:06.840401+0000","last_became_active":"2023-08-11T15:51:05.834329+0000","last_became_peered":"2023-08-11T15:51:05.834329+0000","last_unstale":"2023-08-11T15:51:06.840401+0000","last_undegraded":"2023-08-11T15:51:06.840401+0000","last_fullsized":"2023-08-11T15:51:06.840401+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:27:54.750554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450003+0000","last_change":"2023-08-11T15:51:01.744716+0000","last_active":"2023-08-11T15:51:07.450003+0000","last_peered":"2023-08-11T15:51:07.450003+0000","last_clean":"2023-08-11T15:51:07.450003+0000","last_became_active":"2023-08-11T15:51:01.744413+0000","last_became_peered":"2023-08-11T15:51:01.744413+0000","last_unstale":"2023-08-11T15:51:07.450003+0000","last_undegraded":"2023-08-11T15:51:07.450003+0000","last_fullsized":"2023-08-11T15:51:07.450003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:43:31.654955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030156+0000","last_change":"2023-08-11T15:51:00.744357+0000","last_active":"2023-08-11T15:51:08.030156+0000","last_peered":"2023-08-11T15:51:08.030156+0000","last_clean":"2023-08-11T15:51:08.030156+0000","last_became_active":"2023-08-11T15:51:00.744005+0000","last_became_peered":"2023-08-11T15:51:00.744005+0000","last_unstale":"2023-08-11T15:51:08.030156+0000","last_undegraded":"2023-08-11T15:51:08.030156+0000","last_fullsized":"2023-08-11T15:51:08.030156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:40:52.791014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842501+0000","last_change":"2023-08-11T15:50:54.660862+0000","last_active":"2023-08-11T15:51:06.842501+0000","last_peered":"2023-08-11T15:51:06.842501+0000","last_clean":"2023-08-11T15:51:06.842501+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:51:06.842501+0000","last_undegraded":"2023-08-11T15:51:06.842501+0000","last_fullsized":"2023-08-11T15:51:06.842501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:01.459444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840964+0000","last_change":"2023-08-11T15:51:05.834675+0000","last_active":"2023-08-11T15:51:06.840964+0000","last_peered":"2023-08-11T15:51:06.840964+0000","last_clean":"2023-08-11T15:51:06.840964+0000","last_became_active":"2023-08-11T15:51:05.833754+0000","last_became_peered":"2023-08-11T15:51:05.833754+0000","last_unstale":"2023-08-11T15:51:06.840964+0000","last_undegraded":"2023-08-11T15:51:06.840964+0000","last_fullsized":"2023-08-11T15:51:06.840964+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:02:13.485965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450824+0000","last_change":"2023-08-11T15:51:01.747929+0000","last_active":"2023-08-11T15:51:07.450824+0000","last_peered":"2023-08-11T15:51:07.450824+0000","last_clean":"2023-08-11T15:51:07.450824+0000","last_became_active":"2023-08-11T15:51:01.747489+0000","last_became_peered":"2023-08-11T15:51:01.747489+0000","last_unstale":"2023-08-11T15:51:07.450824+0000","last_undegraded":"2023-08-11T15:51:07.450824+0000","last_fullsized":"2023-08-11T15:51:07.450824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:16:47.148363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449531+0000","last_change":"2023-08-11T15:51:00.735611+0000","last_active":"2023-08-11T15:51:07.449531+0000","last_peered":"2023-08-11T15:51:07.449531+0000","last_clean":"2023-08-11T15:51:07.449531+0000","last_became_active":"2023-08-11T15:51:00.735334+0000","last_became_peered":"2023-08-11T15:51:00.735334+0000","last_unstale":"2023-08-11T15:51:07.449531+0000","last_undegraded":"2023-08-11T15:51:07.449531+0000","last_fullsized":"2023-08-11T15:51:07.449531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:21:47.785823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574361+0000","last_change":"2023-08-11T15:50:54.660575+0000","last_active":"2023-08-11T15:51:08.574361+0000","last_peered":"2023-08-11T15:51:08.574361+0000","last_clean":"2023-08-11T15:51:08.574361+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:51:08.574361+0000","last_undegraded":"2023-08-11T15:51:08.574361+0000","last_fullsized":"2023-08-11T15:51:08.574361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:44:19.275242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574984+0000","last_change":"2023-08-11T15:51:05.851204+0000","last_active":"2023-08-11T15:51:08.574984+0000","last_peered":"2023-08-11T15:51:08.574984+0000","last_clean":"2023-08-11T15:51:08.574984+0000","last_became_active":"2023-08-11T15:51:05.850707+0000","last_became_peered":"2023-08-11T15:51:05.850707+0000","last_unstale":"2023-08-11T15:51:08.574984+0000","last_undegraded":"2023-08-11T15:51:08.574984+0000","last_fullsized":"2023-08-11T15:51:08.574984+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:46:46.279070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841686+0000","last_change":"2023-08-11T15:51:01.754726+0000","last_active":"2023-08-11T15:51:06.841686+0000","last_peered":"2023-08-11T15:51:06.841686+0000","last_clean":"2023-08-11T15:51:06.841686+0000","last_became_active":"2023-08-11T15:51:01.754139+0000","last_became_peered":"2023-08-11T15:51:01.754139+0000","last_unstale":"2023-08-11T15:51:06.841686+0000","last_undegraded":"2023-08-11T15:51:06.841686+0000","last_fullsized":"2023-08-11T15:51:06.841686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839697+0000","last_change":"2023-08-11T15:51:00.753588+0000","last_active":"2023-08-11T15:51:06.839697+0000","last_peered":"2023-08-11T15:51:06.839697+0000","last_clean":"2023-08-11T15:51:06.839697+0000","last_became_active":"2023-08-11T15:51:00.753328+0000","last_became_peered":"2023-08-11T15:51:00.753328+0000","last_unstale":"2023-08-11T15:51:06.839697+0000","last_undegraded":"2023-08-11T15:51:06.839697+0000","last_fullsized":"2023-08-11T15:51:06.839697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:32.917550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842875+0000","last_change":"2023-08-11T15:50:54.669675+0000","last_active":"2023-08-11T15:51:06.842875+0000","last_peered":"2023-08-11T15:51:06.842875+0000","last_clean":"2023-08-11T15:51:06.842875+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:51:06.842875+0000","last_undegraded":"2023-08-11T15:51:06.842875+0000","last_fullsized":"2023-08-11T15:51:06.842875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:50.905375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845192+0000","last_change":"2023-08-11T15:51:05.839222+0000","last_active":"2023-08-11T15:51:06.845192+0000","last_peered":"2023-08-11T15:51:06.845192+0000","last_clean":"2023-08-11T15:51:06.845192+0000","last_became_active":"2023-08-11T15:51:05.838710+0000","last_became_peered":"2023-08-11T15:51:05.838710+0000","last_unstale":"2023-08-11T15:51:06.845192+0000","last_undegraded":"2023-08-11T15:51:06.845192+0000","last_fullsized":"2023-08-11T15:51:06.845192+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:07.568296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644528+0000","last_change":"2023-08-11T15:51:01.744538+0000","last_active":"2023-08-11T15:51:08.644528+0000","last_peered":"2023-08-11T15:51:08.644528+0000","last_clean":"2023-08-11T15:51:08.644528+0000","last_became_active":"2023-08-11T15:51:01.744177+0000","last_became_peered":"2023-08-11T15:51:01.744177+0000","last_unstale":"2023-08-11T15:51:08.644528+0000","last_undegraded":"2023-08-11T15:51:08.644528+0000","last_fullsized":"2023-08-11T15:51:08.644528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:36.996581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277663+0000","last_change":"2023-08-11T15:51:00.736542+0000","last_active":"2023-08-11T15:51:07.277663+0000","last_peered":"2023-08-11T15:51:07.277663+0000","last_clean":"2023-08-11T15:51:07.277663+0000","last_became_active":"2023-08-11T15:51:00.736156+0000","last_became_peered":"2023-08-11T15:51:00.736156+0000","last_unstale":"2023-08-11T15:51:07.277663+0000","last_undegraded":"2023-08-11T15:51:07.277663+0000","last_fullsized":"2023-08-11T15:51:07.277663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843800+0000","last_change":"2023-08-11T15:50:54.663730+0000","last_active":"2023-08-11T15:51:06.843800+0000","last_peered":"2023-08-11T15:51:06.843800+0000","last_clean":"2023-08-11T15:51:06.843800+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:51:06.843800+0000","last_undegraded":"2023-08-11T15:51:06.843800+0000","last_fullsized":"2023-08-11T15:51:06.843800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:36.826532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843222+0000","last_change":"2023-08-11T15:50:48.426393+0000","last_active":"2023-08-11T15:51:06.843222+0000","last_peered":"2023-08-11T15:51:06.843222+0000","last_clean":"2023-08-11T15:51:06.843222+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:51:06.843222+0000","last_undegraded":"2023-08-11T15:51:06.843222+0000","last_fullsized":"2023-08-11T15:51:06.843222+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_clean_scrub_stamp":"2023-08-11T15:50:47.133560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:38:47.340557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843724+0000","last_change":"2023-08-11T15:51:05.850670+0000","last_active":"2023-08-11T15:51:06.843724+0000","last_peered":"2023-08-11T15:51:06.843724+0000","last_clean":"2023-08-11T15:51:06.843724+0000","last_became_active":"2023-08-11T15:51:05.850026+0000","last_became_peered":"2023-08-11T15:51:05.850026+0000","last_unstale":"2023-08-11T15:51:06.843724+0000","last_undegraded":"2023-08-11T15:51:06.843724+0000","last_fullsized":"2023-08-11T15:51:06.843724+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:08.068219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450801+0000","last_change":"2023-08-11T15:51:01.748517+0000","last_active":"2023-08-11T15:51:07.450801+0000","last_peered":"2023-08-11T15:51:07.450801+0000","last_clean":"2023-08-11T15:51:07.450801+0000","last_became_active":"2023-08-11T15:51:01.748222+0000","last_became_peered":"2023-08-11T15:51:01.748222+0000","last_unstale":"2023-08-11T15:51:07.450801+0000","last_undegraded":"2023-08-11T15:51:07.450801+0000","last_fullsized":"2023-08-11T15:51:07.450801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:24:23.784093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277502+0000","last_change":"2023-08-11T15:51:00.734516+0000","last_active":"2023-08-11T15:51:07.277502+0000","last_peered":"2023-08-11T15:51:07.277502+0000","last_clean":"2023-08-11T15:51:07.277502+0000","last_became_active":"2023-08-11T15:51:00.734143+0000","last_became_peered":"2023-08-11T15:51:00.734143+0000","last_unstale":"2023-08-11T15:51:07.277502+0000","last_undegraded":"2023-08-11T15:51:07.277502+0000","last_fullsized":"2023-08-11T15:51:07.277502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:02.960249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030585+0000","last_change":"2023-08-11T15:51:05.848396+0000","last_active":"2023-08-11T15:51:08.030585+0000","last_peered":"2023-08-11T15:51:08.030585+0000","last_clean":"2023-08-11T15:51:08.030585+0000","last_became_active":"2023-08-11T15:51:05.846255+0000","last_became_peered":"2023-08-11T15:51:05.846255+0000","last_unstale":"2023-08-11T15:51:08.030585+0000","last_undegraded":"2023-08-11T15:51:08.030585+0000","last_fullsized":"2023-08-11T15:51:08.030585+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:11:39.853592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.031055+0000","last_change":"2023-08-11T15:51:01.745387+0000","last_active":"2023-08-11T15:51:08.031055+0000","last_peered":"2023-08-11T15:51:08.031055+0000","last_clean":"2023-08-11T15:51:08.031055+0000","last_became_active":"2023-08-11T15:51:01.745076+0000","last_became_peered":"2023-08-11T15:51:01.745076+0000","last_unstale":"2023-08-11T15:51:08.031055+0000","last_undegraded":"2023-08-11T15:51:08.031055+0000","last_fullsized":"2023-08-11T15:51:08.031055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:46:26.462256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030159+0000","last_change":"2023-08-11T15:51:00.749075+0000","last_active":"2023-08-11T15:51:08.030159+0000","last_peered":"2023-08-11T15:51:08.030159+0000","last_clean":"2023-08-11T15:51:08.030159+0000","last_became_active":"2023-08-11T15:51:00.748709+0000","last_became_peered":"2023-08-11T15:51:00.748709+0000","last_unstale":"2023-08-11T15:51:08.030159+0000","last_undegraded":"2023-08-11T15:51:08.030159+0000","last_fullsized":"2023-08-11T15:51:08.030159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:18:35.477914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844801+0000","last_change":"2023-08-11T15:51:05.843837+0000","last_active":"2023-08-11T15:51:06.844801+0000","last_peered":"2023-08-11T15:51:06.844801+0000","last_clean":"2023-08-11T15:51:06.844801+0000","last_became_active":"2023-08-11T15:51:05.843261+0000","last_became_peered":"2023-08-11T15:51:05.843261+0000","last_unstale":"2023-08-11T15:51:06.844801+0000","last_undegraded":"2023-08-11T15:51:06.844801+0000","last_fullsized":"2023-08-11T15:51:06.844801+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:08.068219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030584+0000","last_change":"2023-08-11T15:51:01.735881+0000","last_active":"2023-08-11T15:51:08.030584+0000","last_peered":"2023-08-11T15:51:08.030584+0000","last_clean":"2023-08-11T15:51:08.030584+0000","last_became_active":"2023-08-11T15:51:01.735297+0000","last_became_peered":"2023-08-11T15:51:01.735297+0000","last_unstale":"2023-08-11T15:51:08.030584+0000","last_undegraded":"2023-08-11T15:51:08.030584+0000","last_fullsized":"2023-08-11T15:51:08.030584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:06:05.316781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644461+0000","last_change":"2023-08-11T15:51:00.754786+0000","last_active":"2023-08-11T15:51:08.644461+0000","last_peered":"2023-08-11T15:51:08.644461+0000","last_clean":"2023-08-11T15:51:08.644461+0000","last_became_active":"2023-08-11T15:51:00.754437+0000","last_became_peered":"2023-08-11T15:51:00.754437+0000","last_unstale":"2023-08-11T15:51:08.644461+0000","last_undegraded":"2023-08-11T15:51:08.644461+0000","last_fullsized":"2023-08-11T15:51:08.644461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:41:49.524739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.449772+0000","last_change":"2023-08-11T15:51:05.833713+0000","last_active":"2023-08-11T15:51:07.449772+0000","last_peered":"2023-08-11T15:51:07.449772+0000","last_clean":"2023-08-11T15:51:07.449772+0000","last_became_active":"2023-08-11T15:51:05.833089+0000","last_became_peered":"2023-08-11T15:51:05.833089+0000","last_unstale":"2023-08-11T15:51:07.449772+0000","last_undegraded":"2023-08-11T15:51:07.449772+0000","last_fullsized":"2023-08-11T15:51:07.449772+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:17:01.770576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030037+0000","last_change":"2023-08-11T15:51:01.748944+0000","last_active":"2023-08-11T15:51:08.030037+0000","last_peered":"2023-08-11T15:51:08.030037+0000","last_clean":"2023-08-11T15:51:08.030037+0000","last_became_active":"2023-08-11T15:51:01.748661+0000","last_became_peered":"2023-08-11T15:51:01.748661+0000","last_unstale":"2023-08-11T15:51:08.030037+0000","last_undegraded":"2023-08-11T15:51:08.030037+0000","last_fullsized":"2023-08-11T15:51:08.030037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:02:46.657720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278376+0000","last_change":"2023-08-11T15:51:00.743809+0000","last_active":"2023-08-11T15:51:07.278376+0000","last_peered":"2023-08-11T15:51:07.278376+0000","last_clean":"2023-08-11T15:51:07.278376+0000","last_became_active":"2023-08-11T15:51:00.743475+0000","last_became_peered":"2023-08-11T15:51:00.743475+0000","last_unstale":"2023-08-11T15:51:07.278376+0000","last_undegraded":"2023-08-11T15:51:07.278376+0000","last_fullsized":"2023-08-11T15:51:07.278376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574614+0000","last_change":"2023-08-11T15:51:05.845552+0000","last_active":"2023-08-11T15:51:08.574614+0000","last_peered":"2023-08-11T15:51:08.574614+0000","last_clean":"2023-08-11T15:51:08.574614+0000","last_became_active":"2023-08-11T15:51:05.845089+0000","last_became_peered":"2023-08-11T15:51:05.845089+0000","last_unstale":"2023-08-11T15:51:08.574614+0000","last_undegraded":"2023-08-11T15:51:08.574614+0000","last_fullsized":"2023-08-11T15:51:08.574614+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:14:44.850189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.840418+0000","last_change":"2023-08-11T15:51:01.754448+0000","last_active":"2023-08-11T15:51:06.840418+0000","last_peered":"2023-08-11T15:51:06.840418+0000","last_clean":"2023-08-11T15:51:06.840418+0000","last_became_active":"2023-08-11T15:51:01.754103+0000","last_became_peered":"2023-08-11T15:51:01.754103+0000","last_unstale":"2023-08-11T15:51:06.840418+0000","last_undegraded":"2023-08-11T15:51:06.840418+0000","last_fullsized":"2023-08-11T15:51:06.840418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:48:19.707007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844315+0000","last_change":"2023-08-11T15:51:00.740399+0000","last_active":"2023-08-11T15:51:06.844315+0000","last_peered":"2023-08-11T15:51:06.844315+0000","last_clean":"2023-08-11T15:51:06.844315+0000","last_became_active":"2023-08-11T15:51:00.740100+0000","last_became_peered":"2023-08-11T15:51:00.740100+0000","last_unstale":"2023-08-11T15:51:06.844315+0000","last_undegraded":"2023-08-11T15:51:06.844315+0000","last_fullsized":"2023-08-11T15:51:06.844315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:07.914582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574234+0000","last_change":"2023-08-11T15:51:05.857518+0000","last_active":"2023-08-11T15:51:08.574234+0000","last_peered":"2023-08-11T15:51:08.574234+0000","last_clean":"2023-08-11T15:51:08.574234+0000","last_became_active":"2023-08-11T15:51:05.857113+0000","last_became_peered":"2023-08-11T15:51:05.857113+0000","last_unstale":"2023-08-11T15:51:08.574234+0000","last_undegraded":"2023-08-11T15:51:08.574234+0000","last_fullsized":"2023-08-11T15:51:08.574234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:27:22.211685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643846+0000","last_change":"2023-08-11T15:51:01.754343+0000","last_active":"2023-08-11T15:51:08.643846+0000","last_peered":"2023-08-11T15:51:08.643846+0000","last_clean":"2023-08-11T15:51:08.643846+0000","last_became_active":"2023-08-11T15:51:01.754068+0000","last_became_peered":"2023-08-11T15:51:01.754068+0000","last_unstale":"2023-08-11T15:51:08.643846+0000","last_undegraded":"2023-08-11T15:51:08.643846+0000","last_fullsized":"2023-08-11T15:51:08.643846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:17.417849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843859+0000","last_change":"2023-08-11T15:51:00.739404+0000","last_active":"2023-08-11T15:51:06.843859+0000","last_peered":"2023-08-11T15:51:06.843859+0000","last_clean":"2023-08-11T15:51:06.843859+0000","last_became_active":"2023-08-11T15:51:00.738824+0000","last_became_peered":"2023-08-11T15:51:00.738824+0000","last_unstale":"2023-08-11T15:51:06.843859+0000","last_undegraded":"2023-08-11T15:51:06.843859+0000","last_fullsized":"2023-08-11T15:51:06.843859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:14.183029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574756+0000","last_change":"2023-08-11T15:51:05.852495+0000","last_active":"2023-08-11T15:51:08.574756+0000","last_peered":"2023-08-11T15:51:08.574756+0000","last_clean":"2023-08-11T15:51:08.574756+0000","last_became_active":"2023-08-11T15:51:05.852028+0000","last_became_peered":"2023-08-11T15:51:05.852028+0000","last_unstale":"2023-08-11T15:51:08.574756+0000","last_undegraded":"2023-08-11T15:51:08.574756+0000","last_fullsized":"2023-08-11T15:51:08.574756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:00:50.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030008+0000","last_change":"2023-08-11T15:51:01.746332+0000","last_active":"2023-08-11T15:51:08.030008+0000","last_peered":"2023-08-11T15:51:08.030008+0000","last_clean":"2023-08-11T15:51:08.030008+0000","last_became_active":"2023-08-11T15:51:01.746037+0000","last_became_peered":"2023-08-11T15:51:01.746037+0000","last_unstale":"2023-08-11T15:51:08.030008+0000","last_undegraded":"2023-08-11T15:51:08.030008+0000","last_fullsized":"2023-08-11T15:51:08.030008+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:01:53.294858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846006+0000","last_change":"2023-08-11T15:51:00.750241+0000","last_active":"2023-08-11T15:51:06.846006+0000","last_peered":"2023-08-11T15:51:06.846006+0000","last_clean":"2023-08-11T15:51:06.846006+0000","last_became_active":"2023-08-11T15:51:00.749526+0000","last_became_peered":"2023-08-11T15:51:00.749526+0000","last_unstale":"2023-08-11T15:51:06.846006+0000","last_undegraded":"2023-08-11T15:51:06.846006+0000","last_fullsized":"2023-08-11T15:51:06.846006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.749276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574297+0000","last_change":"2023-08-11T15:51:05.852589+0000","last_active":"2023-08-11T15:51:08.574297+0000","last_peered":"2023-08-11T15:51:08.574297+0000","last_clean":"2023-08-11T15:51:08.574297+0000","last_became_active":"2023-08-11T15:51:05.852123+0000","last_became_peered":"2023-08-11T15:51:05.852123+0000","last_unstale":"2023-08-11T15:51:08.574297+0000","last_undegraded":"2023-08-11T15:51:08.574297+0000","last_fullsized":"2023-08-11T15:51:08.574297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:52.516197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841591+0000","last_change":"2023-08-11T15:51:01.740461+0000","last_active":"2023-08-11T15:51:06.841591+0000","last_peered":"2023-08-11T15:51:06.841591+0000","last_clean":"2023-08-11T15:51:06.841591+0000","last_became_active":"2023-08-11T15:51:01.740152+0000","last_became_peered":"2023-08-11T15:51:01.740152+0000","last_unstale":"2023-08-11T15:51:06.841591+0000","last_undegraded":"2023-08-11T15:51:06.841591+0000","last_fullsized":"2023-08-11T15:51:06.841591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:09.551106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846676+0000","last_change":"2023-08-11T15:51:00.749804+0000","last_active":"2023-08-11T15:51:06.846676+0000","last_peered":"2023-08-11T15:51:06.846676+0000","last_clean":"2023-08-11T15:51:06.846676+0000","last_became_active":"2023-08-11T15:51:00.749397+0000","last_became_peered":"2023-08-11T15:51:00.749397+0000","last_unstale":"2023-08-11T15:51:06.846676+0000","last_undegraded":"2023-08-11T15:51:06.846676+0000","last_fullsized":"2023-08-11T15:51:06.846676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:16.381545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843683+0000","last_change":"2023-08-11T15:51:05.857250+0000","last_active":"2023-08-11T15:51:06.843683+0000","last_peered":"2023-08-11T15:51:06.843683+0000","last_clean":"2023-08-11T15:51:06.843683+0000","last_became_active":"2023-08-11T15:51:05.856843+0000","last_became_peered":"2023-08-11T15:51:05.856843+0000","last_unstale":"2023-08-11T15:51:06.843683+0000","last_undegraded":"2023-08-11T15:51:06.843683+0000","last_fullsized":"2023-08-11T15:51:06.843683+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:30:03.847116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.846267+0000","last_change":"2023-08-11T15:51:05.849584+0000","last_active":"2023-08-11T15:51:06.846267+0000","last_peered":"2023-08-11T15:51:06.846267+0000","last_clean":"2023-08-11T15:51:06.846267+0000","last_became_active":"2023-08-11T15:51:05.849144+0000","last_became_peered":"2023-08-11T15:51:05.849144+0000","last_unstale":"2023-08-11T15:51:06.846267+0000","last_undegraded":"2023-08-11T15:51:06.846267+0000","last_fullsized":"2023-08-11T15:51:06.846267+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:19:51.734914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643988+0000","last_change":"2023-08-11T15:51:00.743098+0000","last_active":"2023-08-11T15:51:08.643988+0000","last_peered":"2023-08-11T15:51:08.643988+0000","last_clean":"2023-08-11T15:51:08.643988+0000","last_became_active":"2023-08-11T15:51:00.742606+0000","last_became_peered":"2023-08-11T15:51:00.742606+0000","last_unstale":"2023-08-11T15:51:08.643988+0000","last_undegraded":"2023-08-11T15:51:08.643988+0000","last_fullsized":"2023-08-11T15:51:08.643988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:38:17.546002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278685+0000","last_change":"2023-08-11T15:51:01.746170+0000","last_active":"2023-08-11T15:51:07.278685+0000","last_peered":"2023-08-11T15:51:07.278685+0000","last_clean":"2023-08-11T15:51:07.278685+0000","last_became_active":"2023-08-11T15:51:01.745672+0000","last_became_peered":"2023-08-11T15:51:01.745672+0000","last_unstale":"2023-08-11T15:51:07.278685+0000","last_undegraded":"2023-08-11T15:51:07.278685+0000","last_fullsized":"2023-08-11T15:51:07.278685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:20:27.436792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278398+0000","last_change":"2023-08-11T15:51:05.844332+0000","last_active":"2023-08-11T15:51:07.278398+0000","last_peered":"2023-08-11T15:51:07.278398+0000","last_clean":"2023-08-11T15:51:07.278398+0000","last_became_active":"2023-08-11T15:51:05.841336+0000","last_became_peered":"2023-08-11T15:51:05.841336+0000","last_unstale":"2023-08-11T15:51:07.278398+0000","last_undegraded":"2023-08-11T15:51:07.278398+0000","last_fullsized":"2023-08-11T15:51:07.278398+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:19:51.734914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844582+0000","last_change":"2023-08-11T15:51:00.750609+0000","last_active":"2023-08-11T15:51:06.844582+0000","last_peered":"2023-08-11T15:51:06.844582+0000","last_clean":"2023-08-11T15:51:06.844582+0000","last_became_active":"2023-08-11T15:51:00.750277+0000","last_became_peered":"2023-08-11T15:51:00.750277+0000","last_unstale":"2023-08-11T15:51:06.844582+0000","last_undegraded":"2023-08-11T15:51:06.844582+0000","last_fullsized":"2023-08-11T15:51:06.844582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:19.559486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841973+0000","last_change":"2023-08-11T15:51:01.750332+0000","last_active":"2023-08-11T15:51:06.841973+0000","last_peered":"2023-08-11T15:51:06.841973+0000","last_clean":"2023-08-11T15:51:06.841973+0000","last_became_active":"2023-08-11T15:51:01.749234+0000","last_became_peered":"2023-08-11T15:51:01.749234+0000","last_unstale":"2023-08-11T15:51:06.841973+0000","last_undegraded":"2023-08-11T15:51:06.841973+0000","last_fullsized":"2023-08-11T15:51:06.841973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:56:08.926778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839995+0000","last_change":"2023-08-11T15:51:00.745865+0000","last_active":"2023-08-11T15:51:06.839995+0000","last_peered":"2023-08-11T15:51:06.839995+0000","last_clean":"2023-08-11T15:51:06.839995+0000","last_became_active":"2023-08-11T15:51:00.745469+0000","last_became_peered":"2023-08-11T15:51:00.745469+0000","last_unstale":"2023-08-11T15:51:06.839995+0000","last_undegraded":"2023-08-11T15:51:06.839995+0000","last_fullsized":"2023-08-11T15:51:06.839995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:25:15.827962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845118+0000","last_change":"2023-08-11T15:51:01.740388+0000","last_active":"2023-08-11T15:51:06.845118+0000","last_peered":"2023-08-11T15:51:06.845118+0000","last_clean":"2023-08-11T15:51:06.845118+0000","last_became_active":"2023-08-11T15:51:01.739972+0000","last_became_peered":"2023-08-11T15:51:01.739972+0000","last_unstale":"2023-08-11T15:51:06.845118+0000","last_undegraded":"2023-08-11T15:51:06.845118+0000","last_fullsized":"2023-08-11T15:51:06.845118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:41.231851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277756+0000","last_change":"2023-08-11T15:51:05.847377+0000","last_active":"2023-08-11T15:51:07.277756+0000","last_peered":"2023-08-11T15:51:07.277756+0000","last_clean":"2023-08-11T15:51:07.277756+0000","last_became_active":"2023-08-11T15:51:05.843902+0000","last_became_peered":"2023-08-11T15:51:05.843902+0000","last_unstale":"2023-08-11T15:51:07.277756+0000","last_undegraded":"2023-08-11T15:51:07.277756+0000","last_fullsized":"2023-08-11T15:51:07.277756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:45.327626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843318+0000","last_change":"2023-08-11T15:51:00.753432+0000","last_active":"2023-08-11T15:51:06.843318+0000","last_peered":"2023-08-11T15:51:06.843318+0000","last_clean":"2023-08-11T15:51:06.843318+0000","last_became_active":"2023-08-11T15:51:00.753099+0000","last_became_peered":"2023-08-11T15:51:00.753099+0000","last_unstale":"2023-08-11T15:51:06.843318+0000","last_undegraded":"2023-08-11T15:51:06.843318+0000","last_fullsized":"2023-08-11T15:51:06.843318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:59.680620+0000","last_clean_scrub_stamp":"2023-08-11T15:50:59.680620+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:44.445917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.278374+0000","last_change":"2023-08-11T15:51:01.754975+0000","last_active":"2023-08-11T15:51:07.278374+0000","last_peered":"2023-08-11T15:51:07.278374+0000","last_clean":"2023-08-11T15:51:07.278374+0000","last_became_active":"2023-08-11T15:51:01.754617+0000","last_became_peered":"2023-08-11T15:51:01.754617+0000","last_unstale":"2023-08-11T15:51:07.278374+0000","last_undegraded":"2023-08-11T15:51:07.278374+0000","last_fullsized":"2023-08-11T15:51:07.278374+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:20.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844645+0000","last_change":"2023-08-11T15:51:05.841963+0000","last_active":"2023-08-11T15:51:06.844645+0000","last_peered":"2023-08-11T15:51:06.844645+0000","last_clean":"2023-08-11T15:51:06.844645+0000","last_became_active":"2023-08-11T15:51:05.841100+0000","last_became_peered":"2023-08-11T15:51:05.841100+0000","last_unstale":"2023-08-11T15:51:06.844645+0000","last_undegraded":"2023-08-11T15:51:06.844645+0000","last_fullsized":"2023-08-11T15:51:06.844645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:04.788591+0000","last_clean_scrub_stamp":"2023-08-11T15:51:04.788591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:00:49.553888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.643596+0000","last_change":"2023-08-11T15:51:01.740691+0000","last_active":"2023-08-11T15:51:08.643596+0000","last_peered":"2023-08-11T15:51:08.643596+0000","last_clean":"2023-08-11T15:51:08.643596+0000","last_became_active":"2023-08-11T15:51:01.740411+0000","last_became_peered":"2023-08-11T15:51:01.740411+0000","last_unstale":"2023-08-11T15:51:08.643596+0000","last_undegraded":"2023-08-11T15:51:08.643596+0000","last_fullsized":"2023-08-11T15:51:08.643596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:14:24.513369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276978+0000","last_change":"2023-08-11T15:51:01.754967+0000","last_active":"2023-08-11T15:51:07.276978+0000","last_peered":"2023-08-11T15:51:07.276978+0000","last_clean":"2023-08-11T15:51:07.276978+0000","last_became_active":"2023-08-11T15:51:01.754670+0000","last_became_peered":"2023-08-11T15:51:01.754670+0000","last_unstale":"2023-08-11T15:51:07.276978+0000","last_undegraded":"2023-08-11T15:51:07.276978+0000","last_fullsized":"2023-08-11T15:51:07.276978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029518+0000","last_change":"2023-08-11T15:51:01.743911+0000","last_active":"2023-08-11T15:51:08.029518+0000","last_peered":"2023-08-11T15:51:08.029518+0000","last_clean":"2023-08-11T15:51:08.029518+0000","last_became_active":"2023-08-11T15:51:01.743608+0000","last_became_peered":"2023-08-11T15:51:01.743608+0000","last_unstale":"2023-08-11T15:51:08.029518+0000","last_undegraded":"2023-08-11T15:51:08.029518+0000","last_fullsized":"2023-08-11T15:51:08.029518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:45:30.132181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.277768+0000","last_change":"2023-08-11T15:51:01.748054+0000","last_active":"2023-08-11T15:51:07.277768+0000","last_peered":"2023-08-11T15:51:07.277768+0000","last_clean":"2023-08-11T15:51:07.277768+0000","last_became_active":"2023-08-11T15:51:01.747751+0000","last_became_peered":"2023-08-11T15:51:01.747751+0000","last_unstale":"2023-08-11T15:51:07.277768+0000","last_undegraded":"2023-08-11T15:51:07.277768+0000","last_fullsized":"2023-08-11T15:51:07.277768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:33.655411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450215+0000","last_change":"2023-08-11T15:51:01.745398+0000","last_active":"2023-08-11T15:51:07.450215+0000","last_peered":"2023-08-11T15:51:07.450215+0000","last_clean":"2023-08-11T15:51:07.450215+0000","last_became_active":"2023-08-11T15:51:01.745013+0000","last_became_peered":"2023-08-11T15:51:01.745013+0000","last_unstale":"2023-08-11T15:51:07.450215+0000","last_undegraded":"2023-08-11T15:51:07.450215+0000","last_fullsized":"2023-08-11T15:51:07.450215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:37:45.341896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844793+0000","last_change":"2023-08-11T15:51:01.747243+0000","last_active":"2023-08-11T15:51:06.844793+0000","last_peered":"2023-08-11T15:51:06.844793+0000","last_clean":"2023-08-11T15:51:06.844793+0000","last_became_active":"2023-08-11T15:51:01.746905+0000","last_became_peered":"2023-08-11T15:51:01.746905+0000","last_unstale":"2023-08-11T15:51:06.844793+0000","last_undegraded":"2023-08-11T15:51:06.844793+0000","last_fullsized":"2023-08-11T15:51:06.844793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450600+0000","last_change":"2023-08-11T15:51:01.743655+0000","last_active":"2023-08-11T15:51:07.450600+0000","last_peered":"2023-08-11T15:51:07.450600+0000","last_clean":"2023-08-11T15:51:07.450600+0000","last_became_active":"2023-08-11T15:51:01.743308+0000","last_became_peered":"2023-08-11T15:51:01.743308+0000","last_unstale":"2023-08-11T15:51:07.450600+0000","last_undegraded":"2023-08-11T15:51:07.450600+0000","last_fullsized":"2023-08-11T15:51:07.450600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:08:59.509573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.450686+0000","last_change":"2023-08-11T15:51:01.747270+0000","last_active":"2023-08-11T15:51:07.450686+0000","last_peered":"2023-08-11T15:51:07.450686+0000","last_clean":"2023-08-11T15:51:07.450686+0000","last_became_active":"2023-08-11T15:51:01.746924+0000","last_became_peered":"2023-08-11T15:51:01.746924+0000","last_unstale":"2023-08-11T15:51:07.450686+0000","last_undegraded":"2023-08-11T15:51:07.450686+0000","last_fullsized":"2023-08-11T15:51:07.450686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:07:01.100072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842527+0000","last_change":"2023-08-11T15:51:01.750707+0000","last_active":"2023-08-11T15:51:06.842527+0000","last_peered":"2023-08-11T15:51:06.842527+0000","last_clean":"2023-08-11T15:51:06.842527+0000","last_became_active":"2023-08-11T15:51:01.750243+0000","last_became_peered":"2023-08-11T15:51:01.750243+0000","last_unstale":"2023-08-11T15:51:06.842527+0000","last_undegraded":"2023-08-11T15:51:06.842527+0000","last_fullsized":"2023-08-11T15:51:06.842527+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:07.276812+0000","last_change":"2023-08-11T15:51:01.744825+0000","last_active":"2023-08-11T15:51:07.276812+0000","last_peered":"2023-08-11T15:51:07.276812+0000","last_clean":"2023-08-11T15:51:07.276812+0000","last_became_active":"2023-08-11T15:51:01.744446+0000","last_became_peered":"2023-08-11T15:51:01.744446+0000","last_unstale":"2023-08-11T15:51:07.276812+0000","last_undegraded":"2023-08-11T15:51:07.276812+0000","last_fullsized":"2023-08-11T15:51:07.276812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:08.520981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.839890+0000","last_change":"2023-08-11T15:51:01.754226+0000","last_active":"2023-08-11T15:51:06.839890+0000","last_peered":"2023-08-11T15:51:06.839890+0000","last_clean":"2023-08-11T15:51:06.839890+0000","last_became_active":"2023-08-11T15:51:01.753864+0000","last_became_peered":"2023-08-11T15:51:01.753864+0000","last_unstale":"2023-08-11T15:51:06.839890+0000","last_undegraded":"2023-08-11T15:51:06.839890+0000","last_fullsized":"2023-08-11T15:51:06.839890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:42:42.499555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.843198+0000","last_change":"2023-08-11T15:51:01.751698+0000","last_active":"2023-08-11T15:51:06.843198+0000","last_peered":"2023-08-11T15:51:06.843198+0000","last_clean":"2023-08-11T15:51:06.843198+0000","last_became_active":"2023-08-11T15:51:01.751183+0000","last_became_peered":"2023-08-11T15:51:01.751183+0000","last_unstale":"2023-08-11T15:51:06.843198+0000","last_undegraded":"2023-08-11T15:51:06.843198+0000","last_fullsized":"2023-08-11T15:51:06.843198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:43.888070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029934+0000","last_change":"2023-08-11T15:51:01.743064+0000","last_active":"2023-08-11T15:51:08.029934+0000","last_peered":"2023-08-11T15:51:08.029934+0000","last_clean":"2023-08-11T15:51:08.029934+0000","last_became_active":"2023-08-11T15:51:01.742783+0000","last_became_peered":"2023-08-11T15:51:01.742783+0000","last_unstale":"2023-08-11T15:51:08.029934+0000","last_undegraded":"2023-08-11T15:51:08.029934+0000","last_fullsized":"2023-08-11T15:51:08.029934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:25:56.529094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574852+0000","last_change":"2023-08-11T15:51:01.746080+0000","last_active":"2023-08-11T15:51:08.574852+0000","last_peered":"2023-08-11T15:51:08.574852+0000","last_clean":"2023-08-11T15:51:08.574852+0000","last_became_active":"2023-08-11T15:51:01.745366+0000","last_became_peered":"2023-08-11T15:51:01.745366+0000","last_unstale":"2023-08-11T15:51:08.574852+0000","last_undegraded":"2023-08-11T15:51:08.574852+0000","last_fullsized":"2023-08-11T15:51:08.574852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:58:33.514033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030489+0000","last_change":"2023-08-11T15:51:01.735183+0000","last_active":"2023-08-11T15:51:08.030489+0000","last_peered":"2023-08-11T15:51:08.030489+0000","last_clean":"2023-08-11T15:51:08.030489+0000","last_became_active":"2023-08-11T15:51:01.734771+0000","last_became_peered":"2023-08-11T15:51:01.734771+0000","last_unstale":"2023-08-11T15:51:08.030489+0000","last_undegraded":"2023-08-11T15:51:08.030489+0000","last_fullsized":"2023-08-11T15:51:08.030489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:28:21.255388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842288+0000","last_change":"2023-08-11T15:51:01.753111+0000","last_active":"2023-08-11T15:51:06.842288+0000","last_peered":"2023-08-11T15:51:06.842288+0000","last_clean":"2023-08-11T15:51:06.842288+0000","last_became_active":"2023-08-11T15:51:01.752792+0000","last_became_peered":"2023-08-11T15:51:01.752792+0000","last_unstale":"2023-08-11T15:51:06.842288+0000","last_undegraded":"2023-08-11T15:51:06.842288+0000","last_fullsized":"2023-08-11T15:51:06.842288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:59.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.028692+0000","last_change":"2023-08-11T15:51:01.737005+0000","last_active":"2023-08-11T15:51:08.028692+0000","last_peered":"2023-08-11T15:51:08.028692+0000","last_clean":"2023-08-11T15:51:08.028692+0000","last_became_active":"2023-08-11T15:51:01.736669+0000","last_became_peered":"2023-08-11T15:51:01.736669+0000","last_unstale":"2023-08-11T15:51:08.028692+0000","last_undegraded":"2023-08-11T15:51:08.028692+0000","last_fullsized":"2023-08-11T15:51:08.028692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:10:00.491572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844298+0000","last_change":"2023-08-11T15:51:01.755893+0000","last_active":"2023-08-11T15:51:06.844298+0000","last_peered":"2023-08-11T15:51:06.844298+0000","last_clean":"2023-08-11T15:51:06.844298+0000","last_became_active":"2023-08-11T15:51:01.755182+0000","last_became_peered":"2023-08-11T15:51:01.755182+0000","last_unstale":"2023-08-11T15:51:06.844298+0000","last_undegraded":"2023-08-11T15:51:06.844298+0000","last_fullsized":"2023-08-11T15:51:06.844298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:03.472522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.842189+0000","last_change":"2023-08-11T15:51:01.739785+0000","last_active":"2023-08-11T15:51:06.842189+0000","last_peered":"2023-08-11T15:51:06.842189+0000","last_clean":"2023-08-11T15:51:06.842189+0000","last_became_active":"2023-08-11T15:51:01.739500+0000","last_became_peered":"2023-08-11T15:51:01.739500+0000","last_unstale":"2023-08-11T15:51:06.842189+0000","last_undegraded":"2023-08-11T15:51:06.842189+0000","last_fullsized":"2023-08-11T15:51:06.842189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.574594+0000","last_change":"2023-08-11T15:51:01.732565+0000","last_active":"2023-08-11T15:51:08.574594+0000","last_peered":"2023-08-11T15:51:08.574594+0000","last_clean":"2023-08-11T15:51:08.574594+0000","last_became_active":"2023-08-11T15:51:01.732076+0000","last_became_peered":"2023-08-11T15:51:01.732076+0000","last_unstale":"2023-08-11T15:51:08.574594+0000","last_undegraded":"2023-08-11T15:51:08.574594+0000","last_fullsized":"2023-08-11T15:51:08.574594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:20.262055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575381+0000","last_change":"2023-08-11T15:51:01.736603+0000","last_active":"2023-08-11T15:51:08.575381+0000","last_peered":"2023-08-11T15:51:08.575381+0000","last_clean":"2023-08-11T15:51:08.575381+0000","last_became_active":"2023-08-11T15:51:01.735835+0000","last_became_peered":"2023-08-11T15:51:01.735835+0000","last_unstale":"2023-08-11T15:51:08.575381+0000","last_undegraded":"2023-08-11T15:51:08.575381+0000","last_fullsized":"2023-08-11T15:51:08.575381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:07.026259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030580+0000","last_change":"2023-08-11T15:51:01.752520+0000","last_active":"2023-08-11T15:51:08.030580+0000","last_peered":"2023-08-11T15:51:08.030580+0000","last_clean":"2023-08-11T15:51:08.030580+0000","last_became_active":"2023-08-11T15:51:01.752203+0000","last_became_peered":"2023-08-11T15:51:01.752203+0000","last_unstale":"2023-08-11T15:51:08.030580+0000","last_undegraded":"2023-08-11T15:51:08.030580+0000","last_fullsized":"2023-08-11T15:51:08.030580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:29:45.923242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.845432+0000","last_change":"2023-08-11T15:51:01.737002+0000","last_active":"2023-08-11T15:51:06.845432+0000","last_peered":"2023-08-11T15:51:06.845432+0000","last_clean":"2023-08-11T15:51:06.845432+0000","last_became_active":"2023-08-11T15:51:01.736720+0000","last_became_peered":"2023-08-11T15:51:01.736720+0000","last_unstale":"2023-08-11T15:51:06.845432+0000","last_undegraded":"2023-08-11T15:51:06.845432+0000","last_fullsized":"2023-08-11T15:51:06.845432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:39:20.571682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841857+0000","last_change":"2023-08-11T15:51:01.752888+0000","last_active":"2023-08-11T15:51:06.841857+0000","last_peered":"2023-08-11T15:51:06.841857+0000","last_clean":"2023-08-11T15:51:06.841857+0000","last_became_active":"2023-08-11T15:51:01.752593+0000","last_became_peered":"2023-08-11T15:51:01.752593+0000","last_unstale":"2023-08-11T15:51:06.841857+0000","last_undegraded":"2023-08-11T15:51:06.841857+0000","last_fullsized":"2023-08-11T15:51:06.841857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:39:00.899813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841369+0000","last_change":"2023-08-11T15:51:01.741200+0000","last_active":"2023-08-11T15:51:06.841369+0000","last_peered":"2023-08-11T15:51:06.841369+0000","last_clean":"2023-08-11T15:51:06.841369+0000","last_became_active":"2023-08-11T15:51:01.738593+0000","last_became_peered":"2023-08-11T15:51:01.738593+0000","last_unstale":"2023-08-11T15:51:06.841369+0000","last_undegraded":"2023-08-11T15:51:06.841369+0000","last_fullsized":"2023-08-11T15:51:06.841369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:02:09.390191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844003+0000","last_change":"2023-08-11T15:51:01.756072+0000","last_active":"2023-08-11T15:51:06.844003+0000","last_peered":"2023-08-11T15:51:06.844003+0000","last_clean":"2023-08-11T15:51:06.844003+0000","last_became_active":"2023-08-11T15:51:01.755522+0000","last_became_peered":"2023-08-11T15:51:01.755522+0000","last_unstale":"2023-08-11T15:51:06.844003+0000","last_undegraded":"2023-08-11T15:51:06.844003+0000","last_fullsized":"2023-08-11T15:51:06.844003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:41.231851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.029178+0000","last_change":"2023-08-11T15:51:01.737005+0000","last_active":"2023-08-11T15:51:08.029178+0000","last_peered":"2023-08-11T15:51:08.029178+0000","last_clean":"2023-08-11T15:51:08.029178+0000","last_became_active":"2023-08-11T15:51:01.736657+0000","last_became_peered":"2023-08-11T15:51:01.736657+0000","last_unstale":"2023-08-11T15:51:08.029178+0000","last_undegraded":"2023-08-11T15:51:08.029178+0000","last_fullsized":"2023-08-11T15:51:08.029178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:20:52.818361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.841216+0000","last_change":"2023-08-11T15:51:01.749800+0000","last_active":"2023-08-11T15:51:06.841216+0000","last_peered":"2023-08-11T15:51:06.841216+0000","last_clean":"2023-08-11T15:51:06.841216+0000","last_became_active":"2023-08-11T15:51:01.749436+0000","last_became_peered":"2023-08-11T15:51:01.749436+0000","last_unstale":"2023-08-11T15:51:06.841216+0000","last_undegraded":"2023-08-11T15:51:06.841216+0000","last_fullsized":"2023-08-11T15:51:06.841216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:36:11.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.644342+0000","last_change":"2023-08-11T15:51:01.739883+0000","last_active":"2023-08-11T15:51:08.644342+0000","last_peered":"2023-08-11T15:51:08.644342+0000","last_clean":"2023-08-11T15:51:08.644342+0000","last_became_active":"2023-08-11T15:51:01.739490+0000","last_became_peered":"2023-08-11T15:51:01.739490+0000","last_unstale":"2023-08-11T15:51:08.644342+0000","last_undegraded":"2023-08-11T15:51:08.644342+0000","last_fullsized":"2023-08-11T15:51:08.644342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:19.428228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.030296+0000","last_change":"2023-08-11T15:51:01.737093+0000","last_active":"2023-08-11T15:51:08.030296+0000","last_peered":"2023-08-11T15:51:08.030296+0000","last_clean":"2023-08-11T15:51:08.030296+0000","last_became_active":"2023-08-11T15:51:01.736803+0000","last_became_peered":"2023-08-11T15:51:01.736803+0000","last_unstale":"2023-08-11T15:51:08.030296+0000","last_undegraded":"2023-08-11T15:51:08.030296+0000","last_fullsized":"2023-08-11T15:51:08.030296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:50:46.864696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:06.844939+0000","last_change":"2023-08-11T15:51:01.753163+0000","last_active":"2023-08-11T15:51:06.844939+0000","last_peered":"2023-08-11T15:51:06.844939+0000","last_clean":"2023-08-11T15:51:06.844939+0000","last_became_active":"2023-08-11T15:51:01.752836+0000","last_became_peered":"2023-08-11T15:51:01.752836+0000","last_unstale":"2023-08-11T15:51:06.844939+0000","last_undegraded":"2023-08-11T15:51:06.844939+0000","last_fullsized":"2023-08-11T15:51:06.844939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:15.195226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-08-11T15:51:08.575171+0000","last_change":"2023-08-11T15:51:01.739059+0000","last_active":"2023-08-11T15:51:08.575171+0000","last_peered":"2023-08-11T15:51:08.575171+0000","last_clean":"2023-08-11T15:51:08.575171+0000","last_became_active":"2023-08-11T15:51:01.738739+0000","last_became_peered":"2023-08-11T15:51:01.738739+0000","last_unstale":"2023-08-11T15:51:08.575171+0000","last_undegraded":"2023-08-11T15:51:08.575171+0000","last_fullsized":"2023-08-11T15:51:08.575171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:51:00.692816+0000","last_clean_scrub_stamp":"2023-08-11T15:51:00.692816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:11:40.167154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":921600,"data_stored":717621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5840000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":123848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":319488,"data_stored":127093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]}]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":123286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":299008,"data_stored":123196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]}]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":897024,"data_stored":714146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.901}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.869}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:51:33.741 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-08-11T15:51:33.741 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-08-11T15:51:33.741 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-08-11T15:51:33.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-08-11T15:51:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.009+0000 7f5560a52640 1 Processor -- start 2023-08-11T15:51:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.010+0000 7f5560a52640 1 -- start start 2023-08-11T15:51:34.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.011+0000 7f5560a52640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f555c15c1f0 0x7f555c0b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.011+0000 7f5560a52640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f555c15c5d0 0x7f555c15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.011+0000 7f5560a52640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 0x7f555c0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.011+0000 7f5560a52640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c0b5170 con 0x7f555c15c5d0 2023-08-11T15:51:34.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.011+0000 7f5560a52640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c15d080 con 0x7f555c15c1f0 2023-08-11T15:51:34.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.011+0000 7f5560a52640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c15d360 con 0x7f555c0d04b0 2023-08-11T15:51:34.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.011+0000 7f5553fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f555c15c5d0 0x7f555c15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.011+0000 7f5553fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f555c15c5d0 0x7f555c15ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46552/0 (socket says 172.21.15.27:46552) 2023-08-11T15:51:34.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.011+0000 7f5553fff640 1 -- 172.21.15.27:0/3387779493 learned_addr learned my addr 172.21.15.27:0/3387779493 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:34.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.012+0000 7f555bfff640 1 --2- 172.21.15.27:0/3387779493 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 0x7f555c0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.012+0000 7f555b7fe640 1 --2- 172.21.15.27:0/3387779493 >> v2:172.21.15.119:3300/0 conn(0x7f555c15c1f0 0x7f555c0b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.012+0000 7f5553fff640 1 -- 172.21.15.27:0/3387779493 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 msgr2=0x7f555c0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:34.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.012+0000 7f5553fff640 1 --2- 172.21.15.27:0/3387779493 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 0x7f555c0d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.012+0000 7f5553fff640 1 -- 172.21.15.27:0/3387779493 >> v2:172.21.15.119:3300/0 conn(0x7f555c15c1f0 msgr2=0x7f555c0b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:34.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.012+0000 7f5553fff640 1 --2- 172.21.15.27:0/3387779493 >> v2:172.21.15.119:3300/0 conn(0x7f555c15c1f0 0x7f555c0b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.012+0000 7f5553fff640 1 -- 172.21.15.27:0/3387779493 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c15d640 con 0x7f555c15c5d0 2023-08-11T15:51:34.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.013+0000 7f5553fff640 1 --2- 172.21.15.27:0/3387779493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c5d0 0x7f555c15ca30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f555c08b310 tx=0x7f554402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22104c7c3013241 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:34.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.013+0000 7f555affd640 1 -- 172.21.15.27:0/3387779493 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544030860 con 0x7f555c15c5d0 2023-08-11T15:51:34.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.013+0000 7f555affd640 1 -- 172.21.15.27:0/3387779493 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55440309c0 con 0x7f555c15c5d0 2023-08-11T15:51:34.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.013+0000 7f555affd640 1 -- 172.21.15.27:0/3387779493 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544030d50 con 0x7f555c15c5d0 2023-08-11T15:51:34.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.014+0000 7f5560a52640 1 -- 172.21.15.27:0/3387779493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c5d0 msgr2=0x7f555c15ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:34.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.014+0000 7f5560a52640 1 --2- 172.21.15.27:0/3387779493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c5d0 0x7f555c15ca30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f555c08b310 tx=0x7f554402b9a0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.014+0000 7f5560a52640 1 -- 172.21.15.27:0/3387779493 shutdown_connections 2023-08-11T15:51:34.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.014+0000 7f5560a52640 1 --2- 172.21.15.27:0/3387779493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c5d0 0x7f555c15ca30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.014+0000 7f5560a52640 1 --2- 172.21.15.27:0/3387779493 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 0x7f555c0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.014+0000 7f5560a52640 1 --2- 172.21.15.27:0/3387779493 >> v2:172.21.15.119:3300/0 conn(0x7f555c15c1f0 0x7f555c0b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.014+0000 7f5560a52640 1 -- 172.21.15.27:0/3387779493 >> 172.21.15.27:0/3387779493 conn(0x7f555c06a250 msgr2=0x7f555c06a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.014+0000 7f5560a52640 1 -- 172.21.15.27:0/3387779493 shutdown_connections 2023-08-11T15:51:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.014+0000 7f5560a52640 1 -- 172.21.15.27:0/3387779493 wait complete. 2023-08-11T15:51:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.016+0000 7f5560a52640 1 Processor -- start 2023-08-11T15:51:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.016+0000 7f5560a52640 1 -- start start 2023-08-11T15:51:34.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.017+0000 7f5560a52640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 0x7f555c0c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.017+0000 7f5560a52640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c1f0 0x7f555c0c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.017+0000 7f5560a52640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f555c15c5d0 0x7f555c14dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.017+0000 7f5560a52640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c0b7260 con 0x7f555c15c1f0 2023-08-11T15:51:34.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.017+0000 7f5560a52640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c0c8b50 con 0x7f555c15c5d0 2023-08-11T15:51:34.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.017+0000 7f5560a52640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c1f4f80 con 0x7f555c0d04b0 2023-08-11T15:51:34.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f5553fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c1f0 0x7f555c0c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f5553fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c1f0 0x7f555c0c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46554/0 (socket says 172.21.15.27:46554) 2023-08-11T15:51:34.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f5553fff640 1 -- 172.21.15.27:0/2768425832 learned_addr learned my addr 172.21.15.27:0/2768425832 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:34.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f555b7fe640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 0x7f555c0c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f555bfff640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f555c15c5d0 0x7f555c14dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f5553fff640 1 -- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 msgr2=0x7f555c0c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:34.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f5553fff640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 0x7f555c0c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f5553fff640 1 -- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f555c15c5d0 msgr2=0x7f555c14dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:34.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f5553fff640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f555c15c5d0 0x7f555c14dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f5553fff640 1 -- 172.21.15.27:0/2768425832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440094d0 con 0x7f555c15c1f0 2023-08-11T15:51:34.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f555bfff640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f555c15c5d0 0x7f555c14dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.018+0000 7f555b7fe640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 0x7f555c0c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.019+0000 7f5553fff640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c1f0 0x7f555c0c5ce0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f554402b9e0 tx=0x7f55440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.019+0000 7f5558ff9640 1 -- 172.21.15.27:0/2768425832 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554403e070 con 0x7f555c15c1f0 2023-08-11T15:51:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.019+0000 7f5558ff9640 1 -- 172.21.15.27:0/2768425832 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5544039a60 con 0x7f555c15c1f0 2023-08-11T15:51:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.019+0000 7f5560a52640 1 -- 172.21.15.27:0/2768425832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c1f5260 con 0x7f555c15c1f0 2023-08-11T15:51:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.019+0000 7f5558ff9640 1 -- 172.21.15.27:0/2768425832 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544039df0 con 0x7f555c15c1f0 2023-08-11T15:51:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.019+0000 7f5560a52640 1 -- 172.21.15.27:0/2768425832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c1f5b00 con 0x7f555c15c1f0 2023-08-11T15:51:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.020+0000 7f5560a52640 1 -- 172.21.15.27:0/2768425832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c0b52e0 con 0x7f555c15c1f0 2023-08-11T15:51:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.021+0000 7f5558ff9640 1 -- 172.21.15.27:0/2768425832 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f554404b400 con 0x7f555c15c1f0 2023-08-11T15:51:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.022+0000 7f5558ff9640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f553c06a050 0x7f553c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.022+0000 7f5558ff9640 1 -- 172.21.15.27:0/2768425832 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f55440aaf40 con 0x7f555c15c1f0 2023-08-11T15:51:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.023+0000 7f555b7fe640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f553c06a050 0x7f553c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.024+0000 7f555b7fe640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f553c06a050 0x7f553c06c510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f554c006fd0 tx=0x7f554c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.026+0000 7f5558ff9640 1 -- 172.21.15.27:0/2768425832 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f555c0b52e0 con 0x7f555c15c1f0 2023-08-11T15:51:34.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.260+0000 7f5560a52640 1 -- 172.21.15.27:0/2768425832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f555c0b52e0 con 0x7f555c15c1f0 2023-08-11T15:51:34.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.260+0000 7f5558ff9640 1 -- 172.21.15.27:0/2768425832 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f555c0b52e0 con 0x7f555c15c1f0 2023-08-11T15:51:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:34.262 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-08-11T15:51:34.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 -- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f553c06a050 msgr2=0x7f553c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:34.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f553c06a050 0x7f553c06c510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f554c006fd0 tx=0x7f554c008040 comp rx=0 tx=0).stop 2023-08-11T15:51:34.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 -- 172.21.15.27:0/2768425832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c1f0 msgr2=0x7f555c0c5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:34.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c1f0 0x7f555c0c5ce0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f554402b9e0 tx=0x7f55440046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 -- 172.21.15.27:0/2768425832 shutdown_connections 2023-08-11T15:51:34.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f553c06a050 0x7f553c06c510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f555c15c5d0 0x7f555c14dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f555c15c1f0 0x7f555c0c5ce0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 --2- 172.21.15.27:0/2768425832 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f555c0d04b0 0x7f555c0c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 -- 172.21.15.27:0/2768425832 >> 172.21.15.27:0/2768425832 conn(0x7f555c06a250 msgr2=0x7f555c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:34.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 -- 172.21.15.27:0/2768425832 shutdown_connections 2023-08-11T15:51:34.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.263+0000 7f5560a52640 1 -- 172.21.15.27:0/2768425832 wait complete. 2023-08-11T15:51:34.284 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-08-11T15:51:34.284 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:34.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.518+0000 7f54764ec640 1 Processor -- start 2023-08-11T15:51:34.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.519+0000 7f54764ec640 1 -- start start 2023-08-11T15:51:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.520+0000 7f54764ec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5470156e50 0x7f547015a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.520+0000 7f54764ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54700cbb70 0x7f54700ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.520+0000 7f54764ec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700cbfb0 con 0x7f54700caef0 2023-08-11T15:51:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.520+0000 7f54764ec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54700ca460 con 0x7f5470156e50 2023-08-11T15:51:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.520+0000 7f54764ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54700cb380 con 0x7f54700cbb70 2023-08-11T15:51:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.521+0000 7f546ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5470156e50 0x7f547015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.521+0000 7f546ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5470156e50 0x7f547015a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46562/0 (socket says 172.21.15.27:46562) 2023-08-11T15:51:34.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.521+0000 7f546ffff640 1 -- 172.21.15.27:0/2851024117 learned_addr learned my addr 172.21.15.27:0/2851024117 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:34.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.521+0000 7f546ffff640 1 -- 172.21.15.27:0/2851024117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54700cbb70 msgr2=0x7f54700ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:34.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.521+0000 7f546f7fe640 1 --2- 172.21.15.27:0/2851024117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54700cbb70 0x7f54700ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.521+0000 7f546ffff640 1 --2- 172.21.15.27:0/2851024117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54700cbb70 0x7f54700ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.521+0000 7f546ffff640 1 -- 172.21.15.27:0/2851024117 >> v1:172.21.15.119:6789/0 conn(0x7f54700caef0 legacy=0x7f547015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:34.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.521+0000 7f546ffff640 1 -- 172.21.15.27:0/2851024117 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54700cc270 con 0x7f5470156e50 2023-08-11T15:51:34.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.522+0000 7f546ffff640 1 --2- 172.21.15.27:0/2851024117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5470156e50 0x7f547015a180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5460009ea0 tx=0x7f546002e130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892966a2557ee910 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:34.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.523+0000 7f546effd640 1 -- 172.21.15.27:0/2851024117 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546002ecd0 con 0x7f5470156e50 2023-08-11T15:51:34.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.523+0000 7f546effd640 1 -- 172.21.15.27:0/2851024117 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546002ee30 con 0x7f5470156e50 2023-08-11T15:51:34.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.523+0000 7f546effd640 1 -- 172.21.15.27:0/2851024117 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460036610 con 0x7f5470156e50 2023-08-11T15:51:34.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.523+0000 7f54764ec640 1 -- 172.21.15.27:0/2851024117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5470156e50 msgr2=0x7f547015a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:34.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.523+0000 7f54764ec640 1 --2- 172.21.15.27:0/2851024117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5470156e50 0x7f547015a180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5460009ea0 tx=0x7f546002e130 comp rx=0 tx=0).stop 2023-08-11T15:51:34.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.523+0000 7f54764ec640 1 -- 172.21.15.27:0/2851024117 shutdown_connections 2023-08-11T15:51:34.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.523+0000 7f54764ec640 1 --2- 172.21.15.27:0/2851024117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54700cbb70 0x7f54700ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.523+0000 7f54764ec640 1 --2- 172.21.15.27:0/2851024117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5470156e50 0x7f547015a180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.523+0000 7f54764ec640 1 -- 172.21.15.27:0/2851024117 >> 172.21.15.27:0/2851024117 conn(0x7f54700afb70 msgr2=0x7f54700aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:34.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.524+0000 7f54764ec640 1 -- 172.21.15.27:0/2851024117 shutdown_connections 2023-08-11T15:51:34.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.524+0000 7f54764ec640 1 -- 172.21.15.27:0/2851024117 wait complete. 2023-08-11T15:51:34.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.526+0000 7f54764ec640 1 Processor -- start 2023-08-11T15:51:34.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.526+0000 7f54764ec640 1 -- start start 2023-08-11T15:51:34.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.527+0000 7f54764ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700caef0 0x7f547014e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.527+0000 7f54764ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54700cbb70 0x7f547014eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.527+0000 7f54764ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5470156e50 0x7f54700b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.527+0000 7f54764ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54700c4990 con 0x7f54700caef0 2023-08-11T15:51:34.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.527+0000 7f54764ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54700b1070 con 0x7f54700cbb70 2023-08-11T15:51:34.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.527+0000 7f54764ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54700b38b0 con 0x7f5470156e50 2023-08-11T15:51:34.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f546ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700caef0 0x7f547014e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f546ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700caef0 0x7f547014e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46568/0 (socket says 172.21.15.27:46568) 2023-08-11T15:51:34.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f546ffff640 1 -- 172.21.15.27:0/3315767681 learned_addr learned my addr 172.21.15.27:0/3315767681 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:34.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f5474a89640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5470156e50 0x7f54700b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f546f7fe640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54700cbb70 0x7f547014eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f546ffff640 1 -- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5470156e50 msgr2=0x7f54700b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:34.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f546ffff640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5470156e50 0x7f54700b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f546ffff640 1 -- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54700cbb70 msgr2=0x7f547014eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f546ffff640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54700cbb70 0x7f547014eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f546ffff640 1 -- 172.21.15.27:0/3315767681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460009b30 con 0x7f54700caef0 2023-08-11T15:51:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.528+0000 7f546f7fe640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54700cbb70 0x7f547014eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.529+0000 7f546ffff640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700caef0 0x7f547014e7f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5460009ea0 tx=0x7f5460004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.529+0000 7f546cff9640 1 -- 172.21.15.27:0/3315767681 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460041640 con 0x7f54700caef0 2023-08-11T15:51:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.529+0000 7f546cff9640 1 -- 172.21.15.27:0/3315767681 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54600417c0 con 0x7f54700caef0 2023-08-11T15:51:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.529+0000 7f546cff9640 1 -- 172.21.15.27:0/3315767681 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460041b70 con 0x7f54700caef0 2023-08-11T15:51:34.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.529+0000 7f54764ec640 1 -- 172.21.15.27:0/3315767681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54700c46b0 con 0x7f54700caef0 2023-08-11T15:51:34.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.529+0000 7f54764ec640 1 -- 172.21.15.27:0/3315767681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54700b3540 con 0x7f54700caef0 2023-08-11T15:51:34.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.530+0000 7f54764ec640 1 -- 172.21.15.27:0/3315767681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5470073990 con 0x7f54700caef0 2023-08-11T15:51:34.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.531+0000 7f546cff9640 1 -- 172.21.15.27:0/3315767681 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5460041cd0 con 0x7f54700caef0 2023-08-11T15:51:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.533+0000 7f546cff9640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f545c06a050 0x7f545c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.533+0000 7f546cff9640 1 -- 172.21.15.27:0/3315767681 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f54600ad470 con 0x7f54700caef0 2023-08-11T15:51:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.533+0000 7f546f7fe640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f545c06a050 0x7f545c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:34.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.534+0000 7f546f7fe640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f545c06a050 0x7f545c06c510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54700c42f0 tx=0x7f5464008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:34.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.537+0000 7f546cff9640 1 -- 172.21.15.27:0/3315767681 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f5460078900 con 0x7f54700caef0 2023-08-11T15:51:34.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.732+0000 7f54764ec640 1 -- 172.21.15.27:0/3315767681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54700b36d0 con 0x7f54700caef0 2023-08-11T15:51:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.736+0000 7f546cff9640 1 -- 172.21.15.27:0/3315767681 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7f54700b36d0 con 0x7f54700caef0 2023-08-11T15:51:34.738 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:34.739 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:34.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 -- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f545c06a050 msgr2=0x7f545c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:34.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f545c06a050 0x7f545c06c510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54700c42f0 tx=0x7f5464008040 comp rx=0 tx=0).stop 2023-08-11T15:51:34.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 -- 172.21.15.27:0/3315767681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700caef0 msgr2=0x7f547014e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:34.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700caef0 0x7f547014e7f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5460009ea0 tx=0x7f5460004a40 comp rx=0 tx=0).stop 2023-08-11T15:51:34.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 -- 172.21.15.27:0/3315767681 shutdown_connections 2023-08-11T15:51:34.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f545c06a050 0x7f545c06c510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5470156e50 0x7f54700b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54700cbb70 0x7f547014eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 --2- 172.21.15.27:0/3315767681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700caef0 0x7f547014e7f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:34.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 -- 172.21.15.27:0/3315767681 >> 172.21.15.27:0/3315767681 conn(0x7f54700afb70 msgr2=0x7f54700af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:34.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 -- 172.21.15.27:0/3315767681 shutdown_connections 2023-08-11T15:51:34.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:34.739+0000 7f54764ec640 1 -- 172.21.15.27:0/3315767681 wait complete. 2023-08-11T15:51:34.761 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:51:34.762 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:51:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.000+0000 7f3fb1ae0640 1 Processor -- start 2023-08-11T15:51:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.001+0000 7f3fb1ae0640 1 -- start start 2023-08-11T15:51:35.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.002+0000 7f3fb1ae0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3fac0cb700 0x7f3fac13fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.002+0000 7f3fb1ae0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac15b6b0 0x7f3fac1401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.002+0000 7f3fb1ae0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac140ba0 con 0x7f3fac154ac0 2023-08-11T15:51:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.002+0000 7f3fb1ae0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac071b10 con 0x7f3fac0cb700 2023-08-11T15:51:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.002+0000 7f3fb1ae0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac15b220 con 0x7f3fac15b6b0 2023-08-11T15:51:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.002+0000 7f3faaffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3fac0cb700 0x7f3fac13fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.002+0000 7f3fab7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3fac154ac0 0x7f3fac0caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48564/0 (socket says 172.21.15.27:48564) 2023-08-11T15:51:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.002+0000 7f3fab7fe640 1 -- 172.21.15.27:0/8776531 learned_addr learned my addr 172.21.15.27:0/8776531 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.003+0000 7f3fabfff640 1 --2- 172.21.15.27:0/8776531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac15b6b0 0x7f3fac1401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.003+0000 7f3faaffd640 1 -- 172.21.15.27:0/8776531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac15b6b0 msgr2=0x7f3fac1401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.003+0000 7f3faaffd640 1 --2- 172.21.15.27:0/8776531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac15b6b0 0x7f3fac1401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.003+0000 7f3faaffd640 1 -- 172.21.15.27:0/8776531 >> v1:172.21.15.27:6789/0 conn(0x7f3fac154ac0 legacy=0x7f3fac0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.003+0000 7f3faaffd640 1 -- 172.21.15.27:0/8776531 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac1408c0 con 0x7f3fac0cb700 2023-08-11T15:51:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.003+0000 7f3faa7fc640 1 -- 172.21.15.27:0/8776531 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862871961 0 0) 0x7f3fac140ba0 con 0x7f3fac154ac0 2023-08-11T15:51:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.004+0000 7f3faaffd640 1 --2- 172.21.15.27:0/8776531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0cb700 0x7f3fac13fc90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00097a0 tx=0x7f3fa002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16492db4e38b2dd5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.004+0000 7f3faa7fc640 1 -- 172.21.15.27:0/8776531 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0031780 con 0x7f3fac0cb700 2023-08-11T15:51:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.004+0000 7f3faa7fc640 1 -- 172.21.15.27:0/8776531 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa00318e0 con 0x7f3fac0cb700 2023-08-11T15:51:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.004+0000 7f3faa7fc640 1 -- 172.21.15.27:0/8776531 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0031c70 con 0x7f3fac0cb700 2023-08-11T15:51:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.005+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/8776531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0cb700 msgr2=0x7f3fac13fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.005+0000 7f3fb1ae0640 1 --2- 172.21.15.27:0/8776531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0cb700 0x7f3fac13fc90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00097a0 tx=0x7f3fa002b820 comp rx=0 tx=0).stop 2023-08-11T15:51:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.005+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/8776531 shutdown_connections 2023-08-11T15:51:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.005+0000 7f3fb1ae0640 1 --2- 172.21.15.27:0/8776531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac15b6b0 0x7f3fac1401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.005+0000 7f3fb1ae0640 1 --2- 172.21.15.27:0/8776531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0cb700 0x7f3fac13fc90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.005+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/8776531 >> 172.21.15.27:0/8776531 conn(0x7f3fac069bf0 msgr2=0x7f3fac06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.005+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/8776531 shutdown_connections 2023-08-11T15:51:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.005+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/8776531 wait complete. 2023-08-11T15:51:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.007+0000 7f3fb1ae0640 1 Processor -- start 2023-08-11T15:51:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.007+0000 7f3fb1ae0640 1 -- start start 2023-08-11T15:51:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.008+0000 7f3fb1ae0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac0cb700 0x7f3fac0cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.008+0000 7f3fb1ae0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac154ac0 0x7f3fac0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.008+0000 7f3fb1ae0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac15b6b0 0x7f3fac0ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.008+0000 7f3fb1ae0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac157830 con 0x7f3fac154ac0 2023-08-11T15:51:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.008+0000 7f3fb1ae0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac156790 con 0x7f3fac15b6b0 2023-08-11T15:51:35.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.008+0000 7f3fb1ae0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac156a70 con 0x7f3fac0cb700 2023-08-11T15:51:35.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.008+0000 7f3faaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac154ac0 0x7f3fac0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3faaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac154ac0 0x7f3fac0ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46598/0 (socket says 172.21.15.27:46598) 2023-08-11T15:51:35.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3faaffd640 1 -- 172.21.15.27:0/525644831 learned_addr learned my addr 172.21.15.27:0/525644831 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:35.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3fab7fe640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac0cb700 0x7f3fac0cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3fabfff640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac15b6b0 0x7f3fac0ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3faaffd640 1 -- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac0cb700 msgr2=0x7f3fac0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3faaffd640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac0cb700 0x7f3fac0cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3faaffd640 1 -- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac15b6b0 msgr2=0x7f3fac0ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3faaffd640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac15b6b0 0x7f3fac0ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3faaffd640 1 -- 172.21.15.27:0/525644831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0009400 con 0x7f3fac154ac0 2023-08-11T15:51:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3fab7fe640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac0cb700 0x7f3fac0cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3fabfff640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac15b6b0 0x7f3fac0ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.009+0000 7f3faaffd640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac154ac0 0x7f3fac0ae860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0034040 tx=0x7f3fa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.010+0000 7f3f93fff640 1 -- 172.21.15.27:0/525644831 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0032820 con 0x7f3fac154ac0 2023-08-11T15:51:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.010+0000 7f3f93fff640 1 -- 172.21.15.27:0/525644831 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0032980 con 0x7f3fac154ac0 2023-08-11T15:51:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.010+0000 7f3f93fff640 1 -- 172.21.15.27:0/525644831 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0032d30 con 0x7f3fac154ac0 2023-08-11T15:51:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.010+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/525644831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac156d50 con 0x7f3fac154ac0 2023-08-11T15:51:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.010+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/525644831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac157310 con 0x7f3fac154ac0 2023-08-11T15:51:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.011+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/525644831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac04e420 con 0x7f3fac154ac0 2023-08-11T15:51:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.012+0000 7f3f93fff640 1 -- 172.21.15.27:0/525644831 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3fa004e440 con 0x7f3fac154ac0 2023-08-11T15:51:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.013+0000 7f3f93fff640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3f9806a050 0x7f3f9806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.013+0000 7f3f93fff640 1 -- 172.21.15.27:0/525644831 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3fa00aaac0 con 0x7f3fac154ac0 2023-08-11T15:51:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.014+0000 7f3fab7fe640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3f9806a050 0x7f3f9806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.015+0000 7f3fab7fe640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3f9806a050 0x7f3f9806c510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c000f60 tx=0x7f3f9c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.017+0000 7f3f93fff640 1 -- 172.21.15.27:0/525644831 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f3fa0075e20 con 0x7f3fac154ac0 2023-08-11T15:51:35.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.212+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/525644831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fac06c020 con 0x7f3fac154ac0 2023-08-11T15:51:35.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.213+0000 7f3f93fff640 1 -- 172.21.15.27:0/525644831 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fa0079440 con 0x7f3fac154ac0 2023-08-11T15:51:35.215 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:51:35.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3f9806a050 msgr2=0x7f3f9806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3f9806a050 0x7f3f9806c510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c000f60 tx=0x7f3f9c009040 comp rx=0 tx=0).stop 2023-08-11T15:51:35.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/525644831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac154ac0 msgr2=0x7f3fac0ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac154ac0 0x7f3fac0ae860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0034040 tx=0x7f3fa00046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/525644831 shutdown_connections 2023-08-11T15:51:35.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3f9806a050 0x7f3f9806c510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fac15b6b0 0x7f3fac0ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac154ac0 0x7f3fac0ae860 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 --2- 172.21.15.27:0/525644831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3fac0cb700 0x7f3fac0cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/525644831 >> 172.21.15.27:0/525644831 conn(0x7f3fac069bf0 msgr2=0x7f3fac15e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:35.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/525644831 shutdown_connections 2023-08-11T15:51:35.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.216+0000 7f3fb1ae0640 1 -- 172.21.15.27:0/525644831 wait complete. 2023-08-11T15:51:35.237 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:51:35.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.472+0000 7fe0c62e0640 1 Processor -- start 2023-08-11T15:51:35.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.472+0000 7fe0c62e0640 1 -- start start 2023-08-11T15:51:35.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.474+0000 7fe0c62e0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe0c013ce80 0x7fe0c013f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.474+0000 7fe0c62e0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe0c013d260 0x7fe0c013fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.474+0000 7fe0c62e0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c00cc720 0x7fe0c00ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.474+0000 7fe0c62e0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c0140670 con 0x7fe0c013ce80 2023-08-11T15:51:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.474+0000 7fe0c62e0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00cd470 con 0x7fe0c013d260 2023-08-11T15:51:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.474+0000 7fe0c62e0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00cd750 con 0x7fe0c00cc720 2023-08-11T15:51:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.474+0000 7fe0c52de640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe0c013ce80 0x7fe0c013f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c52de640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe0c013ce80 0x7fe0c013f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46604/0 (socket says 172.21.15.27:46604) 2023-08-11T15:51:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c52de640 1 -- 172.21.15.27:0/3343579654 learned_addr learned my addr 172.21.15.27:0/3343579654 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:35.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c5adf640 1 --2- 172.21.15.27:0/3343579654 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c00cc720 0x7fe0c00ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c4add640 1 --2- 172.21.15.27:0/3343579654 >> v2:172.21.15.119:3300/0 conn(0x7fe0c013d260 0x7fe0c013fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c52de640 1 -- 172.21.15.27:0/3343579654 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c00cc720 msgr2=0x7fe0c00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:35.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c52de640 1 --2- 172.21.15.27:0/3343579654 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c00cc720 0x7fe0c00ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c52de640 1 -- 172.21.15.27:0/3343579654 >> v2:172.21.15.119:3300/0 conn(0x7fe0c013d260 msgr2=0x7fe0c013fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:35.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c52de640 1 --2- 172.21.15.27:0/3343579654 >> v2:172.21.15.119:3300/0 conn(0x7fe0c013d260 0x7fe0c013fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c52de640 1 -- 172.21.15.27:0/3343579654 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00ad7d0 con 0x7fe0c013ce80 2023-08-11T15:51:35.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c4add640 1 --2- 172.21.15.27:0/3343579654 >> v2:172.21.15.119:3300/0 conn(0x7fe0c013d260 0x7fe0c013fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:35.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.475+0000 7fe0c5adf640 1 --2- 172.21.15.27:0/3343579654 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c00cc720 0x7fe0c00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:35.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.476+0000 7fe0c52de640 1 --2- 172.21.15.27:0/3343579654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c013ce80 0x7fe0c013f810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00cc260 tx=0x7fe0bc02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd0cf20645cf2dc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:35.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.476+0000 7fe0b7fff640 1 -- 172.21.15.27:0/3343579654 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc031880 con 0x7fe0c013ce80 2023-08-11T15:51:35.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0b7fff640 1 -- 172.21.15.27:0/3343579654 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc0319e0 con 0x7fe0c013ce80 2023-08-11T15:51:35.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0b7fff640 1 -- 172.21.15.27:0/3343579654 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc031d70 con 0x7fe0c013ce80 2023-08-11T15:51:35.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3343579654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c013ce80 msgr2=0x7fe0c013f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0c62e0640 1 --2- 172.21.15.27:0/3343579654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c013ce80 0x7fe0c013f810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00cc260 tx=0x7fe0bc02b920 comp rx=0 tx=0).stop 2023-08-11T15:51:35.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3343579654 shutdown_connections 2023-08-11T15:51:35.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0c62e0640 1 --2- 172.21.15.27:0/3343579654 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c00cc720 0x7fe0c00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0c62e0640 1 --2- 172.21.15.27:0/3343579654 >> v2:172.21.15.119:3300/0 conn(0x7fe0c013d260 0x7fe0c013fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0c62e0640 1 --2- 172.21.15.27:0/3343579654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c013ce80 0x7fe0c013f810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3343579654 >> 172.21.15.27:0/3343579654 conn(0x7fe0c0069800 msgr2=0x7fe0c0069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:35.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3343579654 shutdown_connections 2023-08-11T15:51:35.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.477+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3343579654 wait complete. 2023-08-11T15:51:35.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.479+0000 7fe0c62e0640 1 Processor -- start 2023-08-11T15:51:35.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.479+0000 7fe0c62e0640 1 -- start start 2023-08-11T15:51:35.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.480+0000 7fe0c62e0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c00cc720 0x7fe0c01f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.480+0000 7fe0c62e0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c013ce80 0x7fe0c00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.480+0000 7fe0c62e0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c013d260 0x7fe0c00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.480+0000 7fe0c62e0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c013fdc0 con 0x7fe0c00cc720 2023-08-11T15:51:35.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.480+0000 7fe0c62e0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c015bfa0 con 0x7fe0c013d260 2023-08-11T15:51:35.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.480+0000 7fe0c62e0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c013d770 con 0x7fe0c013ce80 2023-08-11T15:51:35.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c52de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c00cc720 0x7fe0c01f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c52de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c00cc720 0x7fe0c01f12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46612/0 (socket says 172.21.15.27:46612) 2023-08-11T15:51:35.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c52de640 1 -- 172.21.15.27:0/3175294571 learned_addr learned my addr 172.21.15.27:0/3175294571 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:35.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c4add640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c013ce80 0x7fe0c00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c5adf640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c013d260 0x7fe0c00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c52de640 1 -- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c013ce80 msgr2=0x7fe0c00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c52de640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c013ce80 0x7fe0c00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c52de640 1 -- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c013d260 msgr2=0x7fe0c00b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c52de640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c013d260 0x7fe0c00b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c52de640 1 -- 172.21.15.27:0/3175294571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc0094d0 con 0x7fe0c00cc720 2023-08-11T15:51:35.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c4add640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c013ce80 0x7fe0c00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:35.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c5adf640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c013d260 0x7fe0c00b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:35.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.481+0000 7fe0c52de640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c00cc720 0x7fe0c01f12b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc000c00 tx=0x7fe0bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:35.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.482+0000 7fe0b5ffb640 1 -- 172.21.15.27:0/3175294571 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc031870 con 0x7fe0c00cc720 2023-08-11T15:51:35.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.482+0000 7fe0b5ffb640 1 -- 172.21.15.27:0/3175294571 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc0443f0 con 0x7fe0c00cc720 2023-08-11T15:51:35.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.482+0000 7fe0b5ffb640 1 -- 172.21.15.27:0/3175294571 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0447a0 con 0x7fe0c00cc720 2023-08-11T15:51:35.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.482+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3175294571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c013da50 con 0x7fe0c00cc720 2023-08-11T15:51:35.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.482+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3175294571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c013e010 con 0x7fe0c00cc720 2023-08-11T15:51:35.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.483+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3175294571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c006ca10 con 0x7fe0c00cc720 2023-08-11T15:51:35.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.484+0000 7fe0b5ffb640 1 -- 172.21.15.27:0/3175294571 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe0bc044900 con 0x7fe0c00cc720 2023-08-11T15:51:35.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.485+0000 7fe0b5ffb640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe0a406a050 0x7fe0a406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.485+0000 7fe0b5ffb640 1 -- 172.21.15.27:0/3175294571 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe0bc0ab060 con 0x7fe0c00cc720 2023-08-11T15:51:35.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.486+0000 7fe0c4add640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe0a406a050 0x7fe0a406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.487+0000 7fe0c4add640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe0a406a050 0x7fe0a406c510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0006fd0 tx=0x7fe0b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:35.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.489+0000 7fe0b5ffb640 1 -- 172.21.15.27:0/3175294571 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe0bc0764f0 con 0x7fe0c00cc720 2023-08-11T15:51:35.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.688+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3175294571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe0c0066e80 con 0x7fe0c00cc720 2023-08-11T15:51:35.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.689+0000 7fe0b5ffb640 1 -- 172.21.15.27:0/3175294571 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe0bc035020 con 0x7fe0c00cc720 2023-08-11T15:51:35.691 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:51:35.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe0a406a050 msgr2=0x7fe0a406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe0a406a050 0x7fe0a406c510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0006fd0 tx=0x7fe0b0008040 comp rx=0 tx=0).stop 2023-08-11T15:51:35.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3175294571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c00cc720 msgr2=0x7fe0c01f12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c00cc720 0x7fe0c01f12b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc000c00 tx=0x7fe0bc0046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3175294571 shutdown_connections 2023-08-11T15:51:35.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe0a406a050 0x7fe0a406c510 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0c013ce80 0x7fe0c00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c013d260 0x7fe0c00b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 --2- 172.21.15.27:0/3175294571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c00cc720 0x7fe0c01f12b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3175294571 >> 172.21.15.27:0/3175294571 conn(0x7fe0c0069800 msgr2=0x7fe0c006c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:35.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3175294571 shutdown_connections 2023-08-11T15:51:35.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.692+0000 7fe0c62e0640 1 -- 172.21.15.27:0/3175294571 wait complete. 2023-08-11T15:51:35.713 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:51:35.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.951+0000 7f79e34ad640 1 Processor -- start 2023-08-11T15:51:35.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.951+0000 7f79e34ad640 1 -- start start 2023-08-11T15:51:35.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.953+0000 7f79e34ad640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79dc0b38f0 0x7f79dc0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.953+0000 7f79e34ad640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79dc0b3090 0x7f79dc160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.953+0000 7f79e34ad640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79dc0b3cd0 0x7f79dc161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.953+0000 7f79e34ad640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc0b0190 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.953+0000 7f79e34ad640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc0af250 con 0x7f79dc0b38f0 2023-08-11T15:51:35.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.953+0000 7f79e34ad640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0af510 con 0x7f79dc0b3090 2023-08-11T15:51:35.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.954+0000 7f79e1a4a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79dc0b3cd0 0x7f79dc161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.954+0000 7f79e1a4a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79dc0b3cd0 0x7f79dc161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46628/0 (socket says 172.21.15.27:46628) 2023-08-11T15:51:35.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.954+0000 7f79e1a4a640 1 -- 172.21.15.27:0/1992583864 learned_addr learned my addr 172.21.15.27:0/1992583864 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:35.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.954+0000 7f79e0a48640 1 --2- 172.21.15.27:0/1992583864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79dc0b3090 0x7f79dc160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.954+0000 7f79e1249640 1 --2- 172.21.15.27:0/1992583864 >> v2:172.21.15.119:3300/0 conn(0x7f79dc0b38f0 0x7f79dc0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.954+0000 7f79e1a4a640 1 -- 172.21.15.27:0/1992583864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79dc0b3090 msgr2=0x7f79dc160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:35.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.954+0000 7f79e1a4a640 1 --2- 172.21.15.27:0/1992583864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79dc0b3090 0x7f79dc160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.954+0000 7f79e1a4a640 1 -- 172.21.15.27:0/1992583864 >> v2:172.21.15.119:3300/0 conn(0x7f79dc0b38f0 msgr2=0x7f79dc0d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:35.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.954+0000 7f79e1a4a640 1 --2- 172.21.15.27:0/1992583864 >> v2:172.21.15.119:3300/0 conn(0x7f79dc0b38f0 0x7f79dc0d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.954+0000 7f79e1a4a640 1 -- 172.21.15.27:0/1992583864 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc0af7f0 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.955+0000 7f79e1a4a640 1 --2- 172.21.15.27:0/1992583864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 0x7f79dc161040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79dc08b230 tx=0x7f79d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abc29e143d445d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:35.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.956+0000 7f79d3fff640 1 -- 172.21.15.27:0/1992583864 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d402ca80 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.956+0000 7f79d3fff640 1 -- 172.21.15.27:0/1992583864 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d402cbe0 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.956+0000 7f79d3fff640 1 -- 172.21.15.27:0/1992583864 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d40343f0 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.956+0000 7f79e34ad640 1 -- 172.21.15.27:0/1992583864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 msgr2=0x7f79dc161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.956+0000 7f79e34ad640 1 --2- 172.21.15.27:0/1992583864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 0x7f79dc161040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79dc08b230 tx=0x7f79d402bee0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.957+0000 7f79e34ad640 1 -- 172.21.15.27:0/1992583864 shutdown_connections 2023-08-11T15:51:35.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.957+0000 7f79e34ad640 1 --2- 172.21.15.27:0/1992583864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 0x7f79dc161040 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.957+0000 7f79e34ad640 1 --2- 172.21.15.27:0/1992583864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79dc0b3090 0x7f79dc160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.957+0000 7f79e34ad640 1 --2- 172.21.15.27:0/1992583864 >> v2:172.21.15.119:3300/0 conn(0x7f79dc0b38f0 0x7f79dc0d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.957+0000 7f79e34ad640 1 -- 172.21.15.27:0/1992583864 >> 172.21.15.27:0/1992583864 conn(0x7f79dc09b960 msgr2=0x7f79dc0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:35.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.957+0000 7f79e34ad640 1 -- 172.21.15.27:0/1992583864 shutdown_connections 2023-08-11T15:51:35.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.957+0000 7f79e34ad640 1 -- 172.21.15.27:0/1992583864 wait complete. 2023-08-11T15:51:35.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.959+0000 7f79e34ad640 1 Processor -- start 2023-08-11T15:51:35.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.959+0000 7f79e34ad640 1 -- start start 2023-08-11T15:51:35.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e34ad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b3090 0x7f79dc0b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e34ad640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79dc0b38f0 0x7f79dc1e7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e34ad640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 0x7f79dc14b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e34ad640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0b7e80 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e34ad640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0b80b0 con 0x7f79dc0b3090 2023-08-11T15:51:35.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e34ad640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0b5d70 con 0x7f79dc0b38f0 2023-08-11T15:51:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e1a4a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 0x7f79dc14b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e1a4a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 0x7f79dc14b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46638/0 (socket says 172.21.15.27:46638) 2023-08-11T15:51:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e1a4a640 1 -- 172.21.15.27:0/2629087860 learned_addr learned my addr 172.21.15.27:0/2629087860 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e0a48640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79dc0b38f0 0x7f79dc1e7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e1249640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b3090 0x7f79dc0b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e1a4a640 1 -- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79dc0b38f0 msgr2=0x7f79dc1e7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.960+0000 7f79e1a4a640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79dc0b38f0 0x7f79dc1e7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.961+0000 7f79e1a4a640 1 -- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b3090 msgr2=0x7f79dc0b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:35.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.961+0000 7f79e1a4a640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b3090 0x7f79dc0b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:35.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.961+0000 7f79e1a4a640 1 -- 172.21.15.27:0/2629087860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4009b30 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.961+0000 7f79e1a4a640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 0x7f79dc14b960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f79d4036040 tx=0x7f79d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:35.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.961+0000 7f79d1ffb640 1 -- 172.21.15.27:0/2629087860 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4009510 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.961+0000 7f79d1ffb640 1 -- 172.21.15.27:0/2629087860 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d403e450 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.961+0000 7f79d1ffb640 1 -- 172.21.15.27:0/2629087860 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d403e800 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.961+0000 7f79e34ad640 1 -- 172.21.15.27:0/2629087860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc0b6050 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.961+0000 7f79e34ad640 1 -- 172.21.15.27:0/2629087860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc0b6610 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.963+0000 7f79e34ad640 1 -- 172.21.15.27:0/2629087860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc0b1fb0 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.963+0000 7f79d1ffb640 1 -- 172.21.15.27:0/2629087860 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f79d402cd50 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.964+0000 7f79d1ffb640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79c006a050 0x7f79c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:35.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.964+0000 7f79d1ffb640 1 -- 172.21.15.27:0/2629087860 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f79d40ab5f0 con 0x7f79dc0b3cd0 2023-08-11T15:51:35.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.965+0000 7f79e1249640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79c006a050 0x7f79c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:35.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.966+0000 7f79e1249640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79c006a050 0x7f79c006c510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009ac0 tx=0x7f79d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:35.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:35.968+0000 7f79d1ffb640 1 -- 172.21.15.27:0/2629087860 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f79dc0b1fb0 con 0x7f79dc0b3cd0 2023-08-11T15:51:36.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.159+0000 7f79e34ad640 1 -- 172.21.15.27:0/2629087860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79dc0b1fb0 con 0x7f79dc0b3cd0 2023-08-11T15:51:36.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.160+0000 7f79d1ffb640 1 -- 172.21.15.27:0/2629087860 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79dc0b1fb0 con 0x7f79dc0b3cd0 2023-08-11T15:51:36.162 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:51:36.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 -- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79c006a050 msgr2=0x7f79c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79c006a050 0x7f79c006c510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009ac0 tx=0x7f79d8008040 comp rx=0 tx=0).stop 2023-08-11T15:51:36.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 -- 172.21.15.27:0/2629087860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 msgr2=0x7f79dc14b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 0x7f79dc14b960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f79d4036040 tx=0x7f79d4004a40 comp rx=0 tx=0).stop 2023-08-11T15:51:36.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 -- 172.21.15.27:0/2629087860 shutdown_connections 2023-08-11T15:51:36.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79c006a050 0x7f79c006c510 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79dc0b38f0 0x7f79dc1e7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79dc0b3cd0 0x7f79dc14b960 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 --2- 172.21.15.27:0/2629087860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b3090 0x7f79dc0b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 -- 172.21.15.27:0/2629087860 >> 172.21.15.27:0/2629087860 conn(0x7f79dc09b960 msgr2=0x7f79dc0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 -- 172.21.15.27:0/2629087860 shutdown_connections 2023-08-11T15:51:36.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.163+0000 7f79e34ad640 1 -- 172.21.15.27:0/2629087860 wait complete. 2023-08-11T15:51:36.184 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:51:36.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.422+0000 7fe2c7487640 1 Processor -- start 2023-08-11T15:51:36.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.422+0000 7fe2c7487640 1 -- start start 2023-08-11T15:51:36.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.424+0000 7fe2c7487640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2c00b3090 0x7fe2c00d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.424+0000 7fe2c7487640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c00af250 con 0x7fe2c0161210 2023-08-11T15:51:36.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.424+0000 7fe2c7487640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c00af510 con 0x7fe2c00b38f0 2023-08-11T15:51:36.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.424+0000 7fe2c7487640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00b0190 con 0x7fe2c00b3090 2023-08-11T15:51:36.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.424+0000 7fe2c5a24640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe2c0161210 0x7fe2c00b05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48568/0 (socket says 172.21.15.27:48568) 2023-08-11T15:51:36.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.424+0000 7fe2c5a24640 1 -- 172.21.15.27:0/3123379339 learned_addr learned my addr 172.21.15.27:0/3123379339 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:36.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.424+0000 7fe2c4a22640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2c00b3090 0x7fe2c00d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.425+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170795909 0 0) 0x7fe2c00af250 con 0x7fe2c0161210 2023-08-11T15:51:36.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.425+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ac003520 con 0x7fe2c0161210 2023-08-11T15:51:36.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.425+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796113838 0 0) 0x7fe2c00af510 con 0x7fe2c00b38f0 2023-08-11T15:51:36.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.425+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c00af250 con 0x7fe2c00b38f0 2023-08-11T15:51:36.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.425+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2797578800 0 0) 0x7fe2ac003520 con 0x7fe2c0161210 2023-08-11T15:51:36.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.425+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2c00b3090 msgr2=0x7fe2c00d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:36.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.425+0000 7fe2b7fff640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2c00b3090 0x7fe2c00d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.425+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 >> v1:172.21.15.119:6789/0 conn(0x7fe2c00b38f0 legacy=0x7fe2c0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.425+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00af7f0 con 0x7fe2c0161210 2023-08-11T15:51:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.425+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fe2b80019f0 con 0x7fe2c0161210 2023-08-11T15:51:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 >> v1:172.21.15.27:6789/0 conn(0x7fe2c0161210 legacy=0x7fe2c00b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2b7fff640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2c00b3090 0x7fe2ac003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2b7fff640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2ac004090 0x7fe2ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2b7fff640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ac004d00 0x7fe2ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00b0190 con 0x7fe2ac004090 2023-08-11T15:51:36.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac00ba60 con 0x7fe2ac004d00 2023-08-11T15:51:36.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac00bc20 con 0x7fe2c00b3090 2023-08-11T15:51:36.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2c4a22640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2ac004090 0x7fe2ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2c5a24640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ac004d00 0x7fe2ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2c5223640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2c00b3090 0x7fe2ac003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2c4a22640 1 -- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2c00b3090 msgr2=0x7fe2ac003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:36.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2c4a22640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2c00b3090 0x7fe2ac003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2c4a22640 1 -- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ac004d00 msgr2=0x7fe2ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:36.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.426+0000 7fe2c4a22640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ac004d00 0x7fe2ac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.427+0000 7fe2c4a22640 1 -- 172.21.15.27:0/3123379339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac003520 con 0x7fe2ac004090 2023-08-11T15:51:36.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.427+0000 7fe2c5223640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2c00b3090 0x7fe2ac003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:36.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.428+0000 7fe2c4a22640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2ac004090 0x7fe2ac0047c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009a90 tx=0x7fe2ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422308028b3ae2a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:36.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.428+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b002c680 con 0x7fe2ac004090 2023-08-11T15:51:36.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.428+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b002c7e0 con 0x7fe2ac004090 2023-08-11T15:51:36.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.428+0000 7fe2b7fff640 1 -- 172.21.15.27:0/3123379339 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b002cb70 con 0x7fe2ac004090 2023-08-11T15:51:36.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.428+0000 7fe2c7487640 1 -- 172.21.15.27:0/3123379339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2ac004090 msgr2=0x7fe2ac0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.428+0000 7fe2c7487640 1 --2- 172.21.15.27:0/3123379339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2ac004090 0x7fe2ac0047c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009a90 tx=0x7fe2ac0057f0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.428+0000 7fe2c5a24640 1 -- 172.21.15.27:0/3123379339 reap_dead start 2023-08-11T15:51:36.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.429+0000 7fe2c7487640 1 -- 172.21.15.27:0/3123379339 shutdown_connections 2023-08-11T15:51:36.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.429+0000 7fe2c7487640 1 -- 172.21.15.27:0/3123379339 >> 172.21.15.27:0/3123379339 conn(0x7fe2c009b960 msgr2=0x7fe2c00d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:36.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.429+0000 7fe2c7487640 1 -- 172.21.15.27:0/3123379339 shutdown_connections 2023-08-11T15:51:36.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.429+0000 7fe2c7487640 1 -- 172.21.15.27:0/3123379339 wait complete. 2023-08-11T15:51:36.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.430+0000 7fe2c7487640 1 Processor -- start 2023-08-11T15:51:36.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.431+0000 7fe2c7487640 1 -- start start 2023-08-11T15:51:36.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c7487640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2ac004090 0x7fe2c00cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c7487640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c00cf3f0 0x7fe2c00cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c7487640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2c01548d0 0x7fe2c0154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c7487640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c01552c0 con 0x7fe2c01548d0 2023-08-11T15:51:36.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c7487640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00cf870 con 0x7fe2c00cf3f0 2023-08-11T15:51:36.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c7487640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00cdfc0 con 0x7fe2ac004090 2023-08-11T15:51:36.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c5a24640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2c01548d0 0x7fe2c0154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c5a24640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2c01548d0 0x7fe2c0154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46658/0 (socket says 172.21.15.27:46658) 2023-08-11T15:51:36.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c5a24640 1 -- 172.21.15.27:0/3110254421 learned_addr learned my addr 172.21.15.27:0/3110254421 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:36.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c5223640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2ac004090 0x7fe2c00cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c4a22640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c00cf3f0 0x7fe2c00cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c5a24640 1 -- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2ac004090 msgr2=0x7fe2c00cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c5a24640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2ac004090 0x7fe2c00cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c5a24640 1 -- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c00cf3f0 msgr2=0x7fe2c00cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c5a24640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c00cf3f0 0x7fe2c00cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.432+0000 7fe2c5a24640 1 -- 172.21.15.27:0/3110254421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00ce2a0 con 0x7fe2c01548d0 2023-08-11T15:51:36.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.433+0000 7fe2c5a24640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2c01548d0 0x7fe2c0154d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8001780 tx=0x7fe2b8007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:36.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.433+0000 7fe2b5ffb640 1 -- 172.21.15.27:0/3110254421 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b800ea40 con 0x7fe2c01548d0 2023-08-11T15:51:36.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.433+0000 7fe2b5ffb640 1 -- 172.21.15.27:0/3110254421 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b800ebc0 con 0x7fe2c01548d0 2023-08-11T15:51:36.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.433+0000 7fe2b5ffb640 1 -- 172.21.15.27:0/3110254421 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8013420 con 0x7fe2c01548d0 2023-08-11T15:51:36.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.433+0000 7fe2c7487640 1 -- 172.21.15.27:0/3110254421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c00ce580 con 0x7fe2c01548d0 2023-08-11T15:51:36.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.433+0000 7fe2c7487640 1 -- 172.21.15.27:0/3110254421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c0155db0 con 0x7fe2c01548d0 2023-08-11T15:51:36.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.434+0000 7fe2c7487640 1 -- 172.21.15.27:0/3110254421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c004e420 con 0x7fe2c01548d0 2023-08-11T15:51:36.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.435+0000 7fe2b5ffb640 1 -- 172.21.15.27:0/3110254421 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe2b8013580 con 0x7fe2c01548d0 2023-08-11T15:51:36.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.436+0000 7fe2b5ffb640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe2a406a080 0x7fe2a406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.436+0000 7fe2b5ffb640 1 -- 172.21.15.27:0/3110254421 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe2b8088bb0 con 0x7fe2c01548d0 2023-08-11T15:51:36.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.437+0000 7fe2c5223640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe2a406a080 0x7fe2a406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.438+0000 7fe2c5223640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe2a406a080 0x7fe2a406c540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc000fb0 tx=0x7fe2bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:36.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.440+0000 7fe2b5ffb640 1 -- 172.21.15.27:0/3110254421 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe2b8053fa0 con 0x7fe2c01548d0 2023-08-11T15:51:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.635+0000 7fe2c7487640 1 -- 172.21.15.27:0/3110254421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2c00b86b0 con 0x7fe2c01548d0 2023-08-11T15:51:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.636+0000 7fe2b5ffb640 1 -- 172.21.15.27:0/3110254421 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2b80575c0 con 0x7fe2c01548d0 2023-08-11T15:51:36.638 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:51:36.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.639+0000 7fe2c7487640 1 -- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe2a406a080 msgr2=0x7fe2a406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.639+0000 7fe2c7487640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe2a406a080 0x7fe2a406c540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc000fb0 tx=0x7fe2bc009040 comp rx=0 tx=0).stop 2023-08-11T15:51:36.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.639+0000 7fe2c7487640 1 -- 172.21.15.27:0/3110254421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2c01548d0 msgr2=0x7fe2c0154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.639+0000 7fe2c7487640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2c01548d0 0x7fe2c0154d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8001780 tx=0x7fe2b8007b70 comp rx=0 tx=0).stop 2023-08-11T15:51:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.639+0000 7fe2c7487640 1 -- 172.21.15.27:0/3110254421 shutdown_connections 2023-08-11T15:51:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.639+0000 7fe2c7487640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe2a406a080 0x7fe2a406c540 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.639+0000 7fe2c7487640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2c01548d0 0x7fe2c0154d80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.639+0000 7fe2c7487640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c00cf3f0 0x7fe2c00cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.639+0000 7fe2c7487640 1 --2- 172.21.15.27:0/3110254421 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe2ac004090 0x7fe2c00cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.640+0000 7fe2c7487640 1 -- 172.21.15.27:0/3110254421 >> 172.21.15.27:0/3110254421 conn(0x7fe2c009b960 msgr2=0x7fe2c00b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.640+0000 7fe2c7487640 1 -- 172.21.15.27:0/3110254421 shutdown_connections 2023-08-11T15:51:36.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.640+0000 7fe2c7487640 1 -- 172.21.15.27:0/3110254421 wait complete. 2023-08-11T15:51:36.660 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:51:36.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.899+0000 7f5692277640 1 Processor -- start 2023-08-11T15:51:36.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.899+0000 7f5692277640 1 -- start start 2023-08-11T15:51:36.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5692277640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 0x7f568c0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5692277640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f568c0ad650 0x7f568c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5692277640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f568c0c9af0 0x7f568c15df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5692277640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c0adab0 con 0x7f568c0ad650 2023-08-11T15:51:36.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5692277640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c0cbe40 con 0x7f568c0c9af0 2023-08-11T15:51:36.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5692277640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0cc400 con 0x7f568c15a380 2023-08-11T15:51:36.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5690a74640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f568c0ad650 0x7f568c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5690a74640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f568c0ad650 0x7f568c0c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46666/0 (socket says 172.21.15.27:46666) 2023-08-11T15:51:36.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5690a74640 1 -- 172.21.15.27:0/2473532268 learned_addr learned my addr 172.21.15.27:0/2473532268 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:36.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5691275640 1 --2- 172.21.15.27:0/2473532268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 0x7f568c0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.901+0000 7f5691a76640 1 --2- 172.21.15.27:0/2473532268 >> v2:172.21.15.119:3300/0 conn(0x7f568c0c9af0 0x7f568c15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.902+0000 7f5690a74640 1 -- 172.21.15.27:0/2473532268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 msgr2=0x7f568c0ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:36.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.902+0000 7f5690a74640 1 --2- 172.21.15.27:0/2473532268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 0x7f568c0ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.902+0000 7f5690a74640 1 -- 172.21.15.27:0/2473532268 >> v2:172.21.15.119:3300/0 conn(0x7f568c0c9af0 msgr2=0x7f568c15df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:36.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.902+0000 7f5690a74640 1 --2- 172.21.15.27:0/2473532268 >> v2:172.21.15.119:3300/0 conn(0x7f568c0c9af0 0x7f568c15df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.902+0000 7f5690a74640 1 -- 172.21.15.27:0/2473532268 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0ca320 con 0x7f568c0ad650 2023-08-11T15:51:36.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.902+0000 7f5691a76640 1 --2- 172.21.15.27:0/2473532268 >> v2:172.21.15.119:3300/0 conn(0x7f568c0c9af0 0x7f568c15df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:36.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.903+0000 7f5690a74640 1 --2- 172.21.15.27:0/2473532268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0ad650 0x7f568c0c95b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f568c09bc40 tx=0x7f567c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d91b57bdc2b936 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:36.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.903+0000 7f5683fff640 1 -- 172.21.15.27:0/2473532268 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c031860 con 0x7f568c0ad650 2023-08-11T15:51:36.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.903+0000 7f5683fff640 1 -- 172.21.15.27:0/2473532268 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c0319c0 con 0x7f568c0ad650 2023-08-11T15:51:36.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.903+0000 7f5683fff640 1 -- 172.21.15.27:0/2473532268 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c031d50 con 0x7f568c0ad650 2023-08-11T15:51:36.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.904+0000 7f5692277640 1 -- 172.21.15.27:0/2473532268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0ad650 msgr2=0x7f568c0c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.904+0000 7f5692277640 1 --2- 172.21.15.27:0/2473532268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0ad650 0x7f568c0c95b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f568c09bc40 tx=0x7f567c02b900 comp rx=0 tx=0).stop 2023-08-11T15:51:36.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.904+0000 7f5692277640 1 -- 172.21.15.27:0/2473532268 shutdown_connections 2023-08-11T15:51:36.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.904+0000 7f5692277640 1 --2- 172.21.15.27:0/2473532268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0ad650 0x7f568c0c95b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.904+0000 7f5692277640 1 --2- 172.21.15.27:0/2473532268 >> v2:172.21.15.119:3300/0 conn(0x7f568c0c9af0 0x7f568c15df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.904+0000 7f5692277640 1 --2- 172.21.15.27:0/2473532268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 0x7f568c0ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.904+0000 7f5692277640 1 -- 172.21.15.27:0/2473532268 >> 172.21.15.27:0/2473532268 conn(0x7f568c0cacf0 msgr2=0x7f568c0cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:36.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.904+0000 7f5692277640 1 -- 172.21.15.27:0/2473532268 shutdown_connections 2023-08-11T15:51:36.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.904+0000 7f5692277640 1 -- 172.21.15.27:0/2473532268 wait complete. 2023-08-11T15:51:36.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.906+0000 7f5692277640 1 Processor -- start 2023-08-11T15:51:36.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.906+0000 7f5692277640 1 -- start start 2023-08-11T15:51:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.907+0000 7f5692277640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f568c0ad650 0x7f568c14f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.907+0000 7f5692277640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0c9af0 0x7f568c0c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.907+0000 7f5692277640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 0x7f568c0c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.907+0000 7f5692277640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c06b2c0 con 0x7f568c0c9af0 2023-08-11T15:51:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.907+0000 7f5692277640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0af450 con 0x7f568c0ad650 2023-08-11T15:51:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.907+0000 7f5692277640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0af730 con 0x7f568c15a380 2023-08-11T15:51:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.907+0000 7f5690a74640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0c9af0 0x7f568c0c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.907+0000 7f5690a74640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0c9af0 0x7f568c0c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46678/0 (socket says 172.21.15.27:46678) 2023-08-11T15:51:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.907+0000 7f5690a74640 1 -- 172.21.15.27:0/4268921425 learned_addr learned my addr 172.21.15.27:0/4268921425 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.907+0000 7f5691275640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f568c0ad650 0x7f568c14f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5691a76640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 0x7f568c0c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5690a74640 1 -- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 msgr2=0x7f568c0c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5690a74640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 0x7f568c0c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5690a74640 1 -- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f568c0ad650 msgr2=0x7f568c14f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5690a74640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f568c0ad650 0x7f568c14f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:36.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5690a74640 1 -- 172.21.15.27:0/4268921425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c0094d0 con 0x7f568c0c9af0 2023-08-11T15:51:36.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5691a76640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 0x7f568c0c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:36.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5690a74640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0c9af0 0x7f568c0c4540 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f567c0097c0 tx=0x7f567c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:36.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5681ffb640 1 -- 172.21.15.27:0/4268921425 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c03ac60 con 0x7f568c0c9af0 2023-08-11T15:51:36.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5681ffb640 1 -- 172.21.15.27:0/4268921425 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c03adc0 con 0x7f568c0c9af0 2023-08-11T15:51:36.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5681ffb640 1 -- 172.21.15.27:0/4268921425 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c044610 con 0x7f568c0c9af0 2023-08-11T15:51:36.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.908+0000 7f5692277640 1 -- 172.21.15.27:0/4268921425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c06b560 con 0x7f568c0c9af0 2023-08-11T15:51:36.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.909+0000 7f5692277640 1 -- 172.21.15.27:0/4268921425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c14fc60 con 0x7f568c0c9af0 2023-08-11T15:51:36.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.910+0000 7f5692277640 1 -- 172.21.15.27:0/4268921425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c04e420 con 0x7f568c0c9af0 2023-08-11T15:51:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.911+0000 7f5681ffb640 1 -- 172.21.15.27:0/4268921425 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f567c04d450 con 0x7f568c0c9af0 2023-08-11T15:51:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.912+0000 7f5681ffb640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f567006a050 0x7f567006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.912+0000 7f5681ffb640 1 -- 172.21.15.27:0/4268921425 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f567c0aae20 con 0x7f568c0c9af0 2023-08-11T15:51:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.912+0000 7f5691275640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f567006a050 0x7f567006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.913+0000 7f5691275640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f567006a050 0x7f567006c510 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5688006fd0 tx=0x7f5688008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:36.916+0000 7f5681ffb640 1 -- 172.21.15.27:0/4268921425 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f567c076210 con 0x7f568c0c9af0 2023-08-11T15:51:37.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.111+0000 7f5692277640 1 -- 172.21.15.27:0/4268921425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f568c0b0bf0 con 0x7f568c0c9af0 2023-08-11T15:51:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.112+0000 7f5681ffb640 1 -- 172.21.15.27:0/4268921425 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f567c035020 con 0x7f568c0c9af0 2023-08-11T15:51:37.113 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:51:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 -- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f567006a050 msgr2=0x7f567006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f567006a050 0x7f567006c510 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5688006fd0 tx=0x7f5688008040 comp rx=0 tx=0).stop 2023-08-11T15:51:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 -- 172.21.15.27:0/4268921425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0c9af0 msgr2=0x7f568c0c4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0c9af0 0x7f568c0c4540 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f567c0097c0 tx=0x7f567c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 -- 172.21.15.27:0/4268921425 shutdown_connections 2023-08-11T15:51:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f567006a050 0x7f567006c510 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f568c15a380 0x7f568c0c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f568c0c9af0 0x7f568c0c4540 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 --2- 172.21.15.27:0/4268921425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f568c0ad650 0x7f568c14f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 -- 172.21.15.27:0/4268921425 >> 172.21.15.27:0/4268921425 conn(0x7f568c0cacf0 msgr2=0x7f568c0ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:37.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.114+0000 7f5692277640 1 -- 172.21.15.27:0/4268921425 shutdown_connections 2023-08-11T15:51:37.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.115+0000 7f5692277640 1 -- 172.21.15.27:0/4268921425 wait complete. 2023-08-11T15:51:37.138 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:51:37.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.378+0000 7ff7c359e640 1 Processor -- start 2023-08-11T15:51:37.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.378+0000 7ff7c359e640 1 -- start start 2023-08-11T15:51:37.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.379+0000 7ff7c359e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff7c40d1b90 0x7ff7c40b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.380+0000 7ff7c359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c415c970 con 0x7ff7c415c590 2023-08-11T15:51:37.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.380+0000 7ff7c359e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c40d0250 con 0x7ff7c40cd0e0 2023-08-11T15:51:37.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.380+0000 7ff7c359e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c40d0ed0 con 0x7ff7c40d1b90 2023-08-11T15:51:37.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.380+0000 7ff7c259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff7c415c590 0x7ff7c40d1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48580/0 (socket says 172.21.15.27:48580) 2023-08-11T15:51:37.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.380+0000 7ff7c259c640 1 -- 172.21.15.27:0/2432493894 learned_addr learned my addr 172.21.15.27:0/2432493894 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:37.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.380+0000 7ff7bbfff640 1 --2- 172.21.15.27:0/2432493894 >> v2:172.21.15.119:3300/0 conn(0x7ff7c40d1b90 0x7ff7c40b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.380+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662930410 0 0) 0x7ff7c415c970 con 0x7ff7c415c590 2023-08-11T15:51:37.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.380+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b0003520 con 0x7ff7c415c590 2023-08-11T15:51:37.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2348545448 0 0) 0x7ff7b0003520 con 0x7ff7c415c590 2023-08-11T15:51:37.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 >> v2:172.21.15.119:3300/0 conn(0x7ff7c40d1b90 msgr2=0x7ff7c40b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:37.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 --2- 172.21.15.27:0/2432493894 >> v2:172.21.15.119:3300/0 conn(0x7ff7c40d1b90 0x7ff7c40b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 >> v1:172.21.15.119:6789/0 conn(0x7ff7c40cd0e0 legacy=0x7ff7c4160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:37.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c40d0530 con 0x7ff7c415c590 2023-08-11T15:51:37.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7ff7b4001670 con 0x7ff7c415c590 2023-08-11T15:51:37.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 >> v1:172.21.15.27:6789/0 conn(0x7ff7c415c590 legacy=0x7ff7c40d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7b0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7b00047c0 0x7ff7b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7b00051f0 0x7ff7b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40d0ed0 con 0x7ff7b0003970 2023-08-11T15:51:37.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b000c150 con 0x7ff7b00047c0 2023-08-11T15:51:37.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b000c310 con 0x7ff7b00051f0 2023-08-11T15:51:37.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.381+0000 7ff7c259c640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7b0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.382+0000 7ff7bbfff640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7b00047c0 0x7ff7b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.382+0000 7ff7c2d9d640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7b00051f0 0x7ff7b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.382+0000 7ff7c259c640 1 -- 172.21.15.27:0/2432493894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7b00051f0 msgr2=0x7ff7b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.382+0000 7ff7c259c640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7b00051f0 0x7ff7b000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.382+0000 7ff7c259c640 1 -- 172.21.15.27:0/2432493894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7b00047c0 msgr2=0x7ff7b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.382+0000 7ff7c259c640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7b00047c0 0x7ff7b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.382+0000 7ff7c259c640 1 -- 172.21.15.27:0/2432493894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b0003520 con 0x7ff7b0003970 2023-08-11T15:51:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.382+0000 7ff7c2d9d640 1 -- 172.21.15.27:0/2432493894 reap_dead start 2023-08-11T15:51:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.383+0000 7ff7c259c640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7b0003d70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400c210 tx=0x7ff7b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d75027bc42b33b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.383+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b402cae0 con 0x7ff7b0003970 2023-08-11T15:51:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.384+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b402cc60 con 0x7ff7b0003970 2023-08-11T15:51:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.384+0000 7ff7c1d9b640 1 -- 172.21.15.27:0/2432493894 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4005b10 con 0x7ff7b0003970 2023-08-11T15:51:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.384+0000 7ff7c359e640 1 -- 172.21.15.27:0/2432493894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 msgr2=0x7ff7b0003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.384+0000 7ff7c359e640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7b0003d70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400c210 tx=0x7ff7b00057f0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.384+0000 7ff7c359e640 1 -- 172.21.15.27:0/2432493894 shutdown_connections 2023-08-11T15:51:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.384+0000 7ff7c359e640 1 --2- 172.21.15.27:0/2432493894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7b0003d70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.384+0000 7ff7c359e640 1 -- 172.21.15.27:0/2432493894 >> 172.21.15.27:0/2432493894 conn(0x7ff7c406a250 msgr2=0x7ff7c406a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.384+0000 7ff7c359e640 1 -- 172.21.15.27:0/2432493894 shutdown_connections 2023-08-11T15:51:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.384+0000 7ff7c359e640 1 -- 172.21.15.27:0/2432493894 wait complete. 2023-08-11T15:51:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.386+0000 7ff7c359e640 1 Processor -- start 2023-08-11T15:51:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.386+0000 7ff7c359e640 1 -- start start 2023-08-11T15:51:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.387+0000 7ff7c359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7c41f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.387+0000 7ff7c359e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7c414a070 0x7ff7c415db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.387+0000 7ff7c359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7c415e0a0 0x7ff7c40cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.387+0000 7ff7c359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c41f1d90 con 0x7ff7b0003970 2023-08-11T15:51:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.387+0000 7ff7c359e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c415e5e0 con 0x7ff7c415e0a0 2023-08-11T15:51:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.387+0000 7ff7c359e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40b3430 con 0x7ff7c414a070 2023-08-11T15:51:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.388+0000 7ff7c259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7c41f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.388+0000 7ff7c259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7c41f1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46684/0 (socket says 172.21.15.27:46684) 2023-08-11T15:51:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.388+0000 7ff7c259c640 1 -- 172.21.15.27:0/433066844 learned_addr learned my addr 172.21.15.27:0/433066844 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.388+0000 7ff7bbfff640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7c414a070 0x7ff7c415db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.388+0000 7ff7c2d9d640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7c415e0a0 0x7ff7c40cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.388+0000 7ff7c259c640 1 -- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7c414a070 msgr2=0x7ff7c415db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.388+0000 7ff7c259c640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7c414a070 0x7ff7c415db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.388+0000 7ff7c259c640 1 -- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7c415e0a0 msgr2=0x7ff7c40cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.388+0000 7ff7c259c640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7c415e0a0 0x7ff7c40cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.388+0000 7ff7c259c640 1 -- 172.21.15.27:0/433066844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c40b3710 con 0x7ff7b0003970 2023-08-11T15:51:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.389+0000 7ff7c259c640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7c41f1850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff7b402c480 tx=0x7ff7b402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.389+0000 7ff7baffd640 1 -- 172.21.15.27:0/433066844 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b403c040 con 0x7ff7b0003970 2023-08-11T15:51:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.389+0000 7ff7baffd640 1 -- 172.21.15.27:0/433066844 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b4004bf0 con 0x7ff7b0003970 2023-08-11T15:51:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.389+0000 7ff7baffd640 1 -- 172.21.15.27:0/433066844 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4007070 con 0x7ff7b0003970 2023-08-11T15:51:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.389+0000 7ff7c359e640 1 -- 172.21.15.27:0/433066844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c40b39f0 con 0x7ff7b0003970 2023-08-11T15:51:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.389+0000 7ff7c359e640 1 -- 172.21.15.27:0/433066844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c414cdb0 con 0x7ff7b0003970 2023-08-11T15:51:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.390+0000 7ff7c359e640 1 -- 172.21.15.27:0/433066844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c404e420 con 0x7ff7b0003970 2023-08-11T15:51:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.391+0000 7ff7baffd640 1 -- 172.21.15.27:0/433066844 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff7b4032070 con 0x7ff7b0003970 2023-08-11T15:51:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.392+0000 7ff7baffd640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff7a406a050 0x7ff7a406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.392+0000 7ff7baffd640 1 -- 172.21.15.27:0/433066844 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff7b40a9920 con 0x7ff7b0003970 2023-08-11T15:51:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.393+0000 7ff7bbfff640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff7a406a050 0x7ff7a406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.394+0000 7ff7bbfff640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff7a406a050 0x7ff7a406c510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff7c41f53a0 tx=0x7ff7ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:37.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.397+0000 7ff7baffd640 1 -- 172.21.15.27:0/433066844 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ff7b4074c80 con 0x7ff7b0003970 2023-08-11T15:51:37.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.628+0000 7ff7c359e640 1 -- 172.21.15.27:0/433066844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7c415bd20 con 0x7ff7b0003970 2023-08-11T15:51:37.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.629+0000 7ff7baffd640 1 -- 172.21.15.27:0/433066844 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7ff7b40782a0 con 0x7ff7b0003970 2023-08-11T15:51:37.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:37.631 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:51:01.220970+0000","modified":"2023-08-11T15:51:25.832166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4315":{"gid":4315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:51:37.632 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2023-08-11T15:51:37.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.632+0000 7ff7c359e640 1 -- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff7a406a050 msgr2=0x7ff7a406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.632+0000 7ff7c359e640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff7a406a050 0x7ff7a406c510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff7c41f53a0 tx=0x7ff7ac008040 comp rx=0 tx=0).stop 2023-08-11T15:51:37.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.632+0000 7ff7c359e640 1 -- 172.21.15.27:0/433066844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 msgr2=0x7ff7c41f1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.632+0000 7ff7c359e640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7c41f1850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff7b402c480 tx=0x7ff7b402e070 comp rx=0 tx=0).stop 2023-08-11T15:51:37.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.632+0000 7ff7c359e640 1 -- 172.21.15.27:0/433066844 shutdown_connections 2023-08-11T15:51:37.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.632+0000 7ff7c359e640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff7a406a050 0x7ff7a406c510 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.632+0000 7ff7c359e640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7c414a070 0x7ff7c415db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.632+0000 7ff7c359e640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7c415e0a0 0x7ff7c40cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.632+0000 7ff7c359e640 1 --2- 172.21.15.27:0/433066844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7b0003970 0x7ff7c41f1850 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.632+0000 7ff7c359e640 1 -- 172.21.15.27:0/433066844 >> 172.21.15.27:0/433066844 conn(0x7ff7c406a250 msgr2=0x7ff7c40d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:37.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.633+0000 7ff7c359e640 1 -- 172.21.15.27:0/433066844 shutdown_connections 2023-08-11T15:51:37.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.633+0000 7ff7c359e640 1 -- 172.21.15.27:0/433066844 wait complete. 2023-08-11T15:51:37.655 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:37.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.893+0000 7f1d74742640 1 Processor -- start 2023-08-11T15:51:37.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.893+0000 7f1d74742640 1 -- start start 2023-08-11T15:51:37.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.895+0000 7f1d74742640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1d6c15a140 0x7f1d6c06b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.895+0000 7f1d74742640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 0x7f1d6c06fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.895+0000 7f1d74742640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6c0c8a30 con 0x7f1d6c0cb430 2023-08-11T15:51:37.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.895+0000 7f1d74742640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6c06ffa0 con 0x7f1d6c15a140 2023-08-11T15:51:37.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.895+0000 7f1d74742640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c0b1580 con 0x7f1d6c06f5d0 2023-08-11T15:51:37.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.896+0000 7f1d724de640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1d6c15a140 0x7f1d6c06b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.896+0000 7f1d724de640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1d6c15a140 0x7f1d6c06b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41080/0 (socket says 172.21.15.27:41080) 2023-08-11T15:51:37.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.896+0000 7f1d724de640 1 -- 172.21.15.27:0/3760942669 learned_addr learned my addr 172.21.15.27:0/3760942669 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:37.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.896+0000 7f1d72cdf640 1 --2- 172.21.15.27:0/3760942669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 0x7f1d6c06fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.897+0000 7f1d72cdf640 1 -- 172.21.15.27:0/3760942669 >> v2:172.21.15.119:3300/0 conn(0x7f1d6c15a140 msgr2=0x7f1d6c06b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:37.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.897+0000 7f1d72cdf640 1 --2- 172.21.15.27:0/3760942669 >> v2:172.21.15.119:3300/0 conn(0x7f1d6c15a140 0x7f1d6c06b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.897+0000 7f1d72cdf640 1 -- 172.21.15.27:0/3760942669 >> v1:172.21.15.119:6789/0 conn(0x7f1d6c0cb430 legacy=0x7f1d6c0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.897+0000 7f1d72cdf640 1 -- 172.21.15.27:0/3760942669 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c06c0c0 con 0x7f1d6c06f5d0 2023-08-11T15:51:37.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.897+0000 7f1d724de640 1 --2- 172.21.15.27:0/3760942669 >> v2:172.21.15.119:3300/0 conn(0x7f1d6c15a140 0x7f1d6c06b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:37.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.898+0000 7f1d72cdf640 1 --2- 172.21.15.27:0/3760942669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 0x7f1d6c06fa60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009d90 tx=0x7f1d6402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9521229a104b3307 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:37.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.899+0000 7f1d714dc640 1 -- 172.21.15.27:0/3760942669 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6402cc50 con 0x7f1d6c06f5d0 2023-08-11T15:51:37.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.899+0000 7f1d714dc640 1 -- 172.21.15.27:0/3760942669 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6402cdb0 con 0x7f1d6c06f5d0 2023-08-11T15:51:37.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.899+0000 7f1d714dc640 1 -- 172.21.15.27:0/3760942669 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64034660 con 0x7f1d6c06f5d0 2023-08-11T15:51:37.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.899+0000 7f1d74742640 1 -- 172.21.15.27:0/3760942669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 msgr2=0x7f1d6c06fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.899+0000 7f1d74742640 1 --2- 172.21.15.27:0/3760942669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 0x7f1d6c06fa60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009d90 tx=0x7f1d6402bea0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.900+0000 7f1d74742640 1 -- 172.21.15.27:0/3760942669 shutdown_connections 2023-08-11T15:51:37.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.900+0000 7f1d74742640 1 --2- 172.21.15.27:0/3760942669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 0x7f1d6c06fa60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.900+0000 7f1d74742640 1 --2- 172.21.15.27:0/3760942669 >> v2:172.21.15.119:3300/0 conn(0x7f1d6c15a140 0x7f1d6c06b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.900+0000 7f1d74742640 1 -- 172.21.15.27:0/3760942669 >> 172.21.15.27:0/3760942669 conn(0x7f1d6c0afb70 msgr2=0x7f1d6c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:37.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.900+0000 7f1d74742640 1 -- 172.21.15.27:0/3760942669 shutdown_connections 2023-08-11T15:51:37.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.900+0000 7f1d74742640 1 -- 172.21.15.27:0/3760942669 wait complete. 2023-08-11T15:51:37.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.902+0000 7f1d74742640 1 Processor -- start 2023-08-11T15:51:37.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.902+0000 7f1d74742640 1 -- start start 2023-08-11T15:51:37.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d74742640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 0x7f1d6c152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d74742640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6c0cb430 0x7f1d6c1530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d74742640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d6c15a140 0x7f1d6c0c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d74742640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1f9290 con 0x7f1d6c0cb430 2023-08-11T15:51:37.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d74742640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1f9570 con 0x7f1d6c15a140 2023-08-11T15:51:37.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d74742640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1f9850 con 0x7f1d6c06f5d0 2023-08-11T15:51:37.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d71cdd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6c0cb430 0x7f1d6c1530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d71cdd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6c0cb430 0x7f1d6c1530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46686/0 (socket says 172.21.15.27:46686) 2023-08-11T15:51:37.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d71cdd640 1 -- 172.21.15.27:0/2457519989 learned_addr learned my addr 172.21.15.27:0/2457519989 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:37.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d72cdf640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d6c15a140 0x7f1d6c0c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.904+0000 7f1d724de640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 0x7f1d6c152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d71cdd640 1 -- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 msgr2=0x7f1d6c152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d71cdd640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 0x7f1d6c152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d71cdd640 1 -- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d6c15a140 msgr2=0x7f1d6c0c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:37.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d71cdd640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d6c15a140 0x7f1d6c0c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:37.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d71cdd640 1 -- 172.21.15.27:0/2457519989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68009460 con 0x7f1d6c0cb430 2023-08-11T15:51:37.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d72cdf640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d6c15a140 0x7f1d6c0c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:37.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d71cdd640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6c0cb430 0x7f1d6c1530e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00c450 tx=0x7f1d5c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:37.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d62ffd640 1 -- 172.21.15.27:0/2457519989 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0052d0 con 0x7f1d6c0cb430 2023-08-11T15:51:37.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d62ffd640 1 -- 172.21.15.27:0/2457519989 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c005430 con 0x7f1d6c0cb430 2023-08-11T15:51:37.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d62ffd640 1 -- 172.21.15.27:0/2457519989 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c012870 con 0x7f1d6c0cb430 2023-08-11T15:51:37.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.905+0000 7f1d74742640 1 -- 172.21.15.27:0/2457519989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d64009a70 con 0x7f1d6c0cb430 2023-08-11T15:51:37.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.906+0000 7f1d74742640 1 -- 172.21.15.27:0/2457519989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c1f9e10 con 0x7f1d6c0cb430 2023-08-11T15:51:37.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.907+0000 7f1d74742640 1 -- 172.21.15.27:0/2457519989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6c073990 con 0x7f1d6c0cb430 2023-08-11T15:51:37.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.908+0000 7f1d62ffd640 1 -- 172.21.15.27:0/2457519989 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1d5c0129d0 con 0x7f1d6c0cb430 2023-08-11T15:51:37.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.909+0000 7f1d62ffd640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d5006a050 0x7f1d5006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:37.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.909+0000 7f1d62ffd640 1 -- 172.21.15.27:0/2457519989 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1d5c0883e0 con 0x7f1d6c0cb430 2023-08-11T15:51:37.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.909+0000 7f1d724de640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d5006a050 0x7f1d5006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:37.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.911+0000 7f1d724de640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d5006a050 0x7f1d5006c510 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0af9f0 tx=0x7f1d68008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:37.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:37.915+0000 7f1d62ffd640 1 -- 172.21.15.27:0/2457519989 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f1d5c0537d0 con 0x7f1d6c0cb430 2023-08-11T15:51:38.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.109+0000 7f1d74742640 1 -- 172.21.15.27:0/2457519989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d6c06fd30 con 0x7f1d6c0cb430 2023-08-11T15:51:38.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.112+0000 7f1d62ffd640 1 -- 172.21.15.27:0/2457519989 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20120 (secure 0 0 0) 0x7f1d5c056df0 con 0x7f1d6c0cb430 2023-08-11T15:51:38.114 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:38.114 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:06.322931+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:38.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.114+0000 7f1d74742640 1 -- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d5006a050 msgr2=0x7f1d5006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:38.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.114+0000 7f1d74742640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d5006a050 0x7f1d5006c510 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0af9f0 tx=0x7f1d68008040 comp rx=0 tx=0).stop 2023-08-11T15:51:38.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.114+0000 7f1d74742640 1 -- 172.21.15.27:0/2457519989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6c0cb430 msgr2=0x7f1d6c1530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:38.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.114+0000 7f1d74742640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6c0cb430 0x7f1d6c1530e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00c450 tx=0x7f1d5c00c910 comp rx=0 tx=0).stop 2023-08-11T15:51:38.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.114+0000 7f1d74742640 1 -- 172.21.15.27:0/2457519989 shutdown_connections 2023-08-11T15:51:38.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.114+0000 7f1d74742640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d5006a050 0x7f1d5006c510 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.114+0000 7f1d74742640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d6c15a140 0x7f1d6c0c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.114+0000 7f1d74742640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6c0cb430 0x7f1d6c1530e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.114+0000 7f1d74742640 1 --2- 172.21.15.27:0/2457519989 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d6c06f5d0 0x7f1d6c152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.114+0000 7f1d74742640 1 -- 172.21.15.27:0/2457519989 >> 172.21.15.27:0/2457519989 conn(0x7f1d6c0afb70 msgr2=0x7f1d6c0cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:38.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.115+0000 7f1d74742640 1 -- 172.21.15.27:0/2457519989 shutdown_connections 2023-08-11T15:51:38.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.115+0000 7f1d74742640 1 -- 172.21.15.27:0/2457519989 wait complete. 2023-08-11T15:51:38.136 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:51:38.137 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:51:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.382+0000 7fb07ab24640 1 Processor -- start 2023-08-11T15:51:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.382+0000 7fb07ab24640 1 -- start start 2023-08-11T15:51:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.384+0000 7fb07ab24640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb074159cb0 0x7fb0740cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.384+0000 7fb07ab24640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb0740cdf80 0x7fb07406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.384+0000 7fb07ab24640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0740ce430 con 0x7fb0741598d0 2023-08-11T15:51:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.384+0000 7fb07ab24640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0740cb8f0 con 0x7fb074159cb0 2023-08-11T15:51:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.384+0000 7fb07ab24640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0740cbbc0 con 0x7fb0740cdf80 2023-08-11T15:51:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.384+0000 7fb073fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb074159cb0 0x7fb0740cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.385+0000 7fb073fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb074159cb0 0x7fb0740cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46688/0 (socket says 172.21.15.27:46688) 2023-08-11T15:51:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.385+0000 7fb073fff640 1 -- 172.21.15.27:0/1460722082 learned_addr learned my addr 172.21.15.27:0/1460722082 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.385+0000 7fb0790c1640 1 --2- 172.21.15.27:0/1460722082 >> v2:172.21.15.119:3300/0 conn(0x7fb0740cdf80 0x7fb07406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.385+0000 7fb073fff640 1 -- 172.21.15.27:0/1460722082 >> v2:172.21.15.119:3300/0 conn(0x7fb0740cdf80 msgr2=0x7fb07406c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.385+0000 7fb073fff640 1 --2- 172.21.15.27:0/1460722082 >> v2:172.21.15.119:3300/0 conn(0x7fb0740cdf80 0x7fb07406c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.385+0000 7fb073fff640 1 -- 172.21.15.27:0/1460722082 >> v1:172.21.15.119:6789/0 conn(0x7fb0741598d0 legacy=0x7fb0740cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.385+0000 7fb073fff640 1 -- 172.21.15.27:0/1460722082 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0740cbea0 con 0x7fb074159cb0 2023-08-11T15:51:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.385+0000 7fb0790c1640 1 --2- 172.21.15.27:0/1460722082 >> v2:172.21.15.119:3300/0 conn(0x7fb0740cdf80 0x7fb07406c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:38.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.387+0000 7fb073fff640 1 --2- 172.21.15.27:0/1460722082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb074159cb0 0x7fb0740cda40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb060009870 tx=0x7fb06002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ff858e87363896 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:38.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.387+0000 7fb0737fe640 1 -- 172.21.15.27:0/1460722082 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060030850 con 0x7fb074159cb0 2023-08-11T15:51:38.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.387+0000 7fb0737fe640 1 -- 172.21.15.27:0/1460722082 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0600309b0 con 0x7fb074159cb0 2023-08-11T15:51:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.387+0000 7fb0737fe640 1 -- 172.21.15.27:0/1460722082 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060030d40 con 0x7fb074159cb0 2023-08-11T15:51:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.387+0000 7fb07ab24640 1 -- 172.21.15.27:0/1460722082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb074159cb0 msgr2=0x7fb0740cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.387+0000 7fb07ab24640 1 --2- 172.21.15.27:0/1460722082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb074159cb0 0x7fb0740cda40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb060009870 tx=0x7fb06002b990 comp rx=0 tx=0).stop 2023-08-11T15:51:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.388+0000 7fb07ab24640 1 -- 172.21.15.27:0/1460722082 shutdown_connections 2023-08-11T15:51:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.388+0000 7fb07ab24640 1 --2- 172.21.15.27:0/1460722082 >> v2:172.21.15.119:3300/0 conn(0x7fb0740cdf80 0x7fb07406c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.388+0000 7fb07ab24640 1 --2- 172.21.15.27:0/1460722082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb074159cb0 0x7fb0740cda40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.388+0000 7fb07ab24640 1 -- 172.21.15.27:0/1460722082 >> 172.21.15.27:0/1460722082 conn(0x7fb0740b0b00 msgr2=0x7fb0740ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.388+0000 7fb07ab24640 1 -- 172.21.15.27:0/1460722082 shutdown_connections 2023-08-11T15:51:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.388+0000 7fb07ab24640 1 -- 172.21.15.27:0/1460722082 wait complete. 2023-08-11T15:51:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.390+0000 7fb07ab24640 1 Processor -- start 2023-08-11T15:51:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.390+0000 7fb07ab24640 1 -- start start 2023-08-11T15:51:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb07ab24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0740cdf80 0x7fb0741436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb07ab24640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741598d0 0x7fb074143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb07ab24640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb074145c60 0x7fb0741460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb07ab24640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07415bbf0 con 0x7fb0741598d0 2023-08-11T15:51:38.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb07ab24640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07415bed0 con 0x7fb0740cdf80 2023-08-11T15:51:38.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb07ab24640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb074145180 con 0x7fb074145c60 2023-08-11T15:51:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb073fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741598d0 0x7fb074143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb073fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741598d0 0x7fb074143cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46698/0 (socket says 172.21.15.27:46698) 2023-08-11T15:51:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb073fff640 1 -- 172.21.15.27:0/2711377544 learned_addr learned my addr 172.21.15.27:0/2711377544 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb0788c0640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0740cdf80 0x7fb0741436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb0790c1640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb074145c60 0x7fb0741460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb073fff640 1 -- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb074145c60 msgr2=0x7fb0741460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.391+0000 7fb073fff640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb074145c60 0x7fb0741460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.392+0000 7fb073fff640 1 -- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0740cdf80 msgr2=0x7fb0741436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.392+0000 7fb073fff640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0740cdf80 0x7fb0741436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.392+0000 7fb073fff640 1 -- 172.21.15.27:0/2711377544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0600094d0 con 0x7fb0741598d0 2023-08-11T15:51:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.392+0000 7fb073fff640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741598d0 0x7fb074143cd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb060000c00 tx=0x7fb060004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.392+0000 7fb0717fa640 1 -- 172.21.15.27:0/2711377544 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060030820 con 0x7fb0741598d0 2023-08-11T15:51:38.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.392+0000 7fb0717fa640 1 -- 172.21.15.27:0/2711377544 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb060039d90 con 0x7fb0741598d0 2023-08-11T15:51:38.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.392+0000 7fb0717fa640 1 -- 172.21.15.27:0/2711377544 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060043670 con 0x7fb0741598d0 2023-08-11T15:51:38.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.392+0000 7fb07ab24640 1 -- 172.21.15.27:0/2711377544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb074145460 con 0x7fb0741598d0 2023-08-11T15:51:38.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.392+0000 7fb07ab24640 1 -- 172.21.15.27:0/2711377544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07415b810 con 0x7fb0741598d0 2023-08-11T15:51:38.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.394+0000 7fb07ab24640 1 -- 172.21.15.27:0/2711377544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb074145980 con 0x7fb0741598d0 2023-08-11T15:51:38.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.394+0000 7fb0717fa640 1 -- 172.21.15.27:0/2711377544 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb06002e070 con 0x7fb0741598d0 2023-08-11T15:51:38.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.395+0000 7fb0717fa640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb05806a080 0x7fb05806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.396+0000 7fb0717fa640 1 -- 172.21.15.27:0/2711377544 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb0600aaa10 con 0x7fb0741598d0 2023-08-11T15:51:38.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.396+0000 7fb0788c0640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb05806a080 0x7fb05806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.397+0000 7fb0788c0640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb05806a080 0x7fb05806c540 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb068000f60 tx=0x7fb068009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:38.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.400+0000 7fb0717fa640 1 -- 172.21.15.27:0/2711377544 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fb074145980 con 0x7fb0741598d0 2023-08-11T15:51:38.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.594+0000 7fb07ab24640 1 -- 172.21.15.27:0/2711377544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb074145980 con 0x7fb0741598d0 2023-08-11T15:51:38.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.595+0000 7fb0717fa640 1 -- 172.21.15.27:0/2711377544 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb074145980 con 0x7fb0741598d0 2023-08-11T15:51:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:51:38.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 -- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb05806a080 msgr2=0x7fb05806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:38.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb05806a080 0x7fb05806c540 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb068000f60 tx=0x7fb068009040 comp rx=0 tx=0).stop 2023-08-11T15:51:38.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 -- 172.21.15.27:0/2711377544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741598d0 msgr2=0x7fb074143cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:38.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741598d0 0x7fb074143cd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb060000c00 tx=0x7fb060004730 comp rx=0 tx=0).stop 2023-08-11T15:51:38.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 -- 172.21.15.27:0/2711377544 shutdown_connections 2023-08-11T15:51:38.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb05806a080 0x7fb05806c540 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb074145c60 0x7fb0741460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741598d0 0x7fb074143cd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 --2- 172.21.15.27:0/2711377544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0740cdf80 0x7fb0741436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 -- 172.21.15.27:0/2711377544 >> 172.21.15.27:0/2711377544 conn(0x7fb0740b0b00 msgr2=0x7fb07414af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 -- 172.21.15.27:0/2711377544 shutdown_connections 2023-08-11T15:51:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.598+0000 7fb07ab24640 1 -- 172.21.15.27:0/2711377544 wait complete. 2023-08-11T15:51:38.619 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:51:38.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.855+0000 7f565f59e640 1 Processor -- start 2023-08-11T15:51:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.855+0000 7f565f59e640 1 -- start start 2023-08-11T15:51:38.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.857+0000 7f565f59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f566015da60 0x7f56600aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.857+0000 7f565f59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56600ad3d0 0x7f56600b0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.857+0000 7f565f59e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56600ce980 con 0x7f56600d0cc0 2023-08-11T15:51:38.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.857+0000 7f565f59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56600cdfe0 con 0x7f566015da60 2023-08-11T15:51:38.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.857+0000 7f565f59e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566015de40 con 0x7f56600ad3d0 2023-08-11T15:51:38.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.857+0000 7f565e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f566015da60 0x7f56600aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.857+0000 7f565e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f566015da60 0x7f56600aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46704/0 (socket says 172.21.15.27:46704) 2023-08-11T15:51:38.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.857+0000 7f565e59c640 1 -- 172.21.15.27:0/91472194 learned_addr learned my addr 172.21.15.27:0/91472194 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:38.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.858+0000 7f565ed9d640 1 --2- 172.21.15.27:0/91472194 >> v2:172.21.15.119:3300/0 conn(0x7f56600ad3d0 0x7f56600b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.858+0000 7f565e59c640 1 -- 172.21.15.27:0/91472194 >> v2:172.21.15.119:3300/0 conn(0x7f56600ad3d0 msgr2=0x7f56600b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:38.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.858+0000 7f565e59c640 1 --2- 172.21.15.27:0/91472194 >> v2:172.21.15.119:3300/0 conn(0x7f56600ad3d0 0x7f56600b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.858+0000 7f565e59c640 1 -- 172.21.15.27:0/91472194 >> v1:172.21.15.119:6789/0 conn(0x7f56600d0cc0 legacy=0x7f56600accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:38.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.858+0000 7f565e59c640 1 -- 172.21.15.27:0/91472194 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56600ae880 con 0x7f566015da60 2023-08-11T15:51:38.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.859+0000 7f565e59c640 1 --2- 172.21.15.27:0/91472194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 0x7f56600aee70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f566006d140 tx=0x7f564802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94eb20c75dbff477 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:38.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.860+0000 7f565d59a640 1 -- 172.21.15.27:0/91472194 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648031870 con 0x7f566015da60 2023-08-11T15:51:38.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.860+0000 7f565d59a640 1 -- 172.21.15.27:0/91472194 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56480319d0 con 0x7f566015da60 2023-08-11T15:51:38.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.860+0000 7f565d59a640 1 -- 172.21.15.27:0/91472194 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648031d60 con 0x7f566015da60 2023-08-11T15:51:38.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.861+0000 7f565f59e640 1 -- 172.21.15.27:0/91472194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 msgr2=0x7f56600aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.861+0000 7f565f59e640 1 --2- 172.21.15.27:0/91472194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 0x7f56600aee70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f566006d140 tx=0x7f564802b910 comp rx=0 tx=0).stop 2023-08-11T15:51:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.861+0000 7f565f59e640 1 -- 172.21.15.27:0/91472194 shutdown_connections 2023-08-11T15:51:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.861+0000 7f565f59e640 1 --2- 172.21.15.27:0/91472194 >> v2:172.21.15.119:3300/0 conn(0x7f56600ad3d0 0x7f56600b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.861+0000 7f565f59e640 1 --2- 172.21.15.27:0/91472194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 0x7f56600aee70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.861+0000 7f565f59e640 1 -- 172.21.15.27:0/91472194 >> 172.21.15.27:0/91472194 conn(0x7f56600993d0 msgr2=0x7f56600d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.861+0000 7f565f59e640 1 -- 172.21.15.27:0/91472194 shutdown_connections 2023-08-11T15:51:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.861+0000 7f565f59e640 1 -- 172.21.15.27:0/91472194 wait complete. 2023-08-11T15:51:38.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.863+0000 7f565f59e640 1 Processor -- start 2023-08-11T15:51:38.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.864+0000 7f565f59e640 1 -- start start 2023-08-11T15:51:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.865+0000 7f565f59e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56600ad3d0 0x7f56600b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.865+0000 7f565f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600d0cc0 0x7f56600b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.865+0000 7f565f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 0x7f56600ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.865+0000 7f565f59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56600c7ad0 con 0x7f566015da60 2023-08-11T15:51:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.865+0000 7f565f59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56600c71a0 con 0x7f56600d0cc0 2023-08-11T15:51:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.865+0000 7f565f59e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56600c7480 con 0x7f56600ad3d0 2023-08-11T15:51:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.865+0000 7f565ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 0x7f56600ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 0x7f56600ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46716/0 (socket says 172.21.15.27:46716) 2023-08-11T15:51:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565ed9d640 1 -- 172.21.15.27:0/1854676978 learned_addr learned my addr 172.21.15.27:0/1854676978 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565dd9b640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600d0cc0 0x7f56600b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565e59c640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56600ad3d0 0x7f56600b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565ed9d640 1 -- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56600ad3d0 msgr2=0x7f56600b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565ed9d640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56600ad3d0 0x7f56600b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565ed9d640 1 -- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600d0cc0 msgr2=0x7f56600b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565ed9d640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600d0cc0 0x7f56600b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565ed9d640 1 -- 172.21.15.27:0/1854676978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5648009470 con 0x7f566015da60 2023-08-11T15:51:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565dd9b640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600d0cc0 0x7f56600b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.866+0000 7f565ed9d640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 0x7f56600ca9d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f565800d800 tx=0x7f565800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:38.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.867+0000 7f5656ffd640 1 -- 172.21.15.27:0/1854676978 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800b6c0 con 0x7f566015da60 2023-08-11T15:51:38.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.867+0000 7f5656ffd640 1 -- 172.21.15.27:0/1854676978 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565800b840 con 0x7f566015da60 2023-08-11T15:51:38.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.867+0000 7f5656ffd640 1 -- 172.21.15.27:0/1854676978 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800bbf0 con 0x7f566015da60 2023-08-11T15:51:38.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.867+0000 7f565f59e640 1 -- 172.21.15.27:0/1854676978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56600c7760 con 0x7f566015da60 2023-08-11T15:51:38.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.867+0000 7f565f59e640 1 -- 172.21.15.27:0/1854676978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56600ccf80 con 0x7f566015da60 2023-08-11T15:51:38.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.868+0000 7f565f59e640 1 -- 172.21.15.27:0/1854676978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56600cd7e0 con 0x7f566015da60 2023-08-11T15:51:38.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.869+0000 7f5656ffd640 1 -- 172.21.15.27:0/1854676978 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f565800bd50 con 0x7f566015da60 2023-08-11T15:51:38.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.870+0000 7f5656ffd640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f564006a050 0x7f564006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:38.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.870+0000 7f5656ffd640 1 -- 172.21.15.27:0/1854676978 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5658088e70 con 0x7f566015da60 2023-08-11T15:51:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.871+0000 7f565e59c640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f564006a050 0x7f564006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.872+0000 7f565e59c640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f564006a050 0x7f564006c510 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5648036000 tx=0x7f564803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:38.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:38.875+0000 7f5656ffd640 1 -- 172.21.15.27:0/1854676978 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f56600cd7e0 con 0x7f566015da60 2023-08-11T15:51:39.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.069+0000 7f565f59e640 1 -- 172.21.15.27:0/1854676978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f56600cedf0 con 0x7f566015da60 2023-08-11T15:51:39.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.070+0000 7f5656ffd640 1 -- 172.21.15.27:0/1854676978 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f56600cedf0 con 0x7f566015da60 2023-08-11T15:51:39.072 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:51:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.072+0000 7f565f59e640 1 -- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f564006a050 msgr2=0x7f564006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f564006a050 0x7f564006c510 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5648036000 tx=0x7f564803c040 comp rx=0 tx=0).stop 2023-08-11T15:51:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 -- 172.21.15.27:0/1854676978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 msgr2=0x7f56600ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 0x7f56600ca9d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f565800d800 tx=0x7f565800dcc0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 -- 172.21.15.27:0/1854676978 shutdown_connections 2023-08-11T15:51:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f564006a050 0x7f564006c510 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566015da60 0x7f56600ca9d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56600d0cc0 0x7f56600b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 --2- 172.21.15.27:0/1854676978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56600ad3d0 0x7f56600b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 -- 172.21.15.27:0/1854676978 >> 172.21.15.27:0/1854676978 conn(0x7f56600993d0 msgr2=0x7f566015a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 -- 172.21.15.27:0/1854676978 shutdown_connections 2023-08-11T15:51:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.073+0000 7f565f59e640 1 -- 172.21.15.27:0/1854676978 wait complete. 2023-08-11T15:51:39.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:51:39.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.326+0000 7f7a9b59e640 1 Processor -- start 2023-08-11T15:51:39.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.327+0000 7f7a9b59e640 1 -- start start 2023-08-11T15:51:39.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.328+0000 7f7a9b59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7a9c15c3f0 0x7f7a9c0c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.328+0000 7f7a9b59e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 0x7f7a9c0afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.328+0000 7f7a9b59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9c1600f0 con 0x7f7a9c0cdb10 2023-08-11T15:51:39.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.328+0000 7f7a9b59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9c0c8c00 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.328+0000 7f7a9b59e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0ca810 con 0x7f7a9c0cd6d0 2023-08-11T15:51:39.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.329+0000 7f7a9a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7a9c15c3f0 0x7f7a9c0c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.329+0000 7f7a9a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7a9c15c3f0 0x7f7a9c0c86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50164/0 (socket says 172.21.15.27:50164) 2023-08-11T15:51:39.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.329+0000 7f7a9a59c640 1 -- 172.21.15.27:0/2294540275 learned_addr learned my addr 172.21.15.27:0/2294540275 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:39.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.329+0000 7f7a9ad9d640 1 --1- 172.21.15.27:0/2294540275 >> v1:172.21.15.27:6789/0 conn(0x7f7a9c0cdb10 0x7f7a9c06ba90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48472/0 (socket says 172.21.15.27:48472) 2023-08-11T15:51:39.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.329+0000 7f7a99d9b640 1 --2- 172.21.15.27:0/2294540275 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 0x7f7a9c0afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.329+0000 7f7a9a59c640 1 -- 172.21.15.27:0/2294540275 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 msgr2=0x7f7a9c0afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:39.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.329+0000 7f7a9a59c640 1 --2- 172.21.15.27:0/2294540275 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 0x7f7a9c0afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.329+0000 7f7a9a59c640 1 -- 172.21.15.27:0/2294540275 >> v1:172.21.15.27:6789/0 conn(0x7f7a9c0cdb10 legacy=0x7f7a9c06ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.329+0000 7f7a9a59c640 1 -- 172.21.15.27:0/2294540275 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c1603d0 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.329+0000 7f7a99d9b640 1 --2- 172.21.15.27:0/2294540275 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 0x7f7a9c0afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:39.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.330+0000 7f7a9a59c640 1 --2- 172.21.15.27:0/2294540275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 0x7f7a9c0c86c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0098a0 tx=0x7f7a8c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb33ae5b8f523bd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9959a640 1 -- 172.21.15.27:0/2294540275 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c02f830 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9959a640 1 -- 172.21.15.27:0/2294540275 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c02f990 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9959a640 1 -- 172.21.15.27:0/2294540275 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c02fd20 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9b59e640 1 -- 172.21.15.27:0/2294540275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 msgr2=0x7f7a9c0c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9b59e640 1 --2- 172.21.15.27:0/2294540275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 0x7f7a9c0c86c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0098a0 tx=0x7f7a8c02ba10 comp rx=0 tx=0).stop 2023-08-11T15:51:39.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9b59e640 1 -- 172.21.15.27:0/2294540275 shutdown_connections 2023-08-11T15:51:39.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9b59e640 1 --2- 172.21.15.27:0/2294540275 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 0x7f7a9c0afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9b59e640 1 --2- 172.21.15.27:0/2294540275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 0x7f7a9c0c86c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9b59e640 1 -- 172.21.15.27:0/2294540275 >> 172.21.15.27:0/2294540275 conn(0x7f7a9c0caab0 msgr2=0x7f7a9c0cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:39.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9b59e640 1 -- 172.21.15.27:0/2294540275 shutdown_connections 2023-08-11T15:51:39.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.331+0000 7f7a9b59e640 1 -- 172.21.15.27:0/2294540275 wait complete. 2023-08-11T15:51:39.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.333+0000 7f7a9b59e640 1 Processor -- start 2023-08-11T15:51:39.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.333+0000 7f7a9b59e640 1 -- start start 2023-08-11T15:51:39.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.334+0000 7f7a9b59e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 0x7f7a9c1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.334+0000 7f7a9b59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0cdb10 0x7f7a9c0c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 0x7f7a9c1f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0ae320 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9b59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0ada30 con 0x7f7a9c0cdb10 2023-08-11T15:51:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9b59e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0adcd0 con 0x7f7a9c0cd6d0 2023-08-11T15:51:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 0x7f7a9c1f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 0x7f7a9c1f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50180/0 (socket says 172.21.15.27:50180) 2023-08-11T15:51:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9ad9d640 1 -- 172.21.15.27:0/3106926337 learned_addr learned my addr 172.21.15.27:0/3106926337 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9a59c640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 0x7f7a9c1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a99d9b640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0cdb10 0x7f7a9c0c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9ad9d640 1 -- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 msgr2=0x7f7a9c1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9ad9d640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 0x7f7a9c1539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9ad9d640 1 -- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0cdb10 msgr2=0x7f7a9c0c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9ad9d640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0cdb10 0x7f7a9c0c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.335+0000 7f7a9ad9d640 1 -- 172.21.15.27:0/3106926337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c0094d0 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.336+0000 7f7a9ad9d640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 0x7f7a9c1f4570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400c7a0 tx=0x7f7a9400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.336+0000 7f7a8affd640 1 -- 172.21.15.27:0/3106926337 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9400a640 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.336+0000 7f7a8affd640 1 -- 172.21.15.27:0/3106926337 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9400a7c0 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.336+0000 7f7a8affd640 1 -- 172.21.15.27:0/3106926337 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9400ab70 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.336+0000 7f7a9b59e640 1 -- 172.21.15.27:0/3106926337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c0adfb0 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.336+0000 7f7a9b59e640 1 -- 172.21.15.27:0/3106926337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c0c7d90 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.337+0000 7f7a9b59e640 1 -- 172.21.15.27:0/3106926337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c0ae490 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.338+0000 7f7a8affd640 1 -- 172.21.15.27:0/3106926337 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7a9400acd0 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.339+0000 7f7a8affd640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a7c06a050 0x7f7a7c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.340+0000 7f7a8affd640 1 -- 172.21.15.27:0/3106926337 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7a94088160 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.340+0000 7f7a9a59c640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a7c06a050 0x7f7a7c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.341+0000 7f7a9a59c640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a7c06a050 0x7f7a7c06c510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c036000 tx=0x7f7a8c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.343+0000 7f7a8affd640 1 -- 172.21.15.27:0/3106926337 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f7a940534c0 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.534+0000 7f7a9b59e640 1 -- 172.21.15.27:0/3106926337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a9c04e420 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.535+0000 7f7a8affd640 1 -- 172.21.15.27:0/3106926337 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a94056ae0 con 0x7f7a9c15c3f0 2023-08-11T15:51:39.537 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:51:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.538+0000 7f7a9b59e640 1 -- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a7c06a050 msgr2=0x7f7a7c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.538+0000 7f7a9b59e640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a7c06a050 0x7f7a7c06c510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c036000 tx=0x7f7a8c008030 comp rx=0 tx=0).stop 2023-08-11T15:51:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.538+0000 7f7a9b59e640 1 -- 172.21.15.27:0/3106926337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 msgr2=0x7f7a9c1f4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.538+0000 7f7a9b59e640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 0x7f7a9c1f4570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400c7a0 tx=0x7f7a9400cc60 comp rx=0 tx=0).stop 2023-08-11T15:51:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.538+0000 7f7a9b59e640 1 -- 172.21.15.27:0/3106926337 shutdown_connections 2023-08-11T15:51:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.538+0000 7f7a9b59e640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a7c06a050 0x7f7a7c06c510 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.538+0000 7f7a9b59e640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a9c15c3f0 0x7f7a9c1f4570 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.538+0000 7f7a9b59e640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0cdb10 0x7f7a9c0c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.538+0000 7f7a9b59e640 1 --2- 172.21.15.27:0/3106926337 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a9c0cd6d0 0x7f7a9c1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.538+0000 7f7a9b59e640 1 -- 172.21.15.27:0/3106926337 >> 172.21.15.27:0/3106926337 conn(0x7f7a9c0caab0 msgr2=0x7f7a9c0705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.539+0000 7f7a9b59e640 1 -- 172.21.15.27:0/3106926337 shutdown_connections 2023-08-11T15:51:39.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.539+0000 7f7a9b59e640 1 -- 172.21.15.27:0/3106926337 wait complete. 2023-08-11T15:51:39.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:51:39.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.798+0000 7f97e2322640 1 Processor -- start 2023-08-11T15:51:39.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.798+0000 7f97e2322640 1 -- start start 2023-08-11T15:51:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.799+0000 7f97e2322640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97dc158a10 0x7f97dc070540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.800+0000 7f97e2322640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97dc0cce20 0x7f97dc0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.800+0000 7f97e2322640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97dc0cc0f0 con 0x7f97dc071130 2023-08-11T15:51:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.800+0000 7f97e2322640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97dc0cd770 con 0x7f97dc158a10 2023-08-11T15:51:39.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.800+0000 7f97e2322640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc158df0 con 0x7f97dc0cce20 2023-08-11T15:51:39.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.800+0000 7f97dbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97dc158a10 0x7f97dc070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.800+0000 7f97e08bf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97dc0cce20 0x7f97dc0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.800+0000 7f97dbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97dc158a10 0x7f97dc070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48098/0 (socket says 172.21.15.27:48098) 2023-08-11T15:51:39.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.800+0000 7f97dbfff640 1 -- 172.21.15.27:0/310334468 learned_addr learned my addr 172.21.15.27:0/310334468 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:39.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.801+0000 7f97dbfff640 1 -- 172.21.15.27:0/310334468 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97dc0cce20 msgr2=0x7f97dc0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:39.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.801+0000 7f97dbfff640 1 --2- 172.21.15.27:0/310334468 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97dc0cce20 0x7f97dc0cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.801+0000 7f97dbfff640 1 -- 172.21.15.27:0/310334468 >> v1:172.21.15.119:6789/0 conn(0x7f97dc071130 legacy=0x7f97dc06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.801+0000 7f97dbfff640 1 -- 172.21.15.27:0/310334468 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc070a80 con 0x7f97dc158a10 2023-08-11T15:51:39.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.803+0000 7f97dbfff640 1 --2- 172.21.15.27:0/310334468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97dc158a10 0x7f97dc070540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009870 tx=0x7f97cc02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b05565b4abc88ca server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.803+0000 7f97daffd640 1 -- 172.21.15.27:0/310334468 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc031640 con 0x7f97dc158a10 2023-08-11T15:51:39.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.803+0000 7f97daffd640 1 -- 172.21.15.27:0/310334468 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc0317a0 con 0x7f97dc158a10 2023-08-11T15:51:39.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.803+0000 7f97daffd640 1 -- 172.21.15.27:0/310334468 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc031b30 con 0x7f97dc158a10 2023-08-11T15:51:39.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.804+0000 7f97e2322640 1 -- 172.21.15.27:0/310334468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97dc158a10 msgr2=0x7f97dc070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.804+0000 7f97e2322640 1 --2- 172.21.15.27:0/310334468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97dc158a10 0x7f97dc070540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009870 tx=0x7f97cc02b8f0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.804+0000 7f97e2322640 1 -- 172.21.15.27:0/310334468 shutdown_connections 2023-08-11T15:51:39.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.804+0000 7f97e2322640 1 --2- 172.21.15.27:0/310334468 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97dc0cce20 0x7f97dc0cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.804+0000 7f97e2322640 1 --2- 172.21.15.27:0/310334468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97dc158a10 0x7f97dc070540 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.804+0000 7f97e2322640 1 -- 172.21.15.27:0/310334468 >> 172.21.15.27:0/310334468 conn(0x7f97dc0afbe0 msgr2=0x7f97dc0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:39.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.804+0000 7f97e2322640 1 -- 172.21.15.27:0/310334468 shutdown_connections 2023-08-11T15:51:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.804+0000 7f97e2322640 1 -- 172.21.15.27:0/310334468 wait complete. 2023-08-11T15:51:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.806+0000 7f97e2322640 1 Processor -- start 2023-08-11T15:51:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.806+0000 7f97e2322640 1 -- start start 2023-08-11T15:51:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.807+0000 7f97e2322640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97dc071130 0x7f97dc0c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.807+0000 7f97e2322640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97dc0cce20 0x7f97dc14c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.807+0000 7f97e2322640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97dc158a10 0x7f97dc14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.807+0000 7f97e2322640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc0c4850 con 0x7f97dc071130 2023-08-11T15:51:39.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.807+0000 7f97e2322640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc0c34c0 con 0x7f97dc158a10 2023-08-11T15:51:39.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.807+0000 7f97e2322640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc0c2af0 con 0x7f97dc0cce20 2023-08-11T15:51:39.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.807+0000 7f97dbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97dc071130 0x7f97dc0c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.808+0000 7f97dbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97dc071130 0x7f97dc0c2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50184/0 (socket says 172.21.15.27:50184) 2023-08-11T15:51:39.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.808+0000 7f97dbfff640 1 -- 172.21.15.27:0/4087386421 learned_addr learned my addr 172.21.15.27:0/4087386421 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.808+0000 7f97db7fe640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97dc0cce20 0x7f97dc14c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.808+0000 7f97e08bf640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97dc158a10 0x7f97dc14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.808+0000 7f97dbfff640 1 -- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97dc0cce20 msgr2=0x7f97dc14c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.808+0000 7f97dbfff640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97dc0cce20 0x7f97dc14c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.808+0000 7f97dbfff640 1 -- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97dc158a10 msgr2=0x7f97dc14c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.808+0000 7f97dbfff640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97dc158a10 0x7f97dc14c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.808+0000 7f97dbfff640 1 -- 172.21.15.27:0/4087386421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc0094d0 con 0x7f97dc071130 2023-08-11T15:51:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.808+0000 7f97dbfff640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97dc071130 0x7f97dc0c2f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97cc02b930 tx=0x7f97cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.809+0000 7f97d8ff9640 1 -- 172.21.15.27:0/4087386421 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc03a930 con 0x7f97dc071130 2023-08-11T15:51:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.809+0000 7f97d8ff9640 1 -- 172.21.15.27:0/4087386421 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc03aa90 con 0x7f97dc071130 2023-08-11T15:51:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.809+0000 7f97d8ff9640 1 -- 172.21.15.27:0/4087386421 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc03ae40 con 0x7f97dc071130 2023-08-11T15:51:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.809+0000 7f97e2322640 1 -- 172.21.15.27:0/4087386421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc0c24e0 con 0x7f97dc071130 2023-08-11T15:51:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.809+0000 7f97e2322640 1 -- 172.21.15.27:0/4087386421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc0c6220 con 0x7f97dc071130 2023-08-11T15:51:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.810+0000 7f97e2322640 1 -- 172.21.15.27:0/4087386421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc04e420 con 0x7f97dc071130 2023-08-11T15:51:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.811+0000 7f97d8ff9640 1 -- 172.21.15.27:0/4087386421 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f97cc02f050 con 0x7f97dc071130 2023-08-11T15:51:39.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.812+0000 7f97d8ff9640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f97c806a050 0x7f97c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.812+0000 7f97d8ff9640 1 -- 172.21.15.27:0/4087386421 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f97cc0aa750 con 0x7f97dc071130 2023-08-11T15:51:39.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.813+0000 7f97db7fe640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f97c806a050 0x7f97c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.814+0000 7f97db7fe640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f97c806a050 0x7f97c806c510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f97dc1f5320 tx=0x7f97d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:39.819+0000 7f97d8ff9640 1 -- 172.21.15.27:0/4087386421 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f97cc075a70 con 0x7f97dc071130 2023-08-11T15:51:40.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.011+0000 7f97e2322640 1 -- 172.21.15.27:0/4087386421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97dc06bc00 con 0x7f97dc071130 2023-08-11T15:51:40.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.012+0000 7f97d8ff9640 1 -- 172.21.15.27:0/4087386421 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97cc079090 con 0x7f97dc071130 2023-08-11T15:51:40.014 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:51:40.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 -- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f97c806a050 msgr2=0x7f97c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f97c806a050 0x7f97c806c510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f97dc1f5320 tx=0x7f97d0009040 comp rx=0 tx=0).stop 2023-08-11T15:51:40.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 -- 172.21.15.27:0/4087386421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97dc071130 msgr2=0x7f97dc0c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97dc071130 0x7f97dc0c2f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97cc02b930 tx=0x7f97cc0046c0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 -- 172.21.15.27:0/4087386421 shutdown_connections 2023-08-11T15:51:40.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f97c806a050 0x7f97c806c510 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97dc0cce20 0x7f97dc14c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97dc158a10 0x7f97dc14c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 --2- 172.21.15.27:0/4087386421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97dc071130 0x7f97dc0c2f80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 -- 172.21.15.27:0/4087386421 >> 172.21.15.27:0/4087386421 conn(0x7f97dc0afbe0 msgr2=0x7f97dc0adc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:40.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 -- 172.21.15.27:0/4087386421 shutdown_connections 2023-08-11T15:51:40.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.015+0000 7f97e2322640 1 -- 172.21.15.27:0/4087386421 wait complete. 2023-08-11T15:51:40.037 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:51:40.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.277+0000 7f3efc4fa640 1 Processor -- start 2023-08-11T15:51:40.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.278+0000 7f3efc4fa640 1 -- start start 2023-08-11T15:51:40.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.279+0000 7f3efc4fa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3ef4151600 0x7f3ef415df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.279+0000 7f3efc4fa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef406c850 con 0x7f3ef415e4b0 2023-08-11T15:51:40.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.279+0000 7f3efc4fa640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef406bfc0 con 0x7f3ef4151220 2023-08-11T15:51:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.279+0000 7f3efc4fa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef415e940 con 0x7f3ef4151600 2023-08-11T15:51:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.279+0000 7f3ef9a95640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3ef4151600 0x7f3ef415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.279+0000 7f3efaa97640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3ef415e4b0 0x7f3ef40ca850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48478/0 (socket says 172.21.15.27:48478) 2023-08-11T15:51:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.279+0000 7f3efaa97640 1 -- 172.21.15.27:0/2229725901 learned_addr learned my addr 172.21.15.27:0/2229725901 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.280+0000 7f3ef9a95640 1 -- 172.21.15.27:0/2229725901 >> v1:172.21.15.119:6789/0 conn(0x7f3ef4151220 legacy=0x7f3ef406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.280+0000 7f3ef9a95640 1 -- 172.21.15.27:0/2229725901 >> v1:172.21.15.27:6789/0 conn(0x7f3ef415e4b0 legacy=0x7f3ef40ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.280+0000 7f3ef9a95640 1 -- 172.21.15.27:0/2229725901 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef406c290 con 0x7f3ef4151600 2023-08-11T15:51:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.280+0000 7f3ef9294640 1 -- 172.21.15.27:0/2229725901 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265636290 0 0) 0x7f3ef406c850 con 0x7f3ef415e4b0 2023-08-11T15:51:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.281+0000 7f3ef9a95640 1 --2- 172.21.15.27:0/2229725901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef4151600 0x7f3ef415df70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40097a0 tx=0x7f3ee402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1131afc16550399 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.281+0000 7f3ef9294640 1 -- 172.21.15.27:0/2229725901 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee402e070 con 0x7f3ef4151600 2023-08-11T15:51:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.282+0000 7f3ef9294640 1 -- 172.21.15.27:0/2229725901 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee40315a0 con 0x7f3ef4151600 2023-08-11T15:51:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.282+0000 7f3ef9294640 1 -- 172.21.15.27:0/2229725901 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4031930 con 0x7f3ef4151600 2023-08-11T15:51:40.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.282+0000 7f3efc4fa640 1 -- 172.21.15.27:0/2229725901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef4151600 msgr2=0x7f3ef415df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.282+0000 7f3efc4fa640 1 --2- 172.21.15.27:0/2229725901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef4151600 0x7f3ef415df70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40097a0 tx=0x7f3ee402b840 comp rx=0 tx=0).stop 2023-08-11T15:51:40.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.282+0000 7f3efc4fa640 1 -- 172.21.15.27:0/2229725901 shutdown_connections 2023-08-11T15:51:40.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.282+0000 7f3efc4fa640 1 --2- 172.21.15.27:0/2229725901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef4151600 0x7f3ef415df70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.282+0000 7f3efc4fa640 1 -- 172.21.15.27:0/2229725901 >> 172.21.15.27:0/2229725901 conn(0x7f3ef40afbe0 msgr2=0x7f3ef40affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:40.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.282+0000 7f3efc4fa640 1 -- 172.21.15.27:0/2229725901 shutdown_connections 2023-08-11T15:51:40.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.282+0000 7f3efc4fa640 1 -- 172.21.15.27:0/2229725901 wait complete. 2023-08-11T15:51:40.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.284+0000 7f3efc4fa640 1 Processor -- start 2023-08-11T15:51:40.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.284+0000 7f3efc4fa640 1 -- start start 2023-08-11T15:51:40.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.285+0000 7f3efc4fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef4151220 0x7f3ef41543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.285+0000 7f3efc4fa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ef4151600 0x7f3ef40adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.285+0000 7f3efc4fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef415e4b0 0x7f3ef40ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.285+0000 7f3efc4fa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4153710 con 0x7f3ef415e4b0 2023-08-11T15:51:40.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.285+0000 7f3efc4fa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40b2620 con 0x7f3ef4151220 2023-08-11T15:51:40.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.285+0000 7f3efc4fa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40b22f0 con 0x7f3ef4151600 2023-08-11T15:51:40.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.286+0000 7f3efaa97640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef415e4b0 0x7f3ef40ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.286+0000 7f3efaa97640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef415e4b0 0x7f3ef40ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50204/0 (socket says 172.21.15.27:50204) 2023-08-11T15:51:40.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.286+0000 7f3efaa97640 1 -- 172.21.15.27:0/3312447380 learned_addr learned my addr 172.21.15.27:0/3312447380 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:40.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.286+0000 7f3efa296640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef4151220 0x7f3ef41543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.286+0000 7f3ef9a95640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ef4151600 0x7f3ef40adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.286+0000 7f3efaa97640 1 -- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ef4151600 msgr2=0x7f3ef40adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.286+0000 7f3efaa97640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ef4151600 0x7f3ef40adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.286+0000 7f3efaa97640 1 -- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef4151220 msgr2=0x7f3ef41543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.286+0000 7f3efaa97640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef4151220 0x7f3ef41543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.286+0000 7f3efaa97640 1 -- 172.21.15.27:0/3312447380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4009400 con 0x7f3ef415e4b0 2023-08-11T15:51:40.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.287+0000 7f3efaa97640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef415e4b0 0x7f3ef40ae070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c7d0 tx=0x7f3eec00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:40.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.287+0000 7f3eeaffd640 1 -- 172.21.15.27:0/3312447380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00a640 con 0x7f3ef415e4b0 2023-08-11T15:51:40.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.287+0000 7f3eeaffd640 1 -- 172.21.15.27:0/3312447380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec00a7a0 con 0x7f3ef415e4b0 2023-08-11T15:51:40.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.287+0000 7f3efc4fa640 1 -- 172.21.15.27:0/3312447380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef4152e70 con 0x7f3ef415e4b0 2023-08-11T15:51:40.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.287+0000 7f3eeaffd640 1 -- 172.21.15.27:0/3312447380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00ab90 con 0x7f3ef415e4b0 2023-08-11T15:51:40.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.287+0000 7f3efc4fa640 1 -- 172.21.15.27:0/3312447380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef4153430 con 0x7f3ef415e4b0 2023-08-11T15:51:40.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.288+0000 7f3efc4fa640 1 -- 172.21.15.27:0/3312447380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef404e420 con 0x7f3ef415e4b0 2023-08-11T15:51:40.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.289+0000 7f3eeaffd640 1 -- 172.21.15.27:0/3312447380 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3eec00acf0 con 0x7f3ef415e4b0 2023-08-11T15:51:40.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.290+0000 7f3eeaffd640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3ed806a050 0x7f3ed806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.290+0000 7f3eeaffd640 1 -- 172.21.15.27:0/3312447380 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3eec0881a0 con 0x7f3ef415e4b0 2023-08-11T15:51:40.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.291+0000 7f3efa296640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3ed806a050 0x7f3ed806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.292+0000 7f3efa296640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3ed806a050 0x7f3ed806c510 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40ae750 tx=0x7f3ef0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:40.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.294+0000 7f3eeaffd640 1 -- 172.21.15.27:0/3312447380 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f3eec0534c0 con 0x7f3ef415e4b0 2023-08-11T15:51:40.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.488+0000 7f3efc4fa640 1 -- 172.21.15.27:0/3312447380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3ef40b2760 con 0x7f3ef415e4b0 2023-08-11T15:51:40.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.489+0000 7f3eeaffd640 1 -- 172.21.15.27:0/3312447380 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3eec056ae0 con 0x7f3ef415e4b0 2023-08-11T15:51:40.490 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:51:40.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.491+0000 7f3efc4fa640 1 -- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3ed806a050 msgr2=0x7f3ed806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.491+0000 7f3efc4fa640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3ed806a050 0x7f3ed806c510 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40ae750 tx=0x7f3ef0009040 comp rx=0 tx=0).stop 2023-08-11T15:51:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.491+0000 7f3efc4fa640 1 -- 172.21.15.27:0/3312447380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef415e4b0 msgr2=0x7f3ef40ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.491+0000 7f3efc4fa640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef415e4b0 0x7f3ef40ae070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c7d0 tx=0x7f3eec00cc90 comp rx=0 tx=0).stop 2023-08-11T15:51:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.491+0000 7f3efc4fa640 1 -- 172.21.15.27:0/3312447380 shutdown_connections 2023-08-11T15:51:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.491+0000 7f3efc4fa640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3ed806a050 0x7f3ed806c510 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.491+0000 7f3efc4fa640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ef4151600 0x7f3ef40adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.491+0000 7f3efc4fa640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef415e4b0 0x7f3ef40ae070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.491+0000 7f3efc4fa640 1 --2- 172.21.15.27:0/3312447380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef4151220 0x7f3ef41543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.491+0000 7f3efc4fa640 1 -- 172.21.15.27:0/3312447380 >> 172.21.15.27:0/3312447380 conn(0x7f3ef40afbe0 msgr2=0x7f3ef415dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.492+0000 7f3efc4fa640 1 -- 172.21.15.27:0/3312447380 shutdown_connections 2023-08-11T15:51:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.492+0000 7f3efc4fa640 1 -- 172.21.15.27:0/3312447380 wait complete. 2023-08-11T15:51:40.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:51:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.756+0000 7f7ab09ce640 1 Processor -- start 2023-08-11T15:51:40.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.756+0000 7f7ab09ce640 1 -- start start 2023-08-11T15:51:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.758+0000 7f7ab09ce640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7aac1590a0 0x7f7aac0c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.758+0000 7f7ab09ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7aac0cd2a0 0x7f7aac0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.758+0000 7f7ab09ce640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aac15ca90 con 0x7f7aac0c94d0 2023-08-11T15:51:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.758+0000 7f7ab09ce640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aac0ca200 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.758+0000 7f7ab09ce640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aac0cdbe0 con 0x7f7aac1590a0 2023-08-11T15:51:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.758+0000 7f7aab59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7aac0cd2a0 0x7f7aac0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.758+0000 7f7aab59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7aac0cd2a0 0x7f7aac0cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50220/0 (socket says 172.21.15.27:50220) 2023-08-11T15:51:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.758+0000 7f7aab59e640 1 -- 172.21.15.27:0/1768553733 learned_addr learned my addr 172.21.15.27:0/1768553733 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.758+0000 7f7aaa59c640 1 --1- 172.21.15.27:0/1768553733 >> v1:172.21.15.27:6789/0 conn(0x7f7aac0c94d0 0x7f7aac15d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48488/0 (socket says 172.21.15.27:48488) 2023-08-11T15:51:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.759+0000 7f7aaad9d640 1 --2- 172.21.15.27:0/1768553733 >> v2:172.21.15.119:3300/0 conn(0x7f7aac1590a0 0x7f7aac0c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.759+0000 7f7aab59e640 1 -- 172.21.15.27:0/1768553733 >> v2:172.21.15.119:3300/0 conn(0x7f7aac1590a0 msgr2=0x7f7aac0c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:40.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.759+0000 7f7aab59e640 1 --2- 172.21.15.27:0/1768553733 >> v2:172.21.15.119:3300/0 conn(0x7f7aac1590a0 0x7f7aac0c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.759+0000 7f7aab59e640 1 -- 172.21.15.27:0/1768553733 >> v1:172.21.15.27:6789/0 conn(0x7f7aac0c94d0 legacy=0x7f7aac15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.759+0000 7f7aab59e640 1 -- 172.21.15.27:0/1768553733 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac0c9960 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.759+0000 7f7aa9d9b640 1 -- 172.21.15.27:0/1768553733 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071135679 0 0) 0x7f7aac15ca90 con 0x7f7aac0c94d0 2023-08-11T15:51:40.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.759+0000 7f7aaad9d640 1 --2- 172.21.15.27:0/1768553733 >> v2:172.21.15.119:3300/0 conn(0x7f7aac1590a0 0x7f7aac0c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:40.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.760+0000 7f7aab59e640 1 --2- 172.21.15.27:0/1768553733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 0x7f7aac0cd6a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7aac066b80 tx=0x7f7aa402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396f83d7aab462a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:40.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7aa9d9b640 1 -- 172.21.15.27:0/1768553733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa402ca80 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7aa9d9b640 1 -- 172.21.15.27:0/1768553733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa402cbe0 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7aa9d9b640 1 -- 172.21.15.27:0/1768553733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40343f0 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1768553733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 msgr2=0x7f7aac0cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7ab09ce640 1 --2- 172.21.15.27:0/1768553733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 0x7f7aac0cd6a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7aac066b80 tx=0x7f7aa402bee0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1768553733 shutdown_connections 2023-08-11T15:51:40.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7ab09ce640 1 --2- 172.21.15.27:0/1768553733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 0x7f7aac0cd6a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7ab09ce640 1 --2- 172.21.15.27:0/1768553733 >> v2:172.21.15.119:3300/0 conn(0x7f7aac1590a0 0x7f7aac0c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1768553733 >> 172.21.15.27:0/1768553733 conn(0x7f7aac0afb70 msgr2=0x7f7aac0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:40.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1768553733 shutdown_connections 2023-08-11T15:51:40.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.761+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1768553733 wait complete. 2023-08-11T15:51:40.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.763+0000 7f7ab09ce640 1 Processor -- start 2023-08-11T15:51:40.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.763+0000 7f7ab09ce640 1 -- start start 2023-08-11T15:51:40.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.764+0000 7f7ab09ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 0x7f7aac1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.764+0000 7f7ab09ce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aac1590a0 0x7f7aac0c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.764+0000 7f7ab09ce640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7aac0c7090 0x7f7aac15b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.764+0000 7f7ab09ce640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0c7e80 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.764+0000 7f7ab09ce640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0c51a0 con 0x7f7aac1590a0 2023-08-11T15:51:40.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.764+0000 7f7ab09ce640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac157d30 con 0x7f7aac0c7090 2023-08-11T15:51:40.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aaad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 0x7f7aac1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aaad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 0x7f7aac1577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50236/0 (socket says 172.21.15.27:50236) 2023-08-11T15:51:40.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aaad9d640 1 -- 172.21.15.27:0/1951233569 learned_addr learned my addr 172.21.15.27:0/1951233569 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:40.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aaa59c640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aac1590a0 0x7f7aac0c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aab59e640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7aac0c7090 0x7f7aac15b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aaad9d640 1 -- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7aac0c7090 msgr2=0x7f7aac15b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aaad9d640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7aac0c7090 0x7f7aac15b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aaad9d640 1 -- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aac1590a0 msgr2=0x7f7aac0c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:40.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aaad9d640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aac1590a0 0x7f7aac0c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:40.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aaad9d640 1 -- 172.21.15.27:0/1951233569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa4009b30 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.765+0000 7f7aab59e640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7aac0c7090 0x7f7aac15b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:40.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.766+0000 7f7aaad9d640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 0x7f7aac1577f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e7e0 tx=0x7f7a9c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:40.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.766+0000 7f7a937fe640 1 -- 172.21.15.27:0/1951233569 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c004030 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.766+0000 7f7a937fe640 1 -- 172.21.15.27:0/1951233569 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c004190 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.766+0000 7f7a937fe640 1 -- 172.21.15.27:0/1951233569 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c004520 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.766+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1951233569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac0c75e0 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.766+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1951233569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac0c7ba0 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.767+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1951233569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac04e420 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.768+0000 7f7a937fe640 1 -- 172.21.15.27:0/1951233569 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7a9c00c6d0 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.769+0000 7f7a937fe640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a9806a080 0x7f7a9806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:40.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.770+0000 7f7a937fe640 1 -- 172.21.15.27:0/1951233569 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7a9c0885e0 con 0x7f7aac0cd2a0 2023-08-11T15:51:40.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.770+0000 7f7aaa59c640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a9806a080 0x7f7a9806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:40.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.771+0000 7f7aaa59c640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a9806a080 0x7f7a9806c540 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0b3e70 tx=0x7f7aa0007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:40.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:40.774+0000 7f7a937fe640 1 -- 172.21.15.27:0/1951233569 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f7a9c053990 con 0x7f7aac0cd2a0 2023-08-11T15:51:41.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.005+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1951233569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7aac072960 con 0x7f7aac0cd2a0 2023-08-11T15:51:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.007+0000 7f7a937fe640 1 -- 172.21.15.27:0/1951233569 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f7a9c056fb0 con 0x7f7aac0cd2a0 2023-08-11T15:51:41.008 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:41.009 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:51:01.220970+0000","modified":"2023-08-11T15:51:25.832166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4315":{"gid":4315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:51:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2023-08-11T15:51:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.009+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a9806a080 msgr2=0x7f7a9806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.009+0000 7f7ab09ce640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a9806a080 0x7f7a9806c540 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0b3e70 tx=0x7f7aa0007c00 comp rx=0 tx=0).stop 2023-08-11T15:51:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.009+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1951233569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 msgr2=0x7f7aac1577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.009+0000 7f7ab09ce640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 0x7f7aac1577f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e7e0 tx=0x7f7a9c00eca0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.010+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1951233569 shutdown_connections 2023-08-11T15:51:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.010+0000 7f7ab09ce640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7a9806a080 0x7f7a9806c540 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.010+0000 7f7ab09ce640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7aac0c7090 0x7f7aac15b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.010+0000 7f7ab09ce640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7aac1590a0 0x7f7aac0c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.010+0000 7f7ab09ce640 1 --2- 172.21.15.27:0/1951233569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7aac0cd2a0 0x7f7aac1577f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.010+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1951233569 >> 172.21.15.27:0/1951233569 conn(0x7f7aac0afb70 msgr2=0x7f7aac070290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:41.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.010+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1951233569 shutdown_connections 2023-08-11T15:51:41.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.010+0000 7f7ab09ce640 1 -- 172.21.15.27:0/1951233569 wait complete. 2023-08-11T15:51:41.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:41.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.267+0000 7f5dc994f640 1 Processor -- start 2023-08-11T15:51:41.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.267+0000 7f5dc994f640 1 -- start start 2023-08-11T15:51:41.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.269+0000 7f5dc994f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5dc415c620 0x7f5dc40b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.269+0000 7f5dc994f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5dc40cfb00 0x7f5dc40d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.269+0000 7f5dc994f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc40cda30 con 0x7f5dc40b82f0 2023-08-11T15:51:41.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.269+0000 7f5dc994f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc40ccc10 con 0x7f5dc40cfb00 2023-08-11T15:51:41.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.269+0000 7f5dc994f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc4161110 con 0x7f5dc415c620 2023-08-11T15:51:41.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.269+0000 7f5dc37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5dc40cfb00 0x7f5dc40d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.269+0000 7f5dc27fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5dc40b82f0 0x7f5dc40cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48498/0 (socket says 172.21.15.27:48498) 2023-08-11T15:51:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.269+0000 7f5dc27fc640 1 -- 172.21.15.27:0/3520771290 learned_addr learned my addr 172.21.15.27:0/3520771290 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.269+0000 7f5dc2ffd640 1 --2- 172.21.15.27:0/3520771290 >> v2:172.21.15.119:3300/0 conn(0x7f5dc415c620 0x7f5dc40b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.270+0000 7f5dc1ffb640 1 -- 172.21.15.27:0/3520771290 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953117783 0 0) 0x7f5dc40cda30 con 0x7f5dc40b82f0 2023-08-11T15:51:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.270+0000 7f5dc1ffb640 1 -- 172.21.15.27:0/3520771290 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db0003520 con 0x7f5dc40b82f0 2023-08-11T15:51:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.270+0000 7f5dc37fe640 1 -- 172.21.15.27:0/3520771290 >> v2:172.21.15.119:3300/0 conn(0x7f5dc415c620 msgr2=0x7f5dc40b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:41.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.270+0000 7f5dc37fe640 1 --2- 172.21.15.27:0/3520771290 >> v2:172.21.15.119:3300/0 conn(0x7f5dc415c620 0x7f5dc40b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.270+0000 7f5dc37fe640 1 -- 172.21.15.27:0/3520771290 >> v1:172.21.15.27:6789/0 conn(0x7f5dc40b82f0 legacy=0x7f5dc40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.270+0000 7f5dc37fe640 1 -- 172.21.15.27:0/3520771290 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40cce20 con 0x7f5dc40cfb00 2023-08-11T15:51:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.271+0000 7f5dc37fe640 1 --2- 172.21.15.27:0/3520771290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40cfb00 0x7f5dc40d0b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40cefa0 tx=0x7f5dbc02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b90c5856cedbd33 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.271+0000 7f5dc1ffb640 1 -- 172.21.15.27:0/3520771290 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc02cc70 con 0x7f5dc40cfb00 2023-08-11T15:51:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.271+0000 7f5dc1ffb640 1 -- 172.21.15.27:0/3520771290 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc02cdd0 con 0x7f5dc40cfb00 2023-08-11T15:51:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.271+0000 7f5dc1ffb640 1 -- 172.21.15.27:0/3520771290 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc034640 con 0x7f5dc40cfb00 2023-08-11T15:51:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.272+0000 7f5dc994f640 1 -- 172.21.15.27:0/3520771290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40cfb00 msgr2=0x7f5dc40d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.272+0000 7f5dc994f640 1 --2- 172.21.15.27:0/3520771290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40cfb00 0x7f5dc40d0b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40cefa0 tx=0x7f5dbc02bec0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.272+0000 7f5dc994f640 1 -- 172.21.15.27:0/3520771290 shutdown_connections 2023-08-11T15:51:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.272+0000 7f5dc994f640 1 --2- 172.21.15.27:0/3520771290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40cfb00 0x7f5dc40d0b00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.272+0000 7f5dc994f640 1 --2- 172.21.15.27:0/3520771290 >> v2:172.21.15.119:3300/0 conn(0x7f5dc415c620 0x7f5dc40b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.272+0000 7f5dc994f640 1 -- 172.21.15.27:0/3520771290 >> 172.21.15.27:0/3520771290 conn(0x7f5dc4071110 msgr2=0x7f5dc40714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.272+0000 7f5dc994f640 1 -- 172.21.15.27:0/3520771290 shutdown_connections 2023-08-11T15:51:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.272+0000 7f5dc994f640 1 -- 172.21.15.27:0/3520771290 wait complete. 2023-08-11T15:51:41.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.274+0000 7f5dc994f640 1 Processor -- start 2023-08-11T15:51:41.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.274+0000 7f5dc994f640 1 -- start start 2023-08-11T15:51:41.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.275+0000 7f5dc994f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40b82f0 0x7f5dc4148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.275+0000 7f5dc994f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5dc40cfb00 0x7f5dc40cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.275+0000 7f5dc994f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc415c620 0x7f5dc4149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.275+0000 7f5dc994f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc4149950 con 0x7f5dc40b82f0 2023-08-11T15:51:41.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.275+0000 7f5dc994f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40c9070 con 0x7f5dc415c620 2023-08-11T15:51:41.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.275+0000 7f5dc994f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40cc930 con 0x7f5dc40cfb00 2023-08-11T15:51:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.275+0000 7f5dc2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40b82f0 0x7f5dc4148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40b82f0 0x7f5dc4148640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50258/0 (socket says 172.21.15.27:50258) 2023-08-11T15:51:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc2ffd640 1 -- 172.21.15.27:0/857746956 learned_addr learned my addr 172.21.15.27:0/857746956 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc27fc640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5dc40cfb00 0x7f5dc40cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc2ffd640 1 -- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5dc40cfb00 msgr2=0x7f5dc40cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc37fe640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc415c620 0x7f5dc4149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc2ffd640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5dc40cfb00 0x7f5dc40cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc2ffd640 1 -- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc415c620 msgr2=0x7f5dc4149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc2ffd640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc415c620 0x7f5dc4149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc2ffd640 1 -- 172.21.15.27:0/857746956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc009a70 con 0x7f5dc40b82f0 2023-08-11T15:51:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc27fc640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5dc40cfb00 0x7f5dc40cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc37fe640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc415c620 0x7f5dc4149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.276+0000 7f5dc2ffd640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40b82f0 0x7f5dc4148640 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5db400e870 tx=0x7f5db400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.277+0000 7f5dbbfff640 1 -- 172.21.15.27:0/857746956 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db40094f0 con 0x7f5dc40b82f0 2023-08-11T15:51:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.277+0000 7f5dbbfff640 1 -- 172.21.15.27:0/857746956 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db400c8e0 con 0x7f5dc40b82f0 2023-08-11T15:51:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.277+0000 7f5dbbfff640 1 -- 172.21.15.27:0/857746956 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400cc90 con 0x7f5dc40b82f0 2023-08-11T15:51:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.277+0000 7f5dc994f640 1 -- 172.21.15.27:0/857746956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc4148cd0 con 0x7f5dc40b82f0 2023-08-11T15:51:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.277+0000 7f5dc994f640 1 -- 172.21.15.27:0/857746956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc4149290 con 0x7f5dc40b82f0 2023-08-11T15:51:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.278+0000 7f5dc994f640 1 -- 172.21.15.27:0/857746956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc41607d0 con 0x7f5dc40b82f0 2023-08-11T15:51:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.279+0000 7f5dbbfff640 1 -- 172.21.15.27:0/857746956 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5db4013470 con 0x7f5dc40b82f0 2023-08-11T15:51:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.280+0000 7f5dbbfff640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5da406a050 0x7f5da406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.280+0000 7f5dbbfff640 1 -- 172.21.15.27:0/857746956 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5db4088900 con 0x7f5dc40b82f0 2023-08-11T15:51:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.281+0000 7f5dc27fc640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5da406a050 0x7f5da406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.282+0000 7f5dc27fc640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5da406a050 0x7f5da406c510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40c98f0 tx=0x7f5dac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.284+0000 7f5dbbfff640 1 -- 172.21.15.27:0/857746956 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f5db4053cc0 con 0x7f5dc40b82f0 2023-08-11T15:51:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.481+0000 7f5dc994f640 1 -- 172.21.15.27:0/857746956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dc4073990 con 0x7f5dc40b82f0 2023-08-11T15:51:41.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.485+0000 7f5dbbfff640 1 -- 172.21.15.27:0/857746956 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f5db4021090 con 0x7f5dc40b82f0 2023-08-11T15:51:41.486 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:41.487 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:41.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.487+0000 7f5dc994f640 1 -- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5da406a050 msgr2=0x7f5da406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.487+0000 7f5dc994f640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5da406a050 0x7f5da406c510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40c98f0 tx=0x7f5dac009040 comp rx=0 tx=0).stop 2023-08-11T15:51:41.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.487+0000 7f5dc994f640 1 -- 172.21.15.27:0/857746956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40b82f0 msgr2=0x7f5dc4148640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.487+0000 7f5dc994f640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40b82f0 0x7f5dc4148640 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5db400e870 tx=0x7f5db400ed30 comp rx=0 tx=0).stop 2023-08-11T15:51:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.488+0000 7f5dc994f640 1 -- 172.21.15.27:0/857746956 shutdown_connections 2023-08-11T15:51:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.488+0000 7f5dc994f640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5da406a050 0x7f5da406c510 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.488+0000 7f5dc994f640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5dc40cfb00 0x7f5dc40cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.488+0000 7f5dc994f640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc415c620 0x7f5dc4149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.488+0000 7f5dc994f640 1 --2- 172.21.15.27:0/857746956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dc40b82f0 0x7f5dc4148640 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.488+0000 7f5dc994f640 1 -- 172.21.15.27:0/857746956 >> 172.21.15.27:0/857746956 conn(0x7f5dc4071110 msgr2=0x7f5dc40b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.488+0000 7f5dc994f640 1 -- 172.21.15.27:0/857746956 shutdown_connections 2023-08-11T15:51:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.488+0000 7f5dc994f640 1 -- 172.21.15.27:0/857746956 wait complete. 2023-08-11T15:51:41.510 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:51:41.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.743+0000 7fc750b0f640 1 Processor -- start 2023-08-11T15:51:41.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.743+0000 7fc750b0f640 1 -- start start 2023-08-11T15:51:41.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.745+0000 7fc750b0f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc74c15da60 0x7fc74c0cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.745+0000 7fc750b0f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc74c0b1420 0x7fc74c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.745+0000 7fc750b0f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74c0cea00 con 0x7fc74c0cee90 2023-08-11T15:51:41.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.745+0000 7fc750b0f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c15de40 con 0x7fc74c15da60 2023-08-11T15:51:41.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.745+0000 7fc750b0f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c0b0e80 con 0x7fc74c0b1420 2023-08-11T15:51:41.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.745+0000 7fc74b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc74c15da60 0x7fc74c0cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.745+0000 7fc74b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc74c15da60 0x7fc74c0cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50260/0 (socket says 172.21.15.27:50260) 2023-08-11T15:51:41.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.745+0000 7fc74b7fe640 1 -- 172.21.15.27:0/174417011 learned_addr learned my addr 172.21.15.27:0/174417011 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:41.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.746+0000 7fc742ffd640 1 --2- 172.21.15.27:0/174417011 >> v2:172.21.15.119:3300/0 conn(0x7fc74c0b1420 0x7fc74c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.746+0000 7fc74b7fe640 1 -- 172.21.15.27:0/174417011 >> v2:172.21.15.119:3300/0 conn(0x7fc74c0b1420 msgr2=0x7fc74c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:41.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.746+0000 7fc74b7fe640 1 --2- 172.21.15.27:0/174417011 >> v2:172.21.15.119:3300/0 conn(0x7fc74c0b1420 0x7fc74c0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.746+0000 7fc74b7fe640 1 -- 172.21.15.27:0/174417011 >> v1:172.21.15.27:6789/0 conn(0x7fc74c0cee90 legacy=0x7fc74c0c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.746+0000 7fc74b7fe640 1 -- 172.21.15.27:0/174417011 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c0d0bd0 con 0x7fc74c15da60 2023-08-11T15:51:41.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.747+0000 7fc74b7fe640 1 --2- 172.21.15.27:0/174417011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c15da60 0x7fc74c0cc970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0caac0 tx=0x7fc73c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23b98740440318b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:41.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.748+0000 7fc74affd640 1 -- 172.21.15.27:0/174417011 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c02cb10 con 0x7fc74c15da60 2023-08-11T15:51:41.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.748+0000 7fc74affd640 1 -- 172.21.15.27:0/174417011 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c02cc70 con 0x7fc74c15da60 2023-08-11T15:51:41.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.748+0000 7fc74affd640 1 -- 172.21.15.27:0/174417011 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c0343f0 con 0x7fc74c15da60 2023-08-11T15:51:41.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.749+0000 7fc750b0f640 1 -- 172.21.15.27:0/174417011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c15da60 msgr2=0x7fc74c0cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.749+0000 7fc750b0f640 1 --2- 172.21.15.27:0/174417011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c15da60 0x7fc74c0cc970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0caac0 tx=0x7fc73c02bf70 comp rx=0 tx=0).stop 2023-08-11T15:51:41.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.749+0000 7fc750b0f640 1 -- 172.21.15.27:0/174417011 shutdown_connections 2023-08-11T15:51:41.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.749+0000 7fc750b0f640 1 --2- 172.21.15.27:0/174417011 >> v2:172.21.15.119:3300/0 conn(0x7fc74c0b1420 0x7fc74c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.749+0000 7fc750b0f640 1 --2- 172.21.15.27:0/174417011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c15da60 0x7fc74c0cc970 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.749+0000 7fc750b0f640 1 -- 172.21.15.27:0/174417011 >> 172.21.15.27:0/174417011 conn(0x7fc74c0d0010 msgr2=0x7fc74c0d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:41.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.749+0000 7fc750b0f640 1 -- 172.21.15.27:0/174417011 shutdown_connections 2023-08-11T15:51:41.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.749+0000 7fc750b0f640 1 -- 172.21.15.27:0/174417011 wait complete. 2023-08-11T15:51:41.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.751+0000 7fc750b0f640 1 Processor -- start 2023-08-11T15:51:41.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.751+0000 7fc750b0f640 1 -- start start 2023-08-11T15:51:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.752+0000 7fc750b0f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c0b1420 0x7fc74c146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.752+0000 7fc750b0f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc74c0cee90 0x7fc74c146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.752+0000 7fc750b0f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc74c15da60 0x7fc74c1f1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.752+0000 7fc750b0f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c147120 con 0x7fc74c0b1420 2023-08-11T15:51:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.752+0000 7fc750b0f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0b3510 con 0x7fc74c0cee90 2023-08-11T15:51:41.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.752+0000 7fc750b0f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0c57c0 con 0x7fc74c15da60 2023-08-11T15:51:41.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc74b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c0b1420 0x7fc74c146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc74b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c0b1420 0x7fc74c146340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50274/0 (socket says 172.21.15.27:50274) 2023-08-11T15:51:41.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc74b7fe640 1 -- 172.21.15.27:0/556880575 learned_addr learned my addr 172.21.15.27:0/556880575 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:41.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc74bfff640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc74c15da60 0x7fc74c1f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc74b7fe640 1 -- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc74c15da60 msgr2=0x7fc74c1f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc74b7fe640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc74c15da60 0x7fc74c1f1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc742ffd640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc74c0cee90 0x7fc74c146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc74b7fe640 1 -- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc74c0cee90 msgr2=0x7fc74c146880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc74b7fe640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc74c0cee90 0x7fc74c146880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc74b7fe640 1 -- 172.21.15.27:0/556880575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c009b30 con 0x7fc74c0b1420 2023-08-11T15:51:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc74bfff640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc74c15da60 0x7fc74c1f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.753+0000 7fc742ffd640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc74c0cee90 0x7fc74c146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.754+0000 7fc74b7fe640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c0b1420 0x7fc74c146340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc73c000f80 tx=0x7fc73c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:41.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.754+0000 7fc748ff9640 1 -- 172.21.15.27:0/556880575 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c009510 con 0x7fc74c0b1420 2023-08-11T15:51:41.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.754+0000 7fc748ff9640 1 -- 172.21.15.27:0/556880575 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c03e450 con 0x7fc74c0b1420 2023-08-11T15:51:41.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.754+0000 7fc748ff9640 1 -- 172.21.15.27:0/556880575 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c03e800 con 0x7fc74c0b1420 2023-08-11T15:51:41.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.754+0000 7fc750b0f640 1 -- 172.21.15.27:0/556880575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74c0c5aa0 con 0x7fc74c0b1420 2023-08-11T15:51:41.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.754+0000 7fc750b0f640 1 -- 172.21.15.27:0/556880575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74c0c6060 con 0x7fc74c0b1420 2023-08-11T15:51:41.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.756+0000 7fc750b0f640 1 -- 172.21.15.27:0/556880575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74c04e420 con 0x7fc74c0b1420 2023-08-11T15:51:41.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.757+0000 7fc748ff9640 1 -- 172.21.15.27:0/556880575 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc73c03e960 con 0x7fc74c0b1420 2023-08-11T15:51:41.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.758+0000 7fc748ff9640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc72c06a050 0x7fc72c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:41.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.758+0000 7fc748ff9640 1 -- 172.21.15.27:0/556880575 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc73c0ab720 con 0x7fc74c0b1420 2023-08-11T15:51:41.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.758+0000 7fc742ffd640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc72c06a050 0x7fc72c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:41.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.760+0000 7fc742ffd640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc72c06a050 0x7fc72c06c510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc734006fd0 tx=0x7fc734008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:41.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.762+0000 7fc748ff9640 1 -- 172.21.15.27:0/556880575 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fc73c076ad0 con 0x7fc74c0b1420 2023-08-11T15:51:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.995+0000 7fc750b0f640 1 -- 172.21.15.27:0/556880575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc74c06bc60 con 0x7fc74c0b1420 2023-08-11T15:51:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.996+0000 7fc748ff9640 1 -- 172.21.15.27:0/556880575 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fc73c07a0f0 con 0x7fc74c0b1420 2023-08-11T15:51:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:41.998 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:51:01.220970+0000","modified":"2023-08-11T15:51:25.832166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4315":{"gid":4315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:51:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2023-08-11T15:51:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 -- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc72c06a050 msgr2=0x7fc72c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc72c06a050 0x7fc72c06c510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc734006fd0 tx=0x7fc734008040 comp rx=0 tx=0).stop 2023-08-11T15:51:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 -- 172.21.15.27:0/556880575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c0b1420 msgr2=0x7fc74c146340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c0b1420 0x7fc74c146340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc73c000f80 tx=0x7fc73c004a40 comp rx=0 tx=0).stop 2023-08-11T15:51:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 -- 172.21.15.27:0/556880575 shutdown_connections 2023-08-11T15:51:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc72c06a050 0x7fc72c06c510 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc74c15da60 0x7fc74c1f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc74c0cee90 0x7fc74c146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 --2- 172.21.15.27:0/556880575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc74c0b1420 0x7fc74c146340 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 -- 172.21.15.27:0/556880575 >> 172.21.15.27:0/556880575 conn(0x7fc74c0d0010 msgr2=0x7fc74c15c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:42.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 -- 172.21.15.27:0/556880575 shutdown_connections 2023-08-11T15:51:42.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:41.999+0000 7fc750b0f640 1 -- 172.21.15.27:0/556880575 wait complete. 2023-08-11T15:51:42.421 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-11T15:51:01.220970+0000', 'modified': '2023-08-11T15:51:25.832166+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4315}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4315': {'gid': 4315, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6835/2813789236', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 2813789236}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 2813789236}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-11T15:51:42.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-11T15:51:42.422 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-11T15:51:42.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.672+0000 7f46f89b2640 1 Processor -- start 2023-08-11T15:51:42.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.673+0000 7f46f89b2640 1 -- start start 2023-08-11T15:51:42.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.675+0000 7f46f89b2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f46f40acec0 0x7f46f40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.675+0000 7f46f89b2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46f4070a10 0x7f46f40ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.675+0000 7f46f89b2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f415dab0 con 0x7f46f415c680 2023-08-11T15:51:42.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.675+0000 7f46f89b2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f4070ea0 con 0x7f46f40acec0 2023-08-11T15:51:42.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.675+0000 7f46f89b2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40ad320 con 0x7f46f4070a10 2023-08-11T15:51:42.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.675+0000 7f46f37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f46f415c680 0x7f46f40ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48504/0 (socket says 172.21.15.27:48504) 2023-08-11T15:51:42.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.675+0000 7f46f37fe640 1 -- 172.21.15.27:0/3769657276 learned_addr learned my addr 172.21.15.27:0/3769657276 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:42.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.675+0000 7f46f3fff640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46f4070a10 0x7f46f40ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46eaffd640 1 --2- 172.21.15.27:0/3769657276 >> v2:172.21.15.119:3300/0 conn(0x7f46f40acec0 0x7f46f40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028326612 0 0) 0x7f46f415dab0 con 0x7f46f415c680 2023-08-11T15:51:42.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46e0003520 con 0x7f46f415c680 2023-08-11T15:51:42.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1323174915 0 0) 0x7f46e0003520 con 0x7f46f415c680 2023-08-11T15:51:42.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46f4070a10 msgr2=0x7f46f40ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46f4070a10 0x7f46f40ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 >> v2:172.21.15.119:3300/0 conn(0x7f46f40acec0 msgr2=0x7f46f40704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 --2- 172.21.15.27:0/3769657276 >> v2:172.21.15.119:3300/0 conn(0x7f46f40acec0 0x7f46f40704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f415cbc0 con 0x7f46f415c680 2023-08-11T15:51:42.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f46e4001670 con 0x7f46f415c680 2023-08-11T15:51:42.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 >> v1:172.21.15.27:6789/0 conn(0x7f46f415c680 legacy=0x7f46f40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e0003970 0x7f46e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46f4070a10 0x7f46e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.676+0000 7f46f2ffd640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e0004d00 0x7f46e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4070ea0 con 0x7f46e0004d00 2023-08-11T15:51:42.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40ad320 con 0x7f46e0003970 2023-08-11T15:51:42.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e000b980 con 0x7f46f4070a10 2023-08-11T15:51:42.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46eaffd640 1 --2- 172.21.15.27:0/3769657276 >> v2:172.21.15.119:3300/0 conn(0x7f46f40acec0 0x7f46f40704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:42.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f3fff640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e0004d00 0x7f46e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f37fe640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e0003970 0x7f46e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f3fff640 1 -- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46f4070a10 msgr2=0x7f46e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46eaffd640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46f4070a10 0x7f46e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f3fff640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46f4070a10 0x7f46e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f3fff640 1 -- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e0003970 msgr2=0x7f46e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f3fff640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e0003970 0x7f46e0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f3fff640 1 -- 172.21.15.27:0/3769657276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e0003520 con 0x7f46e0004d00 2023-08-11T15:51:42.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46f37fe640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e0003970 0x7f46e0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:42.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.677+0000 7f46eaffd640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46f4070a10 0x7f46e00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:42.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.679+0000 7f46f3fff640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e0004d00 0x7f46e0005180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00e170 tx=0x7f46e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f565e8da15b1561 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:42.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.679+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec02cc20 con 0x7f46e0004d00 2023-08-11T15:51:42.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.679+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec00f040 con 0x7f46e0004d00 2023-08-11T15:51:42.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.679+0000 7f46f2ffd640 1 -- 172.21.15.27:0/3769657276 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec034440 con 0x7f46e0004d00 2023-08-11T15:51:42.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.679+0000 7f46f89b2640 1 -- 172.21.15.27:0/3769657276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e0004d00 msgr2=0x7f46e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.679+0000 7f46f89b2640 1 --2- 172.21.15.27:0/3769657276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e0004d00 0x7f46e0005180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00e170 tx=0x7f46e00057f0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.679+0000 7f46f3fff640 1 -- 172.21.15.27:0/3769657276 reap_dead start 2023-08-11T15:51:42.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.680+0000 7f46f89b2640 1 -- 172.21.15.27:0/3769657276 shutdown_connections 2023-08-11T15:51:42.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.680+0000 7f46f89b2640 1 -- 172.21.15.27:0/3769657276 >> 172.21.15.27:0/3769657276 conn(0x7f46f40ceef0 msgr2=0x7f46f40cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:42.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.680+0000 7f46f89b2640 1 -- 172.21.15.27:0/3769657276 shutdown_connections 2023-08-11T15:51:42.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.680+0000 7f46f89b2640 1 -- 172.21.15.27:0/3769657276 wait complete. 2023-08-11T15:51:42.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.682+0000 7f46f89b2640 1 Processor -- start 2023-08-11T15:51:42.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.682+0000 7f46f89b2640 1 -- start start 2023-08-11T15:51:42.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.683+0000 7f46f89b2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46e0004d00 0x7f46f414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.683+0000 7f46f89b2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46f40ad7c0 0x7f46f40adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.683+0000 7f46f89b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46f40aea30 0x7f46f40aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.683+0000 7f46f89b2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40ae180 con 0x7f46f40ad7c0 2023-08-11T15:51:42.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.683+0000 7f46f89b2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f41f31a0 con 0x7f46f40aea30 2023-08-11T15:51:42.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.683+0000 7f46f89b2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f41f3480 con 0x7f46e0004d00 2023-08-11T15:51:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.683+0000 7f46eaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46f40ad7c0 0x7f46f40adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.683+0000 7f46eaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46f40ad7c0 0x7f46f40adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50292/0 (socket says 172.21.15.27:50292) 2023-08-11T15:51:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.683+0000 7f46eaffd640 1 -- 172.21.15.27:0/2759887474 learned_addr learned my addr 172.21.15.27:0/2759887474 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46f37fe640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46e0004d00 0x7f46f414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46f3fff640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46f40aea30 0x7f46f40aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46eaffd640 1 -- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46e0004d00 msgr2=0x7f46f414e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46eaffd640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46e0004d00 0x7f46f414e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46eaffd640 1 -- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46f40aea30 msgr2=0x7f46f40aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46eaffd640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46f40aea30 0x7f46f40aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46eaffd640 1 -- 172.21.15.27:0/2759887474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f41f3760 con 0x7f46f40ad7c0 2023-08-11T15:51:42.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46eaffd640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46f40ad7c0 0x7f46f40adc40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f46dc010c00 tx=0x7f46dc00e540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:42.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46f0ff9640 1 -- 172.21.15.27:0/2759887474 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc00ee50 con 0x7f46f40ad7c0 2023-08-11T15:51:42.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46f0ff9640 1 -- 172.21.15.27:0/2759887474 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc01c400 con 0x7f46f40ad7c0 2023-08-11T15:51:42.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46f0ff9640 1 -- 172.21.15.27:0/2759887474 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc01c770 con 0x7f46f40ad7c0 2023-08-11T15:51:42.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.684+0000 7f46f89b2640 1 -- 172.21.15.27:0/2759887474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f41f3a40 con 0x7f46f40ad7c0 2023-08-11T15:51:42.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.685+0000 7f46f89b2640 1 -- 172.21.15.27:0/2759887474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f406d770 con 0x7f46f40ad7c0 2023-08-11T15:51:42.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.688+0000 7f46f0ff9640 1 -- 172.21.15.27:0/2759887474 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f46dc01c8d0 con 0x7f46f40ad7c0 2023-08-11T15:51:42.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.689+0000 7f46f0ff9640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46d406a050 0x7f46d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.689+0000 7f46f0ff9640 1 -- 172.21.15.27:0/2759887474 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f46dc089030 con 0x7f46f40ad7c0 2023-08-11T15:51:42.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.690+0000 7f46fc78d740 1 Processor -- start 2023-08-11T15:51:42.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.690+0000 7f46f37fe640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46d406a050 0x7f46d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.690+0000 7f46fc78d740 1 -- start start 2023-08-11T15:51:42.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de60880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de60dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de5deb0 0x562d6de345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46fc78d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6dda2ef0 con 0x562d6de7c2c0 2023-08-11T15:51:42.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46fc78d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6dda05e0 con 0x562d6de7e7a0 2023-08-11T15:51:42.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46fc78d740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d6dd9bed0 con 0x562d6de5deb0 2023-08-11T15:51:42.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46eaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de60dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46eaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de60dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50304/0 (socket says 172.21.15.27:50304) 2023-08-11T15:51:42.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46eaffd640 1 -- 172.21.15.27:0/127261665 learned_addr learned my addr 172.21.15.27:0/127261665 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:42.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46f37fe640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46d406a050 0x7f46d406c510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f46e4001eb0 tx=0x7f46e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:42.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46f37fe640 1 --2- 172.21.15.27:0/127261665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de60880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46f3fff640 1 --2- 172.21.15.27:0/127261665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de5deb0 0x562d6de345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46eaffd640 1 -- 172.21.15.27:0/127261665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de5deb0 msgr2=0x562d6de345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46eaffd640 1 --2- 172.21.15.27:0/127261665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de5deb0 0x562d6de345f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46eaffd640 1 -- 172.21.15.27:0/127261665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 msgr2=0x562d6de60880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46eaffd640 1 --2- 172.21.15.27:0/127261665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de60880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.691+0000 7f46eaffd640 1 -- 172.21.15.27:0/127261665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d6dda0b40 con 0x562d6de7c2c0 2023-08-11T15:51:42.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.692+0000 7f46f37fe640 1 --2- 172.21.15.27:0/127261665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de60880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:42.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.692+0000 7f46eaffd640 1 --2- 172.21.15.27:0/127261665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de60dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0545b0 tx=0x7f46dc058800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8aabccd6035507 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.693+0000 7f46e97fa640 1 -- 172.21.15.27:0/127261665 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc010070 con 0x562d6de7c2c0 2023-08-11T15:51:42.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.693+0000 7f46e97fa640 1 -- 172.21.15.27:0/127261665 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc056450 con 0x562d6de7c2c0 2023-08-11T15:51:42.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.693+0000 7f46e97fa640 1 -- 172.21.15.27:0/127261665 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc06f080 con 0x562d6de7c2c0 2023-08-11T15:51:42.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.693+0000 7f46fc78d740 1 -- 172.21.15.27:0/127261665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 msgr2=0x562d6de60dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.693+0000 7f46fc78d740 1 --2- 172.21.15.27:0/127261665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de60dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0545b0 tx=0x7f46dc058800 comp rx=0 tx=0).stop 2023-08-11T15:51:42.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.693+0000 7f46fc78d740 1 -- 172.21.15.27:0/127261665 shutdown_connections 2023-08-11T15:51:42.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.693+0000 7f46fc78d740 1 --2- 172.21.15.27:0/127261665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de5deb0 0x562d6de345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.693+0000 7f46fc78d740 1 --2- 172.21.15.27:0/127261665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de60dc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.693+0000 7f46fc78d740 1 --2- 172.21.15.27:0/127261665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de60880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.693+0000 7f46fc78d740 1 -- 172.21.15.27:0/127261665 >> 172.21.15.27:0/127261665 conn(0x562d6de32d00 msgr2=0x562d6de7b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.694+0000 7f46fc78d740 1 -- 172.21.15.27:0/127261665 shutdown_connections 2023-08-11T15:51:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.694+0000 7f46fc78d740 1 -- 172.21.15.27:0/127261665 wait complete. 2023-08-11T15:51:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.694+0000 7f46fc78d740 1 -- start start 2023-08-11T15:51:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.695+0000 7f46fc78d740 1 Processor -- start 2023-08-11T15:51:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de62f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de59ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de64610 0x562d6de75ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46fc78d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6dd3bdc0 con 0x562d6de7c2c0 2023-08-11T15:51:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46fc78d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6dd3ba60 con 0x562d6de7e7a0 2023-08-11T15:51:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46fc78d740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d6db01f20 con 0x562d6de64610 2023-08-11T15:51:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46fc78d740 10 client.0 _ll_register_callbacks cb 0x562d6dd93d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de62f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de62f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50312/0 (socket says 172.21.15.27:50312) 2023-08-11T15:51:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46f37fe640 1 -- 172.21.15.27:0/3928495277 learned_addr learned my addr 172.21.15.27:0/3928495277 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46eaffd640 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de59ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.696+0000 7f46f3fff640 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de64610 0x562d6de75ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.697+0000 7f46f37fe640 1 -- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de64610 msgr2=0x562d6de75ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.697+0000 7f46f37fe640 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de64610 0x562d6de75ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.697+0000 7f46f37fe640 1 -- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 msgr2=0x562d6de59ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.697+0000 7f46f37fe640 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de59ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.697+0000 7f46f37fe640 1 -- 172.21.15.27:0/3928495277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f406d770 con 0x562d6de7c2c0 2023-08-11T15:51:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.697+0000 7f46f37fe640 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de62f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f46e4013a20 tx=0x7f46e4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75a5ade1ca19d9e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:42.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.697+0000 7f46d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-08-11T15:51:42.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.697+0000 7f46d27fc640 1 -- 172.21.15.27:0/3928495277 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4025070 con 0x562d6de7c2c0 2023-08-11T15:51:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.697+0000 7f46d27fc640 1 -- 172.21.15.27:0/3928495277 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f46e401b040 con 0x562d6de7c2c0 2023-08-11T15:51:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.697+0000 7f46fc78d740 1 -- 172.21.15.27:0/3928495277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d6dd73cf0 con 0x562d6de7c2c0 2023-08-11T15:51:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46d27fc640 1 -- 172.21.15.27:0/3928495277 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46e4012ae0 con 0x562d6de7c2c0 2023-08-11T15:51:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46fc78d740 10 client.4860 fetch_fsmap learned FSMap version 9 2023-08-11T15:51:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46fc78d740 1 -- 172.21.15.27:0/3928495277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x562d6dd9a550 con 0x562d6de7c2c0 2023-08-11T15:51:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46d27fc640 1 -- 172.21.15.27:0/3928495277 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562d6dd73cf0 con 0x562d6de7c2c0 2023-08-11T15:51:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46d27fc640 1 -- 172.21.15.27:0/3928495277 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f46e4012070 con 0x562d6de7c2c0 2023-08-11T15:51:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46fc78d740 10 client.4860 fetch_fsmap finished waiting for FSMap version 9 2023-08-11T15:51:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46fc78d740 10 client.4860 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T15:51:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46fc78d740 20 client.4860 populate_metadata read hostname 'smithi027' 2023-08-11T15:51:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46fc78d740 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x562d6de5c300 0x562d6de5c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46fc78d740 4 client.4860 mds_command: new command op to 4315 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:51:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.698+0000 7f46fc78d740 1 -- 172.21.15.27:0/3928495277 --> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d6dd70c60 con 0x562d6de5c300 2023-08-11T15:51:42.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.699+0000 7f46eaffd640 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x562d6de5c300 0x562d6de5c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.699+0000 7f46eaffd640 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x562d6de5c300 0x562d6de5c6e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233201f9c377e605 server_cookie=97e78b976561d83 in_seq=0 out_seq=0 2023-08-11T15:51:42.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.699+0000 7f46d27fc640 10 client.4860 ms_handle_connect on v2:172.21.15.119:6834/2813789236 2023-08-11T15:51:42.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.701+0000 7f46d27fc640 1 -- 172.21.15.27:0/3928495277 <== mds.0 v2:172.21.15.119:6834/2813789236 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x562d6dd70c60 con 0x562d6de5c300 2023-08-11T15:51:42.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.701+0000 7f46d27fc640 10 client.4860 handle_command_reply: tid=0 2023-08-11T15:51:42.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.701+0000 7f46fc78d740 1 client.4860 shutdown 2023-08-11T15:51:42.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.701+0000 7f46fc78d740 1 -- 172.21.15.27:0/3928495277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 msgr2=0x562d6de62f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.701+0000 7f46fc78d740 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de62f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f46e4013a20 tx=0x7f46e4001a00 comp rx=0 tx=0).stop 2023-08-11T15:51:42.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.701+0000 7f46fc78d740 1 -- 172.21.15.27:0/3928495277 shutdown_connections 2023-08-11T15:51:42.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.701+0000 7f46fc78d740 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x562d6de5c300 0x562d6de5c6e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.702+0000 7f46fc78d740 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de64610 0x562d6de75ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.702+0000 7f46fc78d740 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de59ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.702+0000 7f46fc78d740 1 --2- 172.21.15.27:0/3928495277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de7c2c0 0x562d6de62f00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.702+0000 7f46fc78d740 1 -- 172.21.15.27:0/3928495277 >> 172.21.15.27:0/3928495277 conn(0x562d6de5deb0 msgr2=0x562d6de608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:42.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.702+0000 7f46d27fc640 0 client.4860 ms_handle_reset on v2:172.21.15.119:6834/2813789236 2023-08-11T15:51:42.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.702+0000 7f46fc78d740 1 -- 172.21.15.27:0/3928495277 shutdown_connections 2023-08-11T15:51:42.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.702+0000 7f46fc78d740 1 -- 172.21.15.27:0/3928495277 wait complete. 2023-08-11T15:51:42.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.702+0000 7f46fc78d740 20 client.4860 trim_cache size 0 max 16384 2023-08-11T15:51:42.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.719+0000 7f46fc78d740 1 Processor -- start 2023-08-11T15:51:42.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.719+0000 7f46fc78d740 1 -- start start 2023-08-11T15:51:42.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.720+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de8f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.720+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de7c2c0 0x562d6de5c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.720+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de59ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.720+0000 7f46fc78d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6ddcd6c0 con 0x562d6de64610 2023-08-11T15:51:42.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.720+0000 7f46fc78d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6de311d0 con 0x562d6de7e7a0 2023-08-11T15:51:42.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.720+0000 7f46fc78d740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d6de93aa0 con 0x562d6de7c2c0 2023-08-11T15:51:42.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.720+0000 7f46f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de8f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.720+0000 7f46f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de8f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50324/0 (socket says 172.21.15.27:50324) 2023-08-11T15:51:42.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.720+0000 7f46f37fe640 1 -- 172.21.15.27:0/922486286 learned_addr learned my addr 172.21.15.27:0/922486286 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:42.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.721+0000 7f46f37fe640 1 -- 172.21.15.27:0/922486286 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de7c2c0 msgr2=0x562d6de5c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.721+0000 7f46f37fe640 1 --2- 172.21.15.27:0/922486286 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de7c2c0 0x562d6de5c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.721+0000 7f46f3fff640 1 --2- 172.21.15.27:0/922486286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de59ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.721+0000 7f46f37fe640 1 -- 172.21.15.27:0/922486286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 msgr2=0x562d6de59ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.721+0000 7f46f37fe640 1 --2- 172.21.15.27:0/922486286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de59ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.721+0000 7f46f37fe640 1 -- 172.21.15.27:0/922486286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d6ddcd9b0 con 0x562d6de64610 2023-08-11T15:51:42.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.721+0000 7f46f3fff640 1 --2- 172.21.15.27:0/922486286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de59ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:42.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.721+0000 7f46f37fe640 1 --2- 172.21.15.27:0/922486286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de8f5f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f46e40139f0 tx=0x7f46e4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f9e444da0dc0bd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:42.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.721+0000 7f46e9ffb640 1 -- 172.21.15.27:0/922486286 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4025070 con 0x562d6de64610 2023-08-11T15:51:42.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46e9ffb640 1 -- 172.21.15.27:0/922486286 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e4026040 con 0x562d6de64610 2023-08-11T15:51:42.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46e9ffb640 1 -- 172.21.15.27:0/922486286 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4012ae0 con 0x562d6de64610 2023-08-11T15:51:42.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46fc78d740 1 -- 172.21.15.27:0/922486286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 msgr2=0x562d6de8f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46fc78d740 1 --2- 172.21.15.27:0/922486286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de8f5f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f46e40139f0 tx=0x7f46e4015000 comp rx=0 tx=0).stop 2023-08-11T15:51:42.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46fc78d740 1 -- 172.21.15.27:0/922486286 shutdown_connections 2023-08-11T15:51:42.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46fc78d740 1 --2- 172.21.15.27:0/922486286 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de7c2c0 0x562d6de5c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46fc78d740 1 --2- 172.21.15.27:0/922486286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de59ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46fc78d740 1 --2- 172.21.15.27:0/922486286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de8f5f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46fc78d740 1 -- 172.21.15.27:0/922486286 >> 172.21.15.27:0/922486286 conn(0x562d6de5deb0 msgr2=0x562d6de757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:42.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46fc78d740 1 -- 172.21.15.27:0/922486286 shutdown_connections 2023-08-11T15:51:42.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.722+0000 7f46fc78d740 1 -- 172.21.15.27:0/922486286 wait complete. 2023-08-11T15:51:42.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.723+0000 7f46fc78d740 1 -- start start 2023-08-11T15:51:42.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.723+0000 7f46fc78d740 1 Processor -- start 2023-08-11T15:51:42.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.724+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de301d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.724+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de7c2c0 0x562d6de7eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.724+0000 7f46fc78d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de7f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.724+0000 7f46fc78d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6dee9ee0 con 0x562d6de64610 2023-08-11T15:51:42.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.724+0000 7f46fc78d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6de8edc0 con 0x562d6de7e7a0 2023-08-11T15:51:42.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.724+0000 7f46fc78d740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d6de2fd40 con 0x562d6de7c2c0 2023-08-11T15:51:42.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.724+0000 7f46fc78d740 10 client.0 _ll_register_callbacks cb 0x7f46dc010f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:42.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de301d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50334/0 (socket says 172.21.15.27:50334) 2023-08-11T15:51:42.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46f37fe640 1 -- 172.21.15.27:0/1808866127 learned_addr learned my addr 172.21.15.27:0/1808866127 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:42.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46f3fff640 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de7f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46eaffd640 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de7c2c0 0x562d6de7eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46f37fe640 1 -- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de7c2c0 msgr2=0x562d6de7eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46f37fe640 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de7c2c0 0x562d6de7eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46f37fe640 1 -- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 msgr2=0x562d6de7f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46f37fe640 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de7f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46f37fe640 1 -- 172.21.15.27:0/1808866127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d6dd9a550 con 0x562d6de64610 2023-08-11T15:51:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.725+0000 7f46eaffd640 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de7c2c0 0x562d6de7eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.726+0000 7f46f37fe640 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de301d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f46e4005a80 tx=0x7f46e401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777e6fd43dc7d3be server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.726+0000 7f46d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-08-11T15:51:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.726+0000 7f46d27fc640 1 -- 172.21.15.27:0/1808866127 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4025070 con 0x562d6de64610 2023-08-11T15:51:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.726+0000 7f46d27fc640 1 -- 172.21.15.27:0/1808866127 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f46e4034860 con 0x562d6de64610 2023-08-11T15:51:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.726+0000 7f46fc78d740 1 -- 172.21.15.27:0/1808866127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d6ddc3eb0 con 0x562d6de64610 2023-08-11T15:51:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.726+0000 7f46d27fc640 1 -- 172.21.15.27:0/1808866127 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46e4012720 con 0x562d6de64610 2023-08-11T15:51:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.726+0000 7f46fc78d740 10 client.4866 fetch_fsmap learned FSMap version 9 2023-08-11T15:51:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.726+0000 7f46fc78d740 1 -- 172.21.15.27:0/1808866127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x562d6dd3aa90 con 0x562d6de64610 2023-08-11T15:51:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.726+0000 7f46d27fc640 1 -- 172.21.15.27:0/1808866127 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562d6ddc3eb0 con 0x562d6de64610 2023-08-11T15:51:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.727+0000 7f46d27fc640 1 -- 172.21.15.27:0/1808866127 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f46e4028070 con 0x562d6de64610 2023-08-11T15:51:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.727+0000 7f46fc78d740 10 client.4866 fetch_fsmap finished waiting for FSMap version 9 2023-08-11T15:51:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.727+0000 7f46fc78d740 10 client.4866 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T15:51:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.727+0000 7f46fc78d740 20 client.4866 populate_metadata read hostname 'smithi027' 2023-08-11T15:51:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.727+0000 7f46fc78d740 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x562d6de59510 0x562d6dee9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.727+0000 7f46fc78d740 4 client.4866 mds_command: new command op to 4315 tid=0[{"prefix": "status"}] 2023-08-11T15:51:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.727+0000 7f46fc78d740 1 -- 172.21.15.27:0/1808866127 --> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d6dd94eb0 con 0x562d6de59510 2023-08-11T15:51:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.727+0000 7f46eaffd640 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x562d6de59510 0x562d6dee9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.728+0000 7f46eaffd640 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x562d6de59510 0x562d6dee9370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23617dd71d940cbd server_cookie=9973c7d31abb6696 in_seq=0 out_seq=0 2023-08-11T15:51:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.728+0000 7f46d27fc640 10 client.4866 ms_handle_connect on v2:172.21.15.119:6834/2813789236 2023-08-11T15:51:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46d27fc640 1 -- 172.21.15.27:0/1808866127 <== mds.0 v2:172.21.15.119:6834/2813789236 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x562d6dd94eb0 con 0x562d6de59510 2023-08-11T15:51:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46d27fc640 10 client.4866 handle_command_reply: tid=0 2023-08-11T15:51:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 client.4866 shutdown 2023-08-11T15:51:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 -- 172.21.15.27:0/1808866127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 msgr2=0x562d6de301d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de301d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f46e4005a80 tx=0x7f46e401f000 comp rx=0 tx=0).stop 2023-08-11T15:51:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 -- 172.21.15.27:0/1808866127 shutdown_connections 2023-08-11T15:51:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:6834/2813789236,v1:172.21.15.119:6835/2813789236] conn(0x562d6de59510 0x562d6dee9370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x562d6de7c2c0 0x562d6de7eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d6de7e7a0 0x562d6de7f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 --2- 172.21.15.27:0/1808866127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d6de64610 0x562d6de301d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46d27fc640 0 client.4866 ms_handle_reset on v2:172.21.15.119:6834/2813789236 2023-08-11T15:51:42.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 -- 172.21.15.27:0/1808866127 >> 172.21.15.27:0/1808866127 conn(0x562d6de5deb0 msgr2=0x562d6de758d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:42.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 -- 172.21.15.27:0/1808866127 shutdown_connections 2023-08-11T15:51:42.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 1 -- 172.21.15.27:0/1808866127 wait complete. 2023-08-11T15:51:42.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.729+0000 7f46fc78d740 20 client.4866 trim_cache size 0 max 16384 2023-08-11T15:51:42.737 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:51:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "bee60691-4c46-4397-a560-d178aa4db950", 2023-08-11T15:51:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-08-11T15:51:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4315, 2023-08-11T15:51:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-08-11T15:51:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-08-11T15:51:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-08-11T15:51:42.738 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 40.975437634000002, 2023-08-11T15:51:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 9, 2023-08-11T15:51:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 30, 2023-08-11T15:51:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 28, 2023-08-11T15:51:42.739 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 47.853580944999997 2023-08-11T15:51:42.739 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:51:42.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 -- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46d406a050 msgr2=0x7f46d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46d406a050 0x7f46d406c510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f46e4001eb0 tx=0x7f46e4009040 comp rx=0 tx=0).stop 2023-08-11T15:51:42.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 -- 172.21.15.27:0/2759887474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46f40ad7c0 msgr2=0x7f46f40adc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:42.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46f40ad7c0 0x7f46f40adc40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f46dc010c00 tx=0x7f46dc00e540 comp rx=0 tx=0).stop 2023-08-11T15:51:42.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 -- 172.21.15.27:0/2759887474 shutdown_connections 2023-08-11T15:51:42.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f46d406a050 0x7f46d406c510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46f40aea30 0x7f46f40aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46f40ad7c0 0x7f46f40adc40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 --2- 172.21.15.27:0/2759887474 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46e0004d00 0x7f46f414e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:42.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 -- 172.21.15.27:0/2759887474 >> 172.21.15.27:0/2759887474 conn(0x7f46f40ceef0 msgr2=0x7f46f40ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:42.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 -- 172.21.15.27:0/2759887474 shutdown_connections 2023-08-11T15:51:42.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:42.730+0000 7f46f89b2640 1 -- 172.21.15.27:0/2759887474 wait complete. 2023-08-11T15:51:42.749 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-08-11T15:51:42.759 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-08-11T15:51:42.759 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-08-11T15:51:42.759 INFO:tasks.ceph_fuse:client.0 config is {} 2023-08-11T15:51:42.760 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-11T15:51:42.760 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-11T15:51:42.760 INFO:tasks.ceph_fuse:client.2 config is {} 2023-08-11T15:51:42.760 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-11T15:51:42.761 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-11T15:51:42.761 INFO:tasks.ceph_fuse:client.1 config is {} 2023-08-11T15:51:42.761 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-11T15:51:42.761 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-11T15:51:42.761 INFO:tasks.ceph_fuse:client.3 config is {} 2023-08-11T15:51:42.762 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-11T15:51:42.762 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-11T15:51:42.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:42.762 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-08-11T15:51:42.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:42.825 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-08-11T15:51:42.928 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2023-08-11T15:51:42.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:51:42.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:42.933 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-08-11T15:51:42.998 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-08-11T15:51:43.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:43.000 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-11T15:51:43.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:43.113 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2023-08-11T15:51:43.247 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-08-11T15:51:43.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-11T15:51:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.453+0000 7f2239e98640 1 Processor -- start 2023-08-11T15:51:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.453+0000 7f2239e98640 1 -- start start 2023-08-11T15:51:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.455+0000 7f2239e98640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22340cc280 0x7f22340caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.455+0000 7f2239e98640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22340ad2a0 0x7f22340c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.455+0000 7f2239e98640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223406b910 con 0x7f223415c520 2023-08-11T15:51:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.455+0000 7f2239e98640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223406bbf0 con 0x7f22340ad2a0 2023-08-11T15:51:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.455+0000 7f2239e98640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223406bed0 con 0x7f22340cc280 2023-08-11T15:51:43.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.456+0000 7f2239697640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22340ad2a0 0x7f22340c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:43.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.456+0000 7f2239697640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22340ad2a0 0x7f22340c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50338/0 (socket says 172.21.15.27:50338) 2023-08-11T15:51:43.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.456+0000 7f2239697640 1 -- 172.21.15.27:0/994219286 learned_addr learned my addr 172.21.15.27:0/994219286 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:43.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.456+0000 7f222bfff640 1 --2- 172.21.15.27:0/994219286 >> v2:172.21.15.119:3300/0 conn(0x7f22340cc280 0x7f22340caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.456+0000 7f2239697640 1 -- 172.21.15.27:0/994219286 >> v2:172.21.15.119:3300/0 conn(0x7f22340cc280 msgr2=0x7f22340caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.456+0000 7f2239697640 1 --2- 172.21.15.27:0/994219286 >> v2:172.21.15.119:3300/0 conn(0x7f22340cc280 0x7f22340caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.456+0000 7f2239697640 1 -- 172.21.15.27:0/994219286 >> v1:172.21.15.119:6789/0 conn(0x7f223415c520 legacy=0x7f22340cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.456+0000 7f2239697640 1 -- 172.21.15.27:0/994219286 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223406c1b0 con 0x7f22340ad2a0 2023-08-11T15:51:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.456+0000 7f222bfff640 1 --2- 172.21.15.27:0/994219286 >> v2:172.21.15.119:3300/0 conn(0x7f22340cc280 0x7f22340caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.457+0000 7f2239697640 1 --2- 172.21.15.27:0/994219286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340ad2a0 0x7f22340c85f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f22340c9f00 tx=0x7f222c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e644e2e7d00b55b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.458+0000 7f222b7fe640 1 -- 172.21.15.27:0/994219286 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c031870 con 0x7f22340ad2a0 2023-08-11T15:51:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.458+0000 7f222b7fe640 1 -- 172.21.15.27:0/994219286 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c0319d0 con 0x7f22340ad2a0 2023-08-11T15:51:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.458+0000 7f222b7fe640 1 -- 172.21.15.27:0/994219286 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c031d60 con 0x7f22340ad2a0 2023-08-11T15:51:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.458+0000 7f2239e98640 1 -- 172.21.15.27:0/994219286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340ad2a0 msgr2=0x7f22340c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.458+0000 7f2239e98640 1 --2- 172.21.15.27:0/994219286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340ad2a0 0x7f22340c85f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f22340c9f00 tx=0x7f222c02b910 comp rx=0 tx=0).stop 2023-08-11T15:51:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.459+0000 7f2239e98640 1 -- 172.21.15.27:0/994219286 shutdown_connections 2023-08-11T15:51:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.459+0000 7f2239e98640 1 --2- 172.21.15.27:0/994219286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340ad2a0 0x7f22340c85f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.459+0000 7f2239e98640 1 --2- 172.21.15.27:0/994219286 >> v2:172.21.15.119:3300/0 conn(0x7f22340cc280 0x7f22340caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.459+0000 7f2239e98640 1 -- 172.21.15.27:0/994219286 >> 172.21.15.27:0/994219286 conn(0x7f22340afb70 msgr2=0x7f22340aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.459+0000 7f2239e98640 1 -- 172.21.15.27:0/994219286 shutdown_connections 2023-08-11T15:51:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.459+0000 7f2239e98640 1 -- 172.21.15.27:0/994219286 wait complete. 2023-08-11T15:51:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.461+0000 7f2239e98640 1 Processor -- start 2023-08-11T15:51:43.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.461+0000 7f2239e98640 1 -- start start 2023-08-11T15:51:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f2239e98640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22340ad2a0 0x7f2234155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f2239e98640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340cc280 0x7f22340c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f2239e98640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223415c520 0x7f22340c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f2239e98640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340c9400 con 0x7f22340cc280 2023-08-11T15:51:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f2239e98640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340c7840 con 0x7f223415c520 2023-08-11T15:51:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f2239e98640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22340c6fb0 con 0x7f22340ad2a0 2023-08-11T15:51:43.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f222bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340cc280 0x7f22340c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:43.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f222bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340cc280 0x7f22340c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50352/0 (socket says 172.21.15.27:50352) 2023-08-11T15:51:43.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f222bfff640 1 -- 172.21.15.27:0/843125305 learned_addr learned my addr 172.21.15.27:0/843125305 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:43.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f2238e96640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22340ad2a0 0x7f2234155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:43.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f2239697640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223415c520 0x7f22340c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:43.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f222bfff640 1 -- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22340ad2a0 msgr2=0x7f2234155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f222bfff640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22340ad2a0 0x7f2234155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f222bfff640 1 -- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223415c520 msgr2=0x7f22340c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.463+0000 7f222bfff640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223415c520 0x7f22340c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.464+0000 7f222bfff640 1 -- 172.21.15.27:0/843125305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c0094d0 con 0x7f22340cc280 2023-08-11T15:51:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.464+0000 7f2239697640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223415c520 0x7f22340c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.464+0000 7f222bfff640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340cc280 0x7f22340c9890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f222400e810 tx=0x7f222400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.464+0000 7f22297fa640 1 -- 172.21.15.27:0/843125305 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224004030 con 0x7f22340cc280 2023-08-11T15:51:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.464+0000 7f22297fa640 1 -- 172.21.15.27:0/843125305 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2224004190 con 0x7f22340cc280 2023-08-11T15:51:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.464+0000 7f22297fa640 1 -- 172.21.15.27:0/843125305 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224004520 con 0x7f22340cc280 2023-08-11T15:51:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.464+0000 7f2239e98640 1 -- 172.21.15.27:0/843125305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22340c7280 con 0x7f22340cc280 2023-08-11T15:51:43.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.464+0000 7f2239e98640 1 -- 172.21.15.27:0/843125305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22340c9090 con 0x7f22340cc280 2023-08-11T15:51:43.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.466+0000 7f2239e98640 1 -- 172.21.15.27:0/843125305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22340c8ef0 con 0x7f22340cc280 2023-08-11T15:51:43.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.466+0000 7f22297fa640 1 -- 172.21.15.27:0/843125305 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f222400c6d0 con 0x7f22340cc280 2023-08-11T15:51:43.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.468+0000 7f22297fa640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f222006a050 0x7f222006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:43.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.468+0000 7f22297fa640 1 -- 172.21.15.27:0/843125305 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f22240884b0 con 0x7f22340cc280 2023-08-11T15:51:43.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.469+0000 7f2238e96640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f222006a050 0x7f222006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:43.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.470+0000 7f2238e96640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f222006a050 0x7f222006c510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22300012e0 tx=0x7f2230009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.472+0000 7f22297fa640 1 -- 172.21.15.27:0/843125305 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f22340c8ef0 con 0x7f22340cc280 2023-08-11T15:51:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.706+0000 7f2239e98640 1 -- 172.21.15.27:0/843125305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f22340c8ef0 con 0x7f22340cc280 2023-08-11T15:51:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.707+0000 7f22297fa640 1 -- 172.21.15.27:0/843125305 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f22340c8ef0 con 0x7f22340cc280 2023-08-11T15:51:43.709 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-11T15:51:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 -- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f222006a050 msgr2=0x7f222006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f222006a050 0x7f222006c510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22300012e0 tx=0x7f2230009040 comp rx=0 tx=0).stop 2023-08-11T15:51:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 -- 172.21.15.27:0/843125305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340cc280 msgr2=0x7f22340c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340cc280 0x7f22340c9890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f222400e810 tx=0x7f222400ecd0 comp rx=0 tx=0).stop 2023-08-11T15:51:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 -- 172.21.15.27:0/843125305 shutdown_connections 2023-08-11T15:51:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f222006a050 0x7f222006c510 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f223415c520 0x7f22340c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22340cc280 0x7f22340c9890 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 --2- 172.21.15.27:0/843125305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22340ad2a0 0x7f2234155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 -- 172.21.15.27:0/843125305 >> 172.21.15.27:0/843125305 conn(0x7f22340afb70 msgr2=0x7f22340cadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.710+0000 7f2239e98640 1 -- 172.21.15.27:0/843125305 shutdown_connections 2023-08-11T15:51:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:43.711+0000 7f2239e98640 1 -- 172.21.15.27:0/843125305 wait complete. 2023-08-11T15:51:43.731 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-11T15:51:43.732 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T15:51:43.732 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-08-11T15:51:43.732 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-11T15:51:43.733 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-11T15:51:43.733 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-11T15:51:43.733 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T15:51:43.734 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T15:51:43.734 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-11T15:51:43.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:43.735 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-08-11T15:51:43.795 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:51:43.795 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:51:43.795 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:51:43.795 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:43.796 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-08-11T15:51:43.796 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:43.796 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:43.796 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:43.797 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-08-11T15:51:43.797 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:43.797 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:43.797 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-08-11T15:51:43.798 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T15:51:43.798 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:43.798 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-08-11T15:51:43.798 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-11T15:51:43.798 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41997sec preferred_lft 41997sec 2023-08-11T15:51:43.799 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2023-08-11T15:51:43.799 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:43.799 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:43.799 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:43.800 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-08-11T15:51:43.800 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-11T15:51:43.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:43.801 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:43.801 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:43.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2023-08-11T15:51:43.802 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2023-08-11T15:51:43.802 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2023-08-11T15:51:43.802 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-11T15:51:43.802 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:44.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:44.059 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T15:51:44.152 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-08-11T15:51:44.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:44.157 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-08-11T15:51:44.216 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-08-11T15:51:44.216 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T15:51:44.216 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-11T15:51:44.216 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-11T15:51:44.216 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T15:51:44.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:44.217 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:44.217 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:44.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-11T15:51:44.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-11T15:51:44.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-11T15:51:44.218 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:44.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:44.435 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-08-11T15:51:44.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:44.496 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2023-08-11T15:51:44.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:44.553 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:44.553 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:44.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-11T15:51:44.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-08-11T15:51:44.554 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:44.724 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-08-11T15:51:44.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:44.725 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:44.725 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:44.726 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-08-11T15:51:44.726 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-11T15:51:44.726 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-08-11T15:51:44.726 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-08-11T15:51:44.727 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-08-11T15:51:44.727 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:45.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:45.027 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:45.027 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:45.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-08-11T15:51:45.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2023-08-11T15:51:45.028 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:45.207 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-11T15:51:45.207 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:51:45.207 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-11T15:51:45.263 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-11T15:51:45.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:51:45.263 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-11T15:51:45.321 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-08-11T15:51:45.431 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:45.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:45.545 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:45.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:45.546 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-08-11T15:51:45.602 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-11T15:51:45.603 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-11T15:51:45.611 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-08-11T15:51:45.716 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:45.716 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:45.810 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:45.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:45.811 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-08-11T15:51:45.834 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-08-11T15:51:45.833+0000 7fa5b6843480 -1 init, newargv = 0x561361af0a20 newargc=15 2023-08-11T15:51:45.835 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[99100]: starting ceph client 2023-08-11T15:51:45.853 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[99100]: starting fuse 2023-08-11T15:51:45.862 INFO:teuthology.orchestra.run.smithi027.stdout:39 2023-08-11T15:51:45.863 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-11T15:51:45.863 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2023-08-11T15:51:45.863 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-08-11T15:51:45.864 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-08-11T15:51:45.864 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-08-11T15:51:45.864 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-08-11T15:51:45.864 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:45.865 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-08-11T15:51:45.865 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-11T15:51:45.865 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-08-11T15:51:45.865 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-11T15:51:45.866 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:45.866 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-08-11T15:51:45.866 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-08-11T15:51:45.866 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-08-11T15:51:45.867 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-08-11T15:51:45.867 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:45.867 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-08-11T15:51:45.867 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-11T15:51:45.868 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-11T15:51:45.868 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-11T15:51:45.868 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-08-11T15:51:45.868 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-08-11T15:51:45.869 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-08-11T15:51:45.869 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-11T15:51:45.869 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:45.869 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-08-11T15:51:45.870 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-08-11T15:51:46.089 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.99100.asok 2023-08-11T15:51:46.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-11T15:51:46.108 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.99100.asok 2023-08-11T15:51:46.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:46.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.99100.asok status 2023-08-11T15:51:46.369 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:51:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-08-11T15:51:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "92684807f016ce045ac66e37fdde6f8c47857bcc", 2023-08-11T15:51:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:51:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-08-11T15:51:46.370 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-08-11T15:51:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-11T15:51:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "99100", 2023-08-11T15:51:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-08-11T15:51:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-08-11T15:51:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-08-11T15:51:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4878, 2023-08-11T15:51:46.371 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-08-11T15:51:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-08-11T15:51:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-08-11T15:51:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 4878 2023-08-11T15:51:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-08-11T15:51:46.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-08-11T15:51:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:51:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2081008354 2023-08-11T15:51:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:51:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-08-11T15:51:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-08-11T15:51:46.373 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:51:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2081008354 2023-08-11T15:51:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.4878 192.168.0.1:0/2081008354", 2023-08-11T15:51:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/2081008354", 2023-08-11T15:51:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-08-11T15:51:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 9, 2023-08-11T15:51:46.374 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 30, 2023-08-11T15:51:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-08-11T15:51:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-08-11T15:51:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-08-11T15:51:46.375 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:51:46.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-11T15:51:46.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.612+0000 7f77bb4fa640 1 Processor -- start 2023-08-11T15:51:46.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.613+0000 7f77bb4fa640 1 -- start start 2023-08-11T15:51:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.614+0000 7f77bb4fa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77b40cb0c0 0x7f77b415a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.614+0000 7f77bb4fa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f77b40cbb70 0x7f77b40708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.614+0000 7f77bb4fa640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b40cc020 con 0x7f77b4156e50 2023-08-11T15:51:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.614+0000 7f77bb4fa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b40ca460 con 0x7f77b40cbb70 2023-08-11T15:51:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.614+0000 7f77bb4fa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b4070470 con 0x7f77b40cb0c0 2023-08-11T15:51:46.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.615+0000 7f77b9a97640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f77b40cbb70 0x7f77b40708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:46.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.615+0000 7f77b9a97640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f77b40cbb70 0x7f77b40708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50354/0 (socket says 172.21.15.27:50354) 2023-08-11T15:51:46.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.615+0000 7f77b9a97640 1 -- 172.21.15.27:0/2817571198 learned_addr learned my addr 172.21.15.27:0/2817571198 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:46.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.615+0000 7f77b8a95640 1 --2- 172.21.15.27:0/2817571198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77b40cb0c0 0x7f77b415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:46.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.615+0000 7f77b9a97640 1 -- 172.21.15.27:0/2817571198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77b40cb0c0 msgr2=0x7f77b415a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:46.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.615+0000 7f77b9a97640 1 --2- 172.21.15.27:0/2817571198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77b40cb0c0 0x7f77b415a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.615+0000 7f77b9a97640 1 -- 172.21.15.27:0/2817571198 >> v1:172.21.15.119:6789/0 conn(0x7f77b4156e50 legacy=0x7f77b40ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:46.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.615+0000 7f77b9a97640 1 -- 172.21.15.27:0/2817571198 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b40cc270 con 0x7f77b40cbb70 2023-08-11T15:51:46.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.616+0000 7f77b9a97640 1 --2- 172.21.15.27:0/2817571198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 0x7f77b40708c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009e50 tx=0x7f77ac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf1c2db0a9bb5a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.617+0000 7f77abfff640 1 -- 172.21.15.27:0/2817571198 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac02ca70 con 0x7f77b40cbb70 2023-08-11T15:51:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.617+0000 7f77abfff640 1 -- 172.21.15.27:0/2817571198 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac02cbd0 con 0x7f77b40cbb70 2023-08-11T15:51:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.617+0000 7f77abfff640 1 -- 172.21.15.27:0/2817571198 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0343f0 con 0x7f77b40cbb70 2023-08-11T15:51:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.617+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2817571198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 msgr2=0x7f77b40708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.617+0000 7f77bb4fa640 1 --2- 172.21.15.27:0/2817571198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 0x7f77b40708c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009e50 tx=0x7f77ac02bed0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.618+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2817571198 shutdown_connections 2023-08-11T15:51:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.618+0000 7f77bb4fa640 1 --2- 172.21.15.27:0/2817571198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 0x7f77b40708c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.618+0000 7f77bb4fa640 1 --2- 172.21.15.27:0/2817571198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77b40cb0c0 0x7f77b415a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.618+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2817571198 >> 172.21.15.27:0/2817571198 conn(0x7f77b40afb70 msgr2=0x7f77b40aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:46.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.618+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2817571198 shutdown_connections 2023-08-11T15:51:46.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.618+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2817571198 wait complete. 2023-08-11T15:51:46.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.620+0000 7f77bb4fa640 1 Processor -- start 2023-08-11T15:51:46.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.620+0000 7f77bb4fa640 1 -- start start 2023-08-11T15:51:46.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77bb4fa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77b40cb0c0 0x7f77b414f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:46.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77bb4fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 0x7f77b40b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:46.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77bb4fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77b4156e50 0x7f77b4155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:46.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77bb4fa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40b24a0 con 0x7f77b40cbb70 2023-08-11T15:51:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77bb4fa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40b1560 con 0x7f77b4156e50 2023-08-11T15:51:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77bb4fa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40b1840 con 0x7f77b40cb0c0 2023-08-11T15:51:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77b8a95640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 0x7f77b40b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77b8a95640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 0x7f77b40b0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50356/0 (socket says 172.21.15.27:50356) 2023-08-11T15:51:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77b8a95640 1 -- 172.21.15.27:0/2961481936 learned_addr learned my addr 172.21.15.27:0/2961481936 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77b9296640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77b40cb0c0 0x7f77b414f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.621+0000 7f77b9a97640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77b4156e50 0x7f77b4155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.622+0000 7f77b8a95640 1 -- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77b40cb0c0 msgr2=0x7f77b414f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.622+0000 7f77b8a95640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77b40cb0c0 0x7f77b414f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.622+0000 7f77b8a95640 1 -- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77b4156e50 msgr2=0x7f77b4155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.622+0000 7f77b8a95640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77b4156e50 0x7f77b4155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.622+0000 7f77b8a95640 1 -- 172.21.15.27:0/2961481936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac009b30 con 0x7f77b40cbb70 2023-08-11T15:51:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.622+0000 7f77b8a95640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 0x7f77b40b0c60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f77a400d5f0 tx=0x7f77a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.622+0000 7f77a9ffb640 1 -- 172.21.15.27:0/2961481936 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a40125f0 con 0x7f77b40cbb70 2023-08-11T15:51:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.622+0000 7f77a9ffb640 1 -- 172.21.15.27:0/2961481936 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4012750 con 0x7f77b40cbb70 2023-08-11T15:51:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.622+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2961481936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b40b1b20 con 0x7f77b40cbb70 2023-08-11T15:51:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.622+0000 7f77a9ffb640 1 -- 172.21.15.27:0/2961481936 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4012b00 con 0x7f77b40cbb70 2023-08-11T15:51:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.623+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2961481936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b414fa70 con 0x7f77b40cbb70 2023-08-11T15:51:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.624+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2961481936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b40b2610 con 0x7f77b40cbb70 2023-08-11T15:51:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.625+0000 7f77a9ffb640 1 -- 172.21.15.27:0/2961481936 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f77a4012c60 con 0x7f77b40cbb70 2023-08-11T15:51:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.625+0000 7f77a9ffb640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f779806a050 0x7f779806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.626+0000 7f77a9ffb640 1 -- 172.21.15.27:0/2961481936 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f77a4088e20 con 0x7f77b40cbb70 2023-08-11T15:51:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.626+0000 7f77b9296640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f779806a050 0x7f779806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:46.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.627+0000 7f77b9296640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f779806a050 0x7f779806c510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f77b0004e50 tx=0x7f77b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:46.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.630+0000 7f77a9ffb640 1 -- 172.21.15.27:0/2961481936 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f77a4054140 con 0x7f77b40cbb70 2023-08-11T15:51:46.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.864+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2961481936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f77b4073990 con 0x7f77b40cbb70 2023-08-11T15:51:46.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.865+0000 7f77a9ffb640 1 -- 172.21.15.27:0/2961481936 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f77a4057760 con 0x7f77b40cbb70 2023-08-11T15:51:46.866 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-11T15:51:46.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.867+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f779806a050 msgr2=0x7f779806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.867+0000 7f77bb4fa640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f779806a050 0x7f779806c510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f77b0004e50 tx=0x7f77b0009040 comp rx=0 tx=0).stop 2023-08-11T15:51:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.867+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2961481936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 msgr2=0x7f77b40b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.867+0000 7f77bb4fa640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 0x7f77b40b0c60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f77a400d5f0 tx=0x7f77a400dab0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.867+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2961481936 shutdown_connections 2023-08-11T15:51:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.867+0000 7f77bb4fa640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f779806a050 0x7f779806c510 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.867+0000 7f77bb4fa640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77b4156e50 0x7f77b4155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.867+0000 7f77bb4fa640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b40cbb70 0x7f77b40b0c60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.867+0000 7f77bb4fa640 1 --2- 172.21.15.27:0/2961481936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77b40cb0c0 0x7f77b414f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.867+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2961481936 >> 172.21.15.27:0/2961481936 conn(0x7f77b40afb70 msgr2=0x7f77b4157290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.868+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2961481936 shutdown_connections 2023-08-11T15:51:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:46.868+0000 7f77bb4fa640 1 -- 172.21.15.27:0/2961481936 wait complete. 2023-08-11T15:51:46.886 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-11T15:51:46.887 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T15:51:46.887 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-08-11T15:51:46.887 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-08-11T15:51:46.887 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-11T15:51:46.887 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-11T15:51:46.888 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T15:51:46.888 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T15:51:46.888 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-08-11T15:51:46.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:46.889 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-08-11T15:51:46.945 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:51:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:51:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:51:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-08-11T15:51:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:46.947 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-08-11T15:51:46.948 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-08-11T15:51:46.948 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T15:51:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-08-11T15:51:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-11T15:51:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41993sec preferred_lft 41993sec 2023-08-11T15:51:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2023-08-11T15:51:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:46.949 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-08-11T15:51:46.949 INFO:teuthology.orchestra.run.smithi027.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-11T15:51:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 3e:d3:b3:c9:79:4a brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-11T15:51:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::b471:35ff:fe61:7a71/64 scope link 2023-08-11T15:51:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:46.950 INFO:teuthology.orchestra.run.smithi027.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-11T15:51:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 3e:d3:b3:c9:79:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-11T15:51:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::3cd3:b3ff:fec9:794a/64 scope link 2023-08-11T15:51:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:46.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:46.952 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T15:51:47.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:51:47.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:51:47.044 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-08-11T15:51:47.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:47.048 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-08-11T15:51:47.140 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-08-11T15:51:47.140 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T15:51:47.140 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-11T15:51:47.141 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-11T15:51:47.141 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T15:51:47.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:47.141 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:47.142 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:47.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-11T15:51:47.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-11T15:51:47.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-11T15:51:47.143 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:47.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:47.401 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-08-11T15:51:47.459 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-11T15:51:47.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:47.461 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2023-08-11T15:51:47.520 INFO:teuthology.orchestra.run.smithi027.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-08-11T15:51:47.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:47.521 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:47.521 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:47.522 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-11T15:51:47.522 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-08-11T15:51:47.522 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:47.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:47.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-11T15:51:47.802 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:51:47.803 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:51:47.803 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:51:47.803 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:47.803 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-08-11T15:51:47.803 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:47.804 INFO:teuthology.orchestra.run.smithi027.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-11T15:51:47.804 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 82:69:9c:ef:da:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-11T15:51:47.804 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-11T15:51:47.804 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:47.804 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8069:9cff:feef:daab/64 scope link 2023-08-11T15:51:47.804 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:47.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:47.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-11T15:51:47.923 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:51:47.923 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:51:47.923 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:51:47.923 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:47.923 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-08-11T15:51:47.924 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:47.924 INFO:teuthology.orchestra.run.smithi027.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-11T15:51:47.924 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 82:69:9c:ef:da:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-11T15:51:47.924 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-11T15:51:47.924 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:47.924 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8069:9cff:feef:daab/64 scope link 2023-08-11T15:51:47.924 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:47.930 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-08-11T15:51:47.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:47.930 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:47.931 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:47.931 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-08-11T15:51:47.931 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-11T15:51:47.931 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-08-11T15:51:47.932 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-08-11T15:51:47.932 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-08-11T15:51:47.932 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:48.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:48.257 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:51:48.257 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:51:48.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.1 up 2023-08-11T15:51:48.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.1 master ceph-brx 2023-08-11T15:51:48.258 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:51:48.446 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-08-11T15:51:48.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:51:48.446 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-08-11T15:51:48.501 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-08-11T15:51:48.502 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:51:48.502 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-08-11T15:51:48.556 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-08-11T15:51:48.670 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:48.670 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:48.780 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:48.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:48.781 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-08-11T15:51:48.839 INFO:teuthology.orchestra.run.smithi027.stdout:39 2023-08-11T15:51:48.840 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [39] 2023-08-11T15:51:48.840 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-08-11T15:51:48.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-08-11T15:51:48.930 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:48.931 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:49.023 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:49.026 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:49.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:49.027 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-08-11T15:51:49.057 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi027.stderr:2023-08-11T15:51:49.054+0000 7fb28f53e480 -1 init, newargv = 0x55ec2633b650 newargc=15 2023-08-11T15:51:49.057 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi027.stderr:ceph-fuse[99612]: starting ceph client 2023-08-11T15:51:49.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi027.stderr:ceph-fuse[99612]: starting fuse 2023-08-11T15:51:49.075 INFO:teuthology.orchestra.run.smithi027.stdout:39 2023-08-11T15:51:49.076 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-08-11T15:51:49.076 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39, 41] 2023-08-11T15:51:49.077 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2023-08-11T15:51:49.077 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-08-11T15:51:49.077 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-08-11T15:51:49.077 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-08-11T15:51:49.077 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-08-11T15:51:49.078 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:49.078 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-08-11T15:51:49.078 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-08-11T15:51:49.078 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-08-11T15:51:49.079 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-08-11T15:51:49.079 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:49.079 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-08-11T15:51:49.080 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-08-11T15:51:49.080 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-08-11T15:51:49.080 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-08-11T15:51:49.080 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:49.081 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-08-11T15:51:49.081 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-11T15:51:49.081 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-11T15:51:49.081 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-11T15:51:49.082 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-08-11T15:51:49.082 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-08-11T15:51:49.082 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-08-11T15:51:49.082 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-11T15:51:49.082 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:51:49.083 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.2")) 2023-08-11T15:51:49.083 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-08-11T15:51:49.305 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.2.99612.asok 2023-08-11T15:51:49.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-11T15:51:49.323 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.99612.asok 2023-08-11T15:51:49.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:49.324 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.99612.asok status 2023-08-11T15:51:49.588 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:51:49.588 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-08-11T15:51:49.589 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "92684807f016ce045ac66e37fdde6f8c47857bcc", 2023-08-11T15:51:49.589 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:51:49.589 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "2", 2023-08-11T15:51:49.589 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-08-11T15:51:49.589 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-08-11T15:51:49.589 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "99612", 2023-08-11T15:51:49.589 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-08-11T15:51:49.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:49.590 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-08-11T15:51:49.590 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-08-11T15:51:49.590 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4890, 2023-08-11T15:51:49.590 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-08-11T15:51:49.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-08-11T15:51:49.590 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-08-11T15:51:49.590 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 4890 2023-08-11T15:51:49.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:49.591 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-08-11T15:51:49.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-08-11T15:51:49.591 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.2:0", 2023-08-11T15:51:49.591 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3006833557 2023-08-11T15:51:49.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:51:49.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:49.592 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-08-11T15:51:49.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-08-11T15:51:49.592 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.2:0", 2023-08-11T15:51:49.592 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3006833557 2023-08-11T15:51:49.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:51:49.592 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.4890 192.168.0.2:0/3006833557", 2023-08-11T15:51:49.592 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.2:0/3006833557", 2023-08-11T15:51:49.592 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-08-11T15:51:49.593 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 9, 2023-08-11T15:51:49.593 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 30, 2023-08-11T15:51:49.593 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-08-11T15:51:49.593 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-08-11T15:51:49.593 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-08-11T15:51:49.593 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:51:49.603 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-11T15:51:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.842+0000 7f8cb759e640 1 Processor -- start 2023-08-11T15:51:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.842+0000 7f8cb759e640 1 -- start start 2023-08-11T15:51:49.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.844+0000 7f8cb759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8cb80c9600 0x7f8cb815e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.844+0000 7f8cb759e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb806f5d0 0x7f8cb806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.844+0000 7f8cb759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb806ff60 con 0x7f8cb815a020 2023-08-11T15:51:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.844+0000 7f8cb759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb815eae0 con 0x7f8cb80c9600 2023-08-11T15:51:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.844+0000 7f8cb759e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80c85f0 con 0x7f8cb806f5d0 2023-08-11T15:51:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.844+0000 7f8cb5d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8cb80c9600 0x7f8cb815e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.844+0000 7f8cb5d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8cb80c9600 0x7f8cb815e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43474/0 (socket says 172.21.15.27:43474) 2023-08-11T15:51:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.844+0000 7f8cb5d9b640 1 -- 172.21.15.27:0/588035804 learned_addr learned my addr 172.21.15.27:0/588035804 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:49.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.845+0000 7f8cb6d9d640 1 --2- 172.21.15.27:0/588035804 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb806f5d0 0x7f8cb806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:49.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.845+0000 7f8cb5d9b640 1 -- 172.21.15.27:0/588035804 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb806f5d0 msgr2=0x7f8cb806fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:49.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.845+0000 7f8cb5d9b640 1 --2- 172.21.15.27:0/588035804 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb806f5d0 0x7f8cb806fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:49.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.845+0000 7f8cb5d9b640 1 -- 172.21.15.27:0/588035804 >> v1:172.21.15.27:6789/0 conn(0x7f8cb815a020 legacy=0x7f8cb8071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:49.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.845+0000 7f8cb5d9b640 1 -- 172.21.15.27:0/588035804 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb80c8840 con 0x7f8cb80c9600 2023-08-11T15:51:49.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.845+0000 7f8cb6d9d640 1 --2- 172.21.15.27:0/588035804 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb806f5d0 0x7f8cb806fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:49.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.846+0000 7f8cb5d9b640 1 --2- 172.21.15.27:0/588035804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb80c9600 0x7f8cb815e5a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009870 tx=0x7f8cac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cde56c2fead3968 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:49.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.846+0000 7f8cb559a640 1 -- 172.21.15.27:0/588035804 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac031640 con 0x7f8cb80c9600 2023-08-11T15:51:49.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.846+0000 7f8cb559a640 1 -- 172.21.15.27:0/588035804 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac0317a0 con 0x7f8cb80c9600 2023-08-11T15:51:49.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.846+0000 7f8cb559a640 1 -- 172.21.15.27:0/588035804 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac031b30 con 0x7f8cb80c9600 2023-08-11T15:51:49.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.847+0000 7f8cb759e640 1 -- 172.21.15.27:0/588035804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb80c9600 msgr2=0x7f8cb815e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.847+0000 7f8cb759e640 1 --2- 172.21.15.27:0/588035804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb80c9600 0x7f8cb815e5a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009870 tx=0x7f8cac02b8f0 comp rx=0 tx=0).stop 2023-08-11T15:51:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.847+0000 7f8cb759e640 1 -- 172.21.15.27:0/588035804 shutdown_connections 2023-08-11T15:51:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.847+0000 7f8cb759e640 1 --2- 172.21.15.27:0/588035804 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb806f5d0 0x7f8cb806fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.847+0000 7f8cb759e640 1 --2- 172.21.15.27:0/588035804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb80c9600 0x7f8cb815e5a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.847+0000 7f8cb759e640 1 -- 172.21.15.27:0/588035804 >> 172.21.15.27:0/588035804 conn(0x7f8cb80afb70 msgr2=0x7f8cb80aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.847+0000 7f8cb759e640 1 -- 172.21.15.27:0/588035804 shutdown_connections 2023-08-11T15:51:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.847+0000 7f8cb759e640 1 -- 172.21.15.27:0/588035804 wait complete. 2023-08-11T15:51:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.849+0000 7f8cb759e640 1 Processor -- start 2023-08-11T15:51:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.849+0000 7f8cb759e640 1 -- start start 2023-08-11T15:51:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.850+0000 7f8cb759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cb806f5d0 0x7f8cb8154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.850+0000 7f8cb759e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb80c9600 0x7f8cb80c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.850+0000 7f8cb759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb815a020 0x7f8cb815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.850+0000 7f8cb759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb8154c70 con 0x7f8cb815a020 2023-08-11T15:51:49.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.850+0000 7f8cb759e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80c2d10 con 0x7f8cb806f5d0 2023-08-11T15:51:49.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.850+0000 7f8cb759e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80c2ff0 con 0x7f8cb80c9600 2023-08-11T15:51:49.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.851+0000 7f8cb6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb815a020 0x7f8cb815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:49.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.851+0000 7f8cb6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb815a020 0x7f8cb815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43484/0 (socket says 172.21.15.27:43484) 2023-08-11T15:51:49.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.851+0000 7f8cb6d9d640 1 -- 172.21.15.27:0/3457456146 learned_addr learned my addr 172.21.15.27:0/3457456146 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:49.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.851+0000 7f8cb5d9b640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb80c9600 0x7f8cb80c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:49.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.851+0000 7f8cb659c640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cb806f5d0 0x7f8cb8154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:49.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.851+0000 7f8cb6d9d640 1 -- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb80c9600 msgr2=0x7f8cb80c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:49.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.851+0000 7f8cb6d9d640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb80c9600 0x7f8cb80c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:49.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.851+0000 7f8cb6d9d640 1 -- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cb806f5d0 msgr2=0x7f8cb8154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:49.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.851+0000 7f8cb6d9d640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cb806f5d0 0x7f8cb8154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:49.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.851+0000 7f8cb6d9d640 1 -- 172.21.15.27:0/3457456146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0094d0 con 0x7f8cb815a020 2023-08-11T15:51:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.852+0000 7f8cb6d9d640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb815a020 0x7f8cb815afb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000cd50 tx=0x7f8cb000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.852+0000 7f8ca6ffd640 1 -- 172.21.15.27:0/3457456146 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000f680 con 0x7f8cb815a020 2023-08-11T15:51:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.852+0000 7f8ca6ffd640 1 -- 172.21.15.27:0/3457456146 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb000f800 con 0x7f8cb815a020 2023-08-11T15:51:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.852+0000 7f8cb759e640 1 -- 172.21.15.27:0/3457456146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb80c32d0 con 0x7f8cb815a020 2023-08-11T15:51:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.852+0000 7f8ca6ffd640 1 -- 172.21.15.27:0/3457456146 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000fbb0 con 0x7f8cb815a020 2023-08-11T15:51:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.852+0000 7f8cb759e640 1 -- 172.21.15.27:0/3457456146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb80c3890 con 0x7f8cb815a020 2023-08-11T15:51:49.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.853+0000 7f8cb759e640 1 -- 172.21.15.27:0/3457456146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb806c850 con 0x7f8cb815a020 2023-08-11T15:51:49.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.854+0000 7f8ca6ffd640 1 -- 172.21.15.27:0/3457456146 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8cb000b040 con 0x7f8cb815a020 2023-08-11T15:51:49.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.855+0000 7f8ca6ffd640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8c9806a050 0x7f8c9806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:49.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.856+0000 7f8ca6ffd640 1 -- 172.21.15.27:0/3457456146 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8cb0089380 con 0x7f8cb815a020 2023-08-11T15:51:49.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.856+0000 7f8cb659c640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8c9806a050 0x7f8c9806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.857+0000 7f8cb659c640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8c9806a050 0x7f8c9806c510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8000f60 tx=0x7f8ca8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:49.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:49.860+0000 7f8ca6ffd640 1 -- 172.21.15.27:0/3457456146 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f8cb0054730 con 0x7f8cb815a020 2023-08-11T15:51:50.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.091+0000 7f8cb759e640 1 -- 172.21.15.27:0/3457456146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8cb80c3a80 con 0x7f8cb815a020 2023-08-11T15:51:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.092+0000 7f8ca6ffd640 1 -- 172.21.15.27:0/3457456146 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f8cb0057d50 con 0x7f8cb815a020 2023-08-11T15:51:50.094 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-11T15:51:50.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.094+0000 7f8cb759e640 1 -- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8c9806a050 msgr2=0x7f8c9806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:50.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.094+0000 7f8cb759e640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8c9806a050 0x7f8c9806c510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8000f60 tx=0x7f8ca8009040 comp rx=0 tx=0).stop 2023-08-11T15:51:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.094+0000 7f8cb759e640 1 -- 172.21.15.27:0/3457456146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb815a020 msgr2=0x7f8cb815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.094+0000 7f8cb759e640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb815a020 0x7f8cb815afb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000cd50 tx=0x7f8cb000a650 comp rx=0 tx=0).stop 2023-08-11T15:51:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.094+0000 7f8cb759e640 1 -- 172.21.15.27:0/3457456146 shutdown_connections 2023-08-11T15:51:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.094+0000 7f8cb759e640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8c9806a050 0x7f8c9806c510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.095+0000 7f8cb759e640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cb80c9600 0x7f8cb80c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.095+0000 7f8cb759e640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cb815a020 0x7f8cb815afb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.095+0000 7f8cb759e640 1 --2- 172.21.15.27:0/3457456146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cb806f5d0 0x7f8cb8154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.095+0000 7f8cb759e640 1 -- 172.21.15.27:0/3457456146 >> 172.21.15.27:0/3457456146 conn(0x7f8cb80afb70 msgr2=0x7f8cb8157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.095+0000 7f8cb759e640 1 -- 172.21.15.27:0/3457456146 shutdown_connections 2023-08-11T15:51:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:50.095+0000 7f8cb759e640 1 -- 172.21.15.27:0/3457456146 wait complete. 2023-08-11T15:51:50.116 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-11T15:51:50.116 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T15:51:50.117 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-08-11T15:51:50.117 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-11T15:51:50.117 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-11T15:51:50.117 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-11T15:51:50.117 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T15:51:50.117 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T15:51:50.118 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-11T15:51:50.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:50.118 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-08-11T15:51:50.177 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:51:50.177 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:51:50.178 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:51:50.178 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:50.178 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-08-11T15:51:50.178 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:50.178 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:50.178 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:50.178 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-08-11T15:51:50.179 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:50.179 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:50.179 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-08-11T15:51:50.179 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:50.179 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:50.179 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T15:51:50.179 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:50.180 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-11T15:51:50.180 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41963sec preferred_lft 41963sec 2023-08-11T15:51:50.180 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link noprefixroute 2023-08-11T15:51:50.180 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:50.181 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-11T15:51:50.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:50.182 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-08-11T15:51:50.182 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-08-11T15:51:50.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2023-08-11T15:51:50.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2023-08-11T15:51:50.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2023-08-11T15:51:50.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-11T15:51:50.183 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-08-11T15:51:50.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:50.688 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T15:51:50.771 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-08-11T15:51:50.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:50.775 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-08-11T15:51:50.830 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-08-11T15:51:50.831 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T15:51:50.831 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-11T15:51:50.831 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-11T15:51:50.831 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T15:51:50.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:50.832 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-08-11T15:51:50.832 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-08-11T15:51:50.832 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-11T15:51:50.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-11T15:51:50.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-11T15:51:50.833 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-08-11T15:51:51.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:51.703 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-08-11T15:51:51.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:51.760 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-08-11T15:51:51.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:51.814 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-08-11T15:51:51.814 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-08-11T15:51:51.814 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-11T15:51:51.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-08-11T15:51:51.815 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-08-11T15:51:51.959 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-08-11T15:51:51.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:51.960 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-08-11T15:51:51.960 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-08-11T15:51:51.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-08-11T15:51:51.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-11T15:51:51.962 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-08-11T15:51:51.962 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-08-11T15:51:51.963 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-08-11T15:51:51.963 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-08-11T15:51:52.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:52.257 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-08-11T15:51:52.258 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-08-11T15:51:52.258 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-08-11T15:51:52.258 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2023-08-11T15:51:52.258 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-08-11T15:51:52.425 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-11T15:51:52.426 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:51:52.426 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-11T15:51:52.473 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-11T15:51:52.473 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:51:52.474 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-11T15:51:52.571 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-08-11T15:51:52.666 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:52.666 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:52.745 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:52.748 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:52.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:52.749 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-08-11T15:51:52.799 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-11T15:51:52.799 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-11T15:51:52.848 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-08-11T15:51:52.929 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:52.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:52.999 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:53.002 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:53.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:53.003 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-08-11T15:51:53.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:2023-08-11T15:51:53.037+0000 7f808f43b480 -1 init, newargv = 0x559f7e1c6090 newargc=15 2023-08-11T15:51:53.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:ceph-fuse[95285]: starting ceph client 2023-08-11T15:51:53.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:ceph-fuse[95285]: starting fuse 2023-08-11T15:51:54.146 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-08-11T15:51:54.252 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:54.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:54.347 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:54.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:54.348 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-08-11T15:51:54.401 INFO:teuthology.orchestra.run.smithi119.stdout:39 2023-08-11T15:51:54.402 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-11T15:51:54.402 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer -- python3 -c ' 2023-08-11T15:51:54.403 DEBUG:teuthology.orchestra.run.smithi119:> import glob 2023-08-11T15:51:54.403 DEBUG:teuthology.orchestra.run.smithi119:> import re 2023-08-11T15:51:54.404 DEBUG:teuthology.orchestra.run.smithi119:> import os 2023-08-11T15:51:54.404 DEBUG:teuthology.orchestra.run.smithi119:> import subprocess 2023-08-11T15:51:54.404 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:51:54.404 DEBUG:teuthology.orchestra.run.smithi119:> def _find_admin_socket(client_name): 2023-08-11T15:51:54.405 DEBUG:teuthology.orchestra.run.smithi119:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-11T15:51:54.405 DEBUG:teuthology.orchestra.run.smithi119:> files = glob.glob(asok_path) 2023-08-11T15:51:54.405 DEBUG:teuthology.orchestra.run.smithi119:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-11T15:51:54.405 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:51:54.406 DEBUG:teuthology.orchestra.run.smithi119:> # Given a non-glob path, it better be there 2023-08-11T15:51:54.406 DEBUG:teuthology.orchestra.run.smithi119:> if "*" not in asok_path: 2023-08-11T15:51:54.406 DEBUG:teuthology.orchestra.run.smithi119:> assert(len(files) == 1) 2023-08-11T15:51:54.406 DEBUG:teuthology.orchestra.run.smithi119:> return files[0] 2023-08-11T15:51:54.406 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:51:54.407 DEBUG:teuthology.orchestra.run.smithi119:> for f in files: 2023-08-11T15:51:54.407 DEBUG:teuthology.orchestra.run.smithi119:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-11T15:51:54.407 DEBUG:teuthology.orchestra.run.smithi119:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-11T15:51:54.407 DEBUG:teuthology.orchestra.run.smithi119:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-11T15:51:54.408 DEBUG:teuthology.orchestra.run.smithi119:> contents = proc_f.read() 2023-08-11T15:51:54.408 DEBUG:teuthology.orchestra.run.smithi119:> if mountpoint in contents: 2023-08-11T15:51:54.408 DEBUG:teuthology.orchestra.run.smithi119:> return f 2023-08-11T15:51:54.408 DEBUG:teuthology.orchestra.run.smithi119:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-11T15:51:54.408 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:51:54.409 DEBUG:teuthology.orchestra.run.smithi119:> print(_find_admin_socket("client.1")) 2023-08-11T15:51:54.409 DEBUG:teuthology.orchestra.run.smithi119:> ' 2023-08-11T15:51:54.583 INFO:teuthology.orchestra.run.smithi119.stdout:/var/run/ceph/ceph-client.1.95285.asok 2023-08-11T15:51:54.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:51:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-11T15:51:54.600 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.95285.asok 2023-08-11T15:51:54.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:54.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.95285.asok status 2023-08-11T15:51:54.841 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-08-11T15:51:54.842 INFO:teuthology.orchestra.run.smithi119.stdout: "metadata": { 2023-08-11T15:51:54.842 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_sha1": "92684807f016ce045ac66e37fdde6f8c47857bcc", 2023-08-11T15:51:54.842 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:51:54.842 INFO:teuthology.orchestra.run.smithi119.stdout: "entity_id": "1", 2023-08-11T15:51:54.842 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": "smithi119", 2023-08-11T15:51:54.842 INFO:teuthology.orchestra.run.smithi119.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-11T15:51:54.842 INFO:teuthology.orchestra.run.smithi119.stdout: "pid": "95285", 2023-08-11T15:51:54.843 INFO:teuthology.orchestra.run.smithi119.stdout: "root": "/" 2023-08-11T15:51:54.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:51:54.843 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_count": 0, 2023-08-11T15:51:54.843 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_pinned_count": 0, 2023-08-11T15:51:54.843 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4610, 2023-08-11T15:51:54.843 INFO:teuthology.orchestra.run.smithi119.stdout: "inst": { 2023-08-11T15:51:54.843 INFO:teuthology.orchestra.run.smithi119.stdout: "name": { 2023-08-11T15:51:54.844 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "client", 2023-08-11T15:51:54.844 INFO:teuthology.orchestra.run.smithi119.stdout: "num": 4610 2023-08-11T15:51:54.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:51:54.844 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-08-11T15:51:54.844 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-08-11T15:51:54.844 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:51:54.844 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 1520606412 2023-08-11T15:51:54.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-08-11T15:51:54.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:51:54.845 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-08-11T15:51:54.845 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-08-11T15:51:54.845 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:51:54.845 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 1520606412 2023-08-11T15:51:54.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:51:54.846 INFO:teuthology.orchestra.run.smithi119.stdout: "inst_str": "client.4610 192.168.0.1:0/1520606412", 2023-08-11T15:51:54.846 INFO:teuthology.orchestra.run.smithi119.stdout: "addr_str": "192.168.0.1:0/1520606412", 2023-08-11T15:51:54.846 INFO:teuthology.orchestra.run.smithi119.stdout: "inode_count": 1, 2023-08-11T15:51:54.846 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_epoch": 9, 2023-08-11T15:51:54.846 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch": 30, 2023-08-11T15:51:54.846 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch_barrier": 0, 2023-08-11T15:51:54.846 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklisted": false, 2023-08-11T15:51:54.847 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs" 2023-08-11T15:51:54.847 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-08-11T15:51:54.856 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-11T15:51:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.101+0000 7ffbf3771640 1 Processor -- start 2023-08-11T15:51:55.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.101+0000 7ffbf3771640 1 -- start start 2023-08-11T15:51:55.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.102+0000 7ffbf3771640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbec15a140 0x7ffbec06b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:55.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.102+0000 7ffbf3771640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec06f5d0 0x7ffbec06fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:55.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.102+0000 7ffbf3771640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbec0c8a30 con 0x7ffbec0cb430 2023-08-11T15:51:55.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.102+0000 7ffbf3771640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbec06ffa0 con 0x7ffbec15a140 2023-08-11T15:51:55.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.102+0000 7ffbf3771640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0b1580 con 0x7ffbec06f5d0 2023-08-11T15:51:55.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.103+0000 7ffbf1d0e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec06f5d0 0x7ffbec06fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:55.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.103+0000 7ffbf150d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbec15a140 0x7ffbec06b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:55.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.103+0000 7ffbf1d0e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec06f5d0 0x7ffbec06fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:38698/0 (socket says 172.21.15.27:38698) 2023-08-11T15:51:55.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.103+0000 7ffbf1d0e640 1 -- 172.21.15.27:0/2350087661 learned_addr learned my addr 172.21.15.27:0/2350087661 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:55.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.104+0000 7ffbf1d0e640 1 -- 172.21.15.27:0/2350087661 >> v2:172.21.15.119:3300/0 conn(0x7ffbec15a140 msgr2=0x7ffbec06b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:55.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.104+0000 7ffbf1d0e640 1 --2- 172.21.15.27:0/2350087661 >> v2:172.21.15.119:3300/0 conn(0x7ffbec15a140 0x7ffbec06b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:55.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.104+0000 7ffbf1d0e640 1 -- 172.21.15.27:0/2350087661 >> v1:172.21.15.119:6789/0 conn(0x7ffbec0cb430 legacy=0x7ffbec0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:55.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.104+0000 7ffbf1d0e640 1 -- 172.21.15.27:0/2350087661 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbec06c0c0 con 0x7ffbec06f5d0 2023-08-11T15:51:55.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.105+0000 7ffbf1d0e640 1 --2- 172.21.15.27:0/2350087661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec06f5d0 0x7ffbec06fa60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009870 tx=0x7ffbe402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70f9a57d67dacae8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:55.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.106+0000 7ffbebfff640 1 -- 172.21.15.27:0/2350087661 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4030840 con 0x7ffbec06f5d0 2023-08-11T15:51:55.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.106+0000 7ffbebfff640 1 -- 172.21.15.27:0/2350087661 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe40309a0 con 0x7ffbec06f5d0 2023-08-11T15:51:55.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.106+0000 7ffbebfff640 1 -- 172.21.15.27:0/2350087661 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4030d30 con 0x7ffbec06f5d0 2023-08-11T15:51:55.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.107+0000 7ffbf3771640 1 -- 172.21.15.27:0/2350087661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec06f5d0 msgr2=0x7ffbec06fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:55.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.107+0000 7ffbf3771640 1 --2- 172.21.15.27:0/2350087661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec06f5d0 0x7ffbec06fa60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009870 tx=0x7ffbe402b980 comp rx=0 tx=0).stop 2023-08-11T15:51:55.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.107+0000 7ffbf3771640 1 -- 172.21.15.27:0/2350087661 shutdown_connections 2023-08-11T15:51:55.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.107+0000 7ffbf3771640 1 --2- 172.21.15.27:0/2350087661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec06f5d0 0x7ffbec06fa60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:55.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.107+0000 7ffbf3771640 1 --2- 172.21.15.27:0/2350087661 >> v2:172.21.15.119:3300/0 conn(0x7ffbec15a140 0x7ffbec06b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:55.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.107+0000 7ffbf3771640 1 -- 172.21.15.27:0/2350087661 >> 172.21.15.27:0/2350087661 conn(0x7ffbec0afb70 msgr2=0x7ffbec0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.107+0000 7ffbf3771640 1 -- 172.21.15.27:0/2350087661 shutdown_connections 2023-08-11T15:51:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.107+0000 7ffbf3771640 1 -- 172.21.15.27:0/2350087661 wait complete. 2023-08-11T15:51:55.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.109+0000 7ffbf3771640 1 Processor -- start 2023-08-11T15:51:55.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.109+0000 7ffbf3771640 1 -- start start 2023-08-11T15:51:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf3771640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbec06f5d0 0x7ffbec0af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf3771640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbec0cb430 0x7ffbec0c3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf3771640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec15a140 0x7ffbec0c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf3771640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0af6d0 con 0x7ffbec0cb430 2023-08-11T15:51:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf3771640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec157ee0 con 0x7ffbec06f5d0 2023-08-11T15:51:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf3771640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec15afd0 con 0x7ffbec15a140 2023-08-11T15:51:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf0d0c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbec0cb430 0x7ffbec0c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf0d0c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbec0cb430 0x7ffbec0c3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43496/0 (socket says 172.21.15.27:43496) 2023-08-11T15:51:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf0d0c640 1 -- 172.21.15.27:0/4267564918 learned_addr learned my addr 172.21.15.27:0/4267564918 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf1d0e640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec15a140 0x7ffbec0c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.110+0000 7ffbf150d640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbec06f5d0 0x7ffbec0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:55.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbf0d0c640 1 -- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec15a140 msgr2=0x7ffbec0c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:55.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbf0d0c640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec15a140 0x7ffbec0c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:55.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbf0d0c640 1 -- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbec06f5d0 msgr2=0x7ffbec0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:55.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbf0d0c640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbec06f5d0 0x7ffbec0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbf0d0c640 1 -- 172.21.15.27:0/4267564918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe40094d0 con 0x7ffbec0cb430 2023-08-11T15:51:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbf0d0c640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbec0cb430 0x7ffbec0c3270 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000c400 tx=0x7ffbe000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbe9ffb640 1 -- 172.21.15.27:0/4267564918 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00052d0 con 0x7ffbec0cb430 2023-08-11T15:51:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbe9ffb640 1 -- 172.21.15.27:0/4267564918 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe0005450 con 0x7ffbec0cb430 2023-08-11T15:51:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbf3771640 1 -- 172.21.15.27:0/4267564918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbec15b2b0 con 0x7ffbec0cb430 2023-08-11T15:51:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbe9ffb640 1 -- 172.21.15.27:0/4267564918 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0007260 con 0x7ffbec0cb430 2023-08-11T15:51:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.111+0000 7ffbf3771640 1 -- 172.21.15.27:0/4267564918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbec15b870 con 0x7ffbec0cb430 2023-08-11T15:51:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.113+0000 7ffbf3771640 1 -- 172.21.15.27:0/4267564918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbec073990 con 0x7ffbec0cb430 2023-08-11T15:51:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.114+0000 7ffbe9ffb640 1 -- 172.21.15.27:0/4267564918 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffbe00073c0 con 0x7ffbec0cb430 2023-08-11T15:51:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.115+0000 7ffbe9ffb640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffbd806a050 0x7ffbd806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.115+0000 7ffbe9ffb640 1 -- 172.21.15.27:0/4267564918 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ffbe0087cf0 con 0x7ffbec0cb430 2023-08-11T15:51:55.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.115+0000 7ffbf150d640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffbd806a050 0x7ffbd806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:55.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.116+0000 7ffbf150d640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffbd806a050 0x7ffbd806c510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc007440 tx=0x7ffbdc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:55.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.119+0000 7ffbe9ffb640 1 -- 172.21.15.27:0/4267564918 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ffbe008c070 con 0x7ffbec0cb430 2023-08-11T15:51:55.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.356+0000 7ffbf3771640 1 -- 172.21.15.27:0/4267564918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffbec06fd30 con 0x7ffbec0cb430 2023-08-11T15:51:55.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.357+0000 7ffbe9ffb640 1 -- 172.21.15.27:0/4267564918 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7ffbe00530b0 con 0x7ffbec0cb430 2023-08-11T15:51:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-11T15:51:55.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 -- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffbd806a050 msgr2=0x7ffbd806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:55.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffbd806a050 0x7ffbd806c510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc007440 tx=0x7ffbdc00b040 comp rx=0 tx=0).stop 2023-08-11T15:51:55.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 -- 172.21.15.27:0/4267564918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbec0cb430 msgr2=0x7ffbec0c3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:55.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbec0cb430 0x7ffbec0c3270 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000c400 tx=0x7ffbe000c8c0 comp rx=0 tx=0).stop 2023-08-11T15:51:55.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 -- 172.21.15.27:0/4267564918 shutdown_connections 2023-08-11T15:51:55.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ffbd806a050 0x7ffbd806c510 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:55.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbec15a140 0x7ffbec0c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:55.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbec0cb430 0x7ffbec0c3270 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:55.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 --2- 172.21.15.27:0/4267564918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbec06f5d0 0x7ffbec0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:55.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 -- 172.21.15.27:0/4267564918 >> 172.21.15.27:0/4267564918 conn(0x7ffbec0afb70 msgr2=0x7ffbec0cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:55.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.360+0000 7ffbf3771640 1 -- 172.21.15.27:0/4267564918 shutdown_connections 2023-08-11T15:51:55.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:55.361+0000 7ffbf3771640 1 -- 172.21.15.27:0/4267564918 wait complete. 2023-08-11T15:51:55.382 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-11T15:51:55.382 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T15:51:55.382 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-08-11T15:51:55.383 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-08-11T15:51:55.383 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-11T15:51:55.383 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-11T15:51:55.383 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T15:51:55.383 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T15:51:55.384 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-08-11T15:51:55.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:55.384 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-08-11T15:51:55.445 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:51:55.446 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:51:55.446 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:51:55.446 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:55.447 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-08-11T15:51:55.447 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:55.447 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:55.447 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:55.447 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-08-11T15:51:55.447 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:55.448 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:55.448 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-08-11T15:51:55.448 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:51:55.448 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:55.448 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T15:51:55.448 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:55.448 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-11T15:51:55.449 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41958sec preferred_lft 41958sec 2023-08-11T15:51:55.449 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link noprefixroute 2023-08-11T15:51:55.449 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:55.449 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-11T15:51:55.449 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 6a:1a:57:8c:46:a1 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:51:55.449 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-11T15:51:55.449 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:55.449 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c0f5:5eff:fe99:60b/64 scope link 2023-08-11T15:51:55.450 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:55.450 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-11T15:51:55.450 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 6a:1a:57:8c:46:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-11T15:51:55.450 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::681a:57ff:fe8c:46a1/64 scope link 2023-08-11T15:51:55.450 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:55.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:55.451 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T15:51:55.546 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-08-11T15:51:55.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:55.551 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-08-11T15:51:55.606 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-08-11T15:51:55.606 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T15:51:55.607 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-11T15:51:55.607 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-11T15:51:55.607 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T15:51:55.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:55.608 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-08-11T15:51:55.608 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-08-11T15:51:55.608 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-11T15:51:55.609 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-11T15:51:55.609 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-11T15:51:55.609 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-08-11T15:51:55.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:55.798 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-08-11T15:51:55.850 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-11T15:51:55.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:55.852 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-08-11T15:51:55.906 INFO:teuthology.orchestra.run.smithi119.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-08-11T15:51:55.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:55.907 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-08-11T15:51:55.907 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-08-11T15:51:55.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-11T15:51:55.908 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-08-11T15:51:55.908 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-08-11T15:51:56.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:56.057 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-11T15:51:56.142 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:51:56.142 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:51:56.142 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:51:56.143 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:56.143 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-08-11T15:51:56.143 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:56.143 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-11T15:51:56.143 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether f6:d7:68:06:a3:0a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-11T15:51:56.144 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-11T15:51:56.144 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:56.144 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f4d7:68ff:fe06:a30a/64 scope link 2023-08-11T15:51:56.144 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:56.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:56.146 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-11T15:51:56.238 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:51:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:51:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:51:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-08-11T15:51:56.239 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:56.240 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-11T15:51:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether f6:d7:68:06:a3:0a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-11T15:51:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-11T15:51:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f4d7:68ff:fe06:a30a/64 scope link 2023-08-11T15:51:56.240 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:51:56.243 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-08-11T15:51:56.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:56.244 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-08-11T15:51:56.244 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-08-11T15:51:56.244 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-08-11T15:51:56.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-11T15:51:56.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-08-11T15:51:56.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-08-11T15:51:56.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-08-11T15:51:56.245 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-08-11T15:51:56.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:56.500 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-08-11T15:51:56.500 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-08-11T15:51:56.500 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.1 up 2023-08-11T15:51:56.500 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.1 master ceph-brx 2023-08-11T15:51:56.501 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-08-11T15:51:56.656 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-08-11T15:51:56.657 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:51:56.657 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-08-11T15:51:56.705 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-08-11T15:51:56.705 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:51:56.706 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-08-11T15:51:56.751 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-08-11T15:51:56.891 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:56.892 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:56.981 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:56.984 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:56.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:56.985 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-08-11T15:51:57.038 INFO:teuthology.orchestra.run.smithi119.stdout:39 2023-08-11T15:51:57.038 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [39] 2023-08-11T15:51:57.038 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-08-11T15:51:57.043 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-08-11T15:51:57.129 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:51:57.130 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:51:57.205 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:51:57.208 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:51:57.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:57.209 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-08-11T15:51:57.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi119.stderr:2023-08-11T15:51:57.235+0000 7f6b4dc4f480 -1 init, newargv = 0x5605c77f8e10 newargc=15 2023-08-11T15:51:57.237 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi119.stderr:ceph-fuse[95804]: starting ceph client 2023-08-11T15:51:57.245 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi119.stderr:ceph-fuse[95804]: starting fuse 2023-08-11T15:51:57.254 INFO:teuthology.orchestra.run.smithi119.stdout:39 2023-08-11T15:51:57.255 INFO:teuthology.orchestra.run.smithi119.stdout:41 2023-08-11T15:51:57.255 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39, 41] 2023-08-11T15:51:57.255 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer -- python3 -c ' 2023-08-11T15:51:57.255 DEBUG:teuthology.orchestra.run.smithi119:> import glob 2023-08-11T15:51:57.256 DEBUG:teuthology.orchestra.run.smithi119:> import re 2023-08-11T15:51:57.256 DEBUG:teuthology.orchestra.run.smithi119:> import os 2023-08-11T15:51:57.256 DEBUG:teuthology.orchestra.run.smithi119:> import subprocess 2023-08-11T15:51:57.256 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:51:57.257 DEBUG:teuthology.orchestra.run.smithi119:> def _find_admin_socket(client_name): 2023-08-11T15:51:57.257 DEBUG:teuthology.orchestra.run.smithi119:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-08-11T15:51:57.257 DEBUG:teuthology.orchestra.run.smithi119:> files = glob.glob(asok_path) 2023-08-11T15:51:57.257 DEBUG:teuthology.orchestra.run.smithi119:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-08-11T15:51:57.258 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:51:57.258 DEBUG:teuthology.orchestra.run.smithi119:> # Given a non-glob path, it better be there 2023-08-11T15:51:57.258 DEBUG:teuthology.orchestra.run.smithi119:> if "*" not in asok_path: 2023-08-11T15:51:57.258 DEBUG:teuthology.orchestra.run.smithi119:> assert(len(files) == 1) 2023-08-11T15:51:57.259 DEBUG:teuthology.orchestra.run.smithi119:> return files[0] 2023-08-11T15:51:57.259 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:51:57.259 DEBUG:teuthology.orchestra.run.smithi119:> for f in files: 2023-08-11T15:51:57.259 DEBUG:teuthology.orchestra.run.smithi119:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-11T15:51:57.260 DEBUG:teuthology.orchestra.run.smithi119:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-11T15:51:57.260 DEBUG:teuthology.orchestra.run.smithi119:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-11T15:51:57.260 DEBUG:teuthology.orchestra.run.smithi119:> contents = proc_f.read() 2023-08-11T15:51:57.260 DEBUG:teuthology.orchestra.run.smithi119:> if mountpoint in contents: 2023-08-11T15:51:57.261 DEBUG:teuthology.orchestra.run.smithi119:> return f 2023-08-11T15:51:57.261 DEBUG:teuthology.orchestra.run.smithi119:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-11T15:51:57.261 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:51:57.261 DEBUG:teuthology.orchestra.run.smithi119:> print(_find_admin_socket("client.3")) 2023-08-11T15:51:57.262 DEBUG:teuthology.orchestra.run.smithi119:> ' 2023-08-11T15:51:57.458 INFO:teuthology.orchestra.run.smithi119.stdout:/var/run/ceph/ceph-client.3.95804.asok 2023-08-11T15:51:57.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:51:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-11T15:51:57.475 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.95804.asok 2023-08-11T15:51:57.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:57.476 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.95804.asok status 2023-08-11T15:51:57.717 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-08-11T15:51:57.718 INFO:teuthology.orchestra.run.smithi119.stdout: "metadata": { 2023-08-11T15:51:57.718 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_sha1": "92684807f016ce045ac66e37fdde6f8c47857bcc", 2023-08-11T15:51:57.718 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:51:57.719 INFO:teuthology.orchestra.run.smithi119.stdout: "entity_id": "3", 2023-08-11T15:51:57.719 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": "smithi119", 2023-08-11T15:51:57.719 INFO:teuthology.orchestra.run.smithi119.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-08-11T15:51:57.719 INFO:teuthology.orchestra.run.smithi119.stdout: "pid": "95804", 2023-08-11T15:51:57.719 INFO:teuthology.orchestra.run.smithi119.stdout: "root": "/" 2023-08-11T15:51:57.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:51:57.720 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_count": 0, 2023-08-11T15:51:57.720 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_pinned_count": 0, 2023-08-11T15:51:57.720 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4619, 2023-08-11T15:51:57.720 INFO:teuthology.orchestra.run.smithi119.stdout: "inst": { 2023-08-11T15:51:57.720 INFO:teuthology.orchestra.run.smithi119.stdout: "name": { 2023-08-11T15:51:57.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "client", 2023-08-11T15:51:57.721 INFO:teuthology.orchestra.run.smithi119.stdout: "num": 4619 2023-08-11T15:51:57.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:51:57.721 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-08-11T15:51:57.721 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-08-11T15:51:57.721 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.2:0", 2023-08-11T15:51:57.721 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3247408671 2023-08-11T15:51:57.721 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-08-11T15:51:57.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:51:57.722 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-08-11T15:51:57.722 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-08-11T15:51:57.722 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.2:0", 2023-08-11T15:51:57.722 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3247408671 2023-08-11T15:51:57.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:51:57.722 INFO:teuthology.orchestra.run.smithi119.stdout: "inst_str": "client.4619 192.168.0.2:0/3247408671", 2023-08-11T15:51:57.722 INFO:teuthology.orchestra.run.smithi119.stdout: "addr_str": "192.168.0.2:0/3247408671", 2023-08-11T15:51:57.723 INFO:teuthology.orchestra.run.smithi119.stdout: "inode_count": 1, 2023-08-11T15:51:57.723 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_epoch": 9, 2023-08-11T15:51:57.723 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch": 30, 2023-08-11T15:51:57.723 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch_barrier": 0, 2023-08-11T15:51:57.723 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklisted": false, 2023-08-11T15:51:57.724 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs" 2023-08-11T15:51:57.724 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-08-11T15:51:57.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:57.732 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-08-11T15:51:57.732 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-11T15:51:57.800 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-08-11T15:51:57.801 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-11T15:51:57.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:57.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-11T15:51:57.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:57.914 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-08-11T15:51:57.914 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.2 2023-08-11T15:51:57.975 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-08-11T15:51:57.976 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-08-11T15:51:57.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:57.976 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-08-11T15:51:58.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:58.086 DEBUG:teuthology.orchestra.run.smithi119:> stat --file-system '--printf=%T 2023-08-11T15:51:58.086 DEBUG:teuthology.orchestra.run.smithi119:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-11T15:51:58.147 INFO:teuthology.orchestra.run.smithi119.stdout:fuseblk 2023-08-11T15:51:58.148 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-11T15:51:58.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:58.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-11T15:51:58.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:58.246 DEBUG:teuthology.orchestra.run.smithi119:> stat --file-system '--printf=%T 2023-08-11T15:51:58.247 DEBUG:teuthology.orchestra.run.smithi119:> ' -- /home/ubuntu/cephtest/mnt.3 2023-08-11T15:51:58.304 INFO:teuthology.orchestra.run.smithi119.stdout:fuseblk 2023-08-11T15:51:58.304 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-08-11T15:51:58.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:51:58.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-08-11T15:51:58.390 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-08-11T15:51:58.409 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:51:58.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.652+0000 7f70f5e2d640 1 Processor -- start 2023-08-11T15:51:58.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.653+0000 7f70f5e2d640 1 -- start start 2023-08-11T15:51:58.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.654+0000 7f70f5e2d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70f0160a20 0x7f70f00ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:58.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.654+0000 7f70f5e2d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f70f00c15c0 0x7f70f00caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.654+0000 7f70f5e2d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f00cb1e0 con 0x7f70f00c1070 2023-08-11T15:51:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.654+0000 7f70f5e2d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f0160e00 con 0x7f70f00c15c0 2023-08-11T15:51:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.654+0000 7f70f5e2d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f01652a0 con 0x7f70f0160a20 2023-08-11T15:51:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.654+0000 7f70e7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f70f00c15c0 0x7f70f00caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.655+0000 7f70effff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f70f00c1070 0x7f70f006b870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52228/0 (socket says 172.21.15.27:52228) 2023-08-11T15:51:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.655+0000 7f70effff640 1 -- 172.21.15.27:0/4154255022 learned_addr learned my addr 172.21.15.27:0/4154255022 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.655+0000 7f70ef7fe640 1 --2- 172.21.15.27:0/4154255022 >> v2:172.21.15.119:3300/0 conn(0x7f70f0160a20 0x7f70f00ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.655+0000 7f70e7fff640 1 -- 172.21.15.27:0/4154255022 >> v2:172.21.15.119:3300/0 conn(0x7f70f0160a20 msgr2=0x7f70f00ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.655+0000 7f70e7fff640 1 --2- 172.21.15.27:0/4154255022 >> v2:172.21.15.119:3300/0 conn(0x7f70f0160a20 0x7f70f00ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.655+0000 7f70e7fff640 1 -- 172.21.15.27:0/4154255022 >> v1:172.21.15.27:6789/0 conn(0x7f70f00c1070 legacy=0x7f70f006b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:58.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.655+0000 7f70e7fff640 1 -- 172.21.15.27:0/4154255022 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00cbe00 con 0x7f70f00c15c0 2023-08-11T15:51:58.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.655+0000 7f70eeffd640 1 -- 172.21.15.27:0/4154255022 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501494429 0 0) 0x7f70f00cb1e0 con 0x7f70f00c1070 2023-08-11T15:51:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.656+0000 7f70e7fff640 1 --2- 172.21.15.27:0/4154255022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c15c0 0x7f70f00caca0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f70f006a6a0 tx=0x7f70d802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fea06b55b1c512 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.657+0000 7f70eeffd640 1 -- 172.21.15.27:0/4154255022 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8031710 con 0x7f70f00c15c0 2023-08-11T15:51:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.657+0000 7f70eeffd640 1 -- 172.21.15.27:0/4154255022 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d802f050 con 0x7f70f00c15c0 2023-08-11T15:51:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.657+0000 7f70eeffd640 1 -- 172.21.15.27:0/4154255022 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8031a90 con 0x7f70f00c15c0 2023-08-11T15:51:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.657+0000 7f70f5e2d640 1 -- 172.21.15.27:0/4154255022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c15c0 msgr2=0x7f70f00caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.657+0000 7f70f5e2d640 1 --2- 172.21.15.27:0/4154255022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c15c0 0x7f70f00caca0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f70f006a6a0 tx=0x7f70d802b7b0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.657+0000 7f70f5e2d640 1 -- 172.21.15.27:0/4154255022 shutdown_connections 2023-08-11T15:51:58.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.657+0000 7f70f5e2d640 1 --2- 172.21.15.27:0/4154255022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c15c0 0x7f70f00caca0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.657+0000 7f70f5e2d640 1 --2- 172.21.15.27:0/4154255022 >> v2:172.21.15.119:3300/0 conn(0x7f70f0160a20 0x7f70f00ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.657+0000 7f70f5e2d640 1 -- 172.21.15.27:0/4154255022 >> 172.21.15.27:0/4154255022 conn(0x7f70f00afb70 msgr2=0x7f70f00aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:58.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.657+0000 7f70f5e2d640 1 -- 172.21.15.27:0/4154255022 shutdown_connections 2023-08-11T15:51:58.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.658+0000 7f70f5e2d640 1 -- 172.21.15.27:0/4154255022 wait complete. 2023-08-11T15:51:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.659+0000 7f70f5e2d640 1 Processor -- start 2023-08-11T15:51:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.659+0000 7f70f5e2d640 1 -- start start 2023-08-11T15:51:58.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70f5e2d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c1070 0x7f70f0161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:58.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70f5e2d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00c15c0 0x7f70f014f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:58.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70f5e2d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f0160a20 0x7f70f006bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:58.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70f5e2d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f0161200 con 0x7f70f00c1070 2023-08-11T15:51:58.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70f5e2d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00c2780 con 0x7f70f00c15c0 2023-08-11T15:51:58.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70f5e2d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00c2a60 con 0x7f70f0160a20 2023-08-11T15:51:58.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c1070 0x7f70f0161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:58.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c1070 0x7f70f0161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43510/0 (socket says 172.21.15.27:43510) 2023-08-11T15:51:58.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70ef7fe640 1 -- 172.21.15.27:0/2912467925 learned_addr learned my addr 172.21.15.27:0/2912467925 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70e7fff640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00c15c0 0x7f70f014f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70effff640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f0160a20 0x7f70f006bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70ef7fe640 1 -- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f0160a20 msgr2=0x7f70f006bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70ef7fe640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f0160a20 0x7f70f006bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70ef7fe640 1 -- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00c15c0 msgr2=0x7f70f014f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70ef7fe640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00c15c0 0x7f70f014f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70ef7fe640 1 -- 172.21.15.27:0/2912467925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8009400 con 0x7f70f00c1070 2023-08-11T15:51:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.661+0000 7f70effff640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f0160a20 0x7f70f006bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.662+0000 7f70ef7fe640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c1070 0x7f70f0161570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f70e000d800 tx=0x7f70e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.662+0000 7f70ecff9640 1 -- 172.21.15.27:0/2912467925 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e000b6c0 con 0x7f70f00c1070 2023-08-11T15:51:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.662+0000 7f70ecff9640 1 -- 172.21.15.27:0/2912467925 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e000b820 con 0x7f70f00c1070 2023-08-11T15:51:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.662+0000 7f70ecff9640 1 -- 172.21.15.27:0/2912467925 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e000bbd0 con 0x7f70f00c1070 2023-08-11T15:51:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.662+0000 7f70f5e2d640 1 -- 172.21.15.27:0/2912467925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f00c2d40 con 0x7f70f00c1070 2023-08-11T15:51:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.662+0000 7f70f5e2d640 1 -- 172.21.15.27:0/2912467925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f00c3300 con 0x7f70f00c1070 2023-08-11T15:51:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.664+0000 7f70f5e2d640 1 -- 172.21.15.27:0/2912467925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f00c3490 con 0x7f70f00c1070 2023-08-11T15:51:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.664+0000 7f70ecff9640 1 -- 172.21.15.27:0/2912467925 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f70e000bd30 con 0x7f70f00c1070 2023-08-11T15:51:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.665+0000 7f70ecff9640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70d006a050 0x7f70d006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.666+0000 7f70ecff9640 1 -- 172.21.15.27:0/2912467925 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f70e0088ea0 con 0x7f70f00c1070 2023-08-11T15:51:58.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.666+0000 7f70e7fff640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70d006a050 0x7f70d006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:58.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.667+0000 7f70e7fff640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70d006a050 0x7f70d006c510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f70d8035000 tx=0x7f70d8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:58.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.670+0000 7f70ecff9640 1 -- 172.21.15.27:0/2912467925 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f70e00541c0 con 0x7f70f00c1070 2023-08-11T15:51:58.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.869+0000 7f70f5e2d640 1 -- 172.21.15.27:0/2912467925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70f0073990 con 0x7f70f00c1070 2023-08-11T15:51:58.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.872+0000 7f70ecff9640 1 -- 172.21.15.27:0/2912467925 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f70e00577e0 con 0x7f70f00c1070 2023-08-11T15:51:58.875 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:51:58.876 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:51:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 -- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70d006a050 msgr2=0x7f70d006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70d006a050 0x7f70d006c510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f70d8035000 tx=0x7f70d8008030 comp rx=0 tx=0).stop 2023-08-11T15:51:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 -- 172.21.15.27:0/2912467925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c1070 msgr2=0x7f70f0161570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c1070 0x7f70f0161570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f70e000d800 tx=0x7f70e000dcc0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 -- 172.21.15.27:0/2912467925 shutdown_connections 2023-08-11T15:51:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f70d006a050 0x7f70d006c510 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f0160a20 0x7f70f006bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f00c15c0 0x7f70f014f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 --2- 172.21.15.27:0/2912467925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70f00c1070 0x7f70f0161570 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:58.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 -- 172.21.15.27:0/2912467925 >> 172.21.15.27:0/2912467925 conn(0x7f70f00afb70 msgr2=0x7f70f00af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:58.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 -- 172.21.15.27:0/2912467925 shutdown_connections 2023-08-11T15:51:58.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:58.875+0000 7f70f5e2d640 1 -- 172.21.15.27:0/2912467925 wait complete. 2023-08-11T15:51:58.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:51:58.897 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:51:59.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.127+0000 7f0595b42640 1 Processor -- start 2023-08-11T15:51:59.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.128+0000 7f0595b42640 1 -- start start 2023-08-11T15:51:59.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.129+0000 7f0595b42640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05900cb700 0x7f059015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.129+0000 7f0595b42640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05900ac160 0x7f059015d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.130+0000 7f0595b42640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0590071ad0 con 0x7f05900abd80 2023-08-11T15:51:59.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.130+0000 7f0595b42640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0590160190 con 0x7f05900cb700 2023-08-11T15:51:59.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.130+0000 7f0595b42640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05900c9720 con 0x7f05900ac160 2023-08-11T15:51:59.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.130+0000 7f058effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05900cb700 0x7f059015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.130+0000 7f058effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05900cb700 0x7f059015cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57062/0 (socket says 172.21.15.27:57062) 2023-08-11T15:51:59.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.130+0000 7f058effd640 1 -- 172.21.15.27:0/2195892581 learned_addr learned my addr 172.21.15.27:0/2195892581 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:59.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.130+0000 7f058ffff640 1 --2- 172.21.15.27:0/2195892581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05900ac160 0x7f059015d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.130+0000 7f058effd640 1 -- 172.21.15.27:0/2195892581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05900ac160 msgr2=0x7f059015d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:59.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.130+0000 7f058effd640 1 --2- 172.21.15.27:0/2195892581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05900ac160 0x7f059015d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.130+0000 7f058effd640 1 -- 172.21.15.27:0/2195892581 >> v1:172.21.15.119:6789/0 conn(0x7f05900abd80 legacy=0x7f05900caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.131+0000 7f058effd640 1 -- 172.21.15.27:0/2195892581 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059015d940 con 0x7f05900cb700 2023-08-11T15:51:59.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.132+0000 7f058effd640 1 --2- 172.21.15.27:0/2195892581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 0x7f059015cba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f05780098a0 tx=0x7f057802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6ece9c81c322b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:59.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.132+0000 7f058e7fc640 1 -- 172.21.15.27:0/2195892581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0578030840 con 0x7f05900cb700 2023-08-11T15:51:59.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.132+0000 7f058e7fc640 1 -- 172.21.15.27:0/2195892581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05780309a0 con 0x7f05900cb700 2023-08-11T15:51:59.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.132+0000 7f058e7fc640 1 -- 172.21.15.27:0/2195892581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0578030d30 con 0x7f05900cb700 2023-08-11T15:51:59.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.133+0000 7f0595b42640 1 -- 172.21.15.27:0/2195892581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 msgr2=0x7f059015cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.133+0000 7f0595b42640 1 --2- 172.21.15.27:0/2195892581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 0x7f059015cba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f05780098a0 tx=0x7f057802b980 comp rx=0 tx=0).stop 2023-08-11T15:51:59.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.133+0000 7f0595b42640 1 -- 172.21.15.27:0/2195892581 shutdown_connections 2023-08-11T15:51:59.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.133+0000 7f0595b42640 1 --2- 172.21.15.27:0/2195892581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05900ac160 0x7f059015d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.133+0000 7f0595b42640 1 --2- 172.21.15.27:0/2195892581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 0x7f059015cba0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.133+0000 7f0595b42640 1 -- 172.21.15.27:0/2195892581 >> 172.21.15.27:0/2195892581 conn(0x7f0590069bf0 msgr2=0x7f059006a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:59.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.133+0000 7f0595b42640 1 -- 172.21.15.27:0/2195892581 shutdown_connections 2023-08-11T15:51:59.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.133+0000 7f0595b42640 1 -- 172.21.15.27:0/2195892581 wait complete. 2023-08-11T15:51:59.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.135+0000 7f0595b42640 1 Processor -- start 2023-08-11T15:51:59.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.135+0000 7f0595b42640 1 -- start start 2023-08-11T15:51:59.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.136+0000 7f0595b42640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05900abd80 0x7f05901545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.136+0000 7f0595b42640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05900ac160 0x7f05900b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.136+0000 7f0595b42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 0x7f05900701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.136+0000 7f0595b42640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05900b30c0 con 0x7f05900cb700 2023-08-11T15:51:59.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.136+0000 7f0595b42640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05900b33a0 con 0x7f05900abd80 2023-08-11T15:51:59.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.136+0000 7f0595b42640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05900b3680 con 0x7f05900ac160 2023-08-11T15:51:59.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.136+0000 7f058ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 0x7f05900701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 0x7f05900701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57064/0 (socket says 172.21.15.27:57064) 2023-08-11T15:51:59.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058ffff640 1 -- 172.21.15.27:0/3081523757 learned_addr learned my addr 172.21.15.27:0/3081523757 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:59.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058effd640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05900ac160 0x7f05900b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058f7fe640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05900abd80 0x7f05901545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058ffff640 1 -- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05900ac160 msgr2=0x7f05900b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058ffff640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05900ac160 0x7f05900b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058ffff640 1 -- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05900abd80 msgr2=0x7f05901545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058ffff640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05900abd80 0x7f05901545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058ffff640 1 -- 172.21.15.27:0/3081523757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05780094d0 con 0x7f05900cb700 2023-08-11T15:51:59.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058f7fe640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05900abd80 0x7f05901545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:51:59.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.137+0000 7f058ffff640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 0x7f05900701d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f058800d800 tx=0x7f058800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:59.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.138+0000 7f0587fff640 1 -- 172.21.15.27:0/3081523757 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058800b6c0 con 0x7f05900cb700 2023-08-11T15:51:59.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.138+0000 7f0587fff640 1 -- 172.21.15.27:0/3081523757 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058800b840 con 0x7f05900cb700 2023-08-11T15:51:59.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.138+0000 7f0587fff640 1 -- 172.21.15.27:0/3081523757 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058800bbf0 con 0x7f05900cb700 2023-08-11T15:51:59.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.138+0000 7f0595b42640 1 -- 172.21.15.27:0/3081523757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05900b3960 con 0x7f05900cb700 2023-08-11T15:51:59.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.138+0000 7f0595b42640 1 -- 172.21.15.27:0/3081523757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05900b0870 con 0x7f05900cb700 2023-08-11T15:51:59.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.139+0000 7f0595b42640 1 -- 172.21.15.27:0/3081523757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059004e420 con 0x7f05900cb700 2023-08-11T15:51:59.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.140+0000 7f0587fff640 1 -- 172.21.15.27:0/3081523757 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f058800bd50 con 0x7f05900cb700 2023-08-11T15:51:59.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.141+0000 7f0587fff640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f057006a050 0x7f057006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.141+0000 7f0587fff640 1 -- 172.21.15.27:0/3081523757 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0588088ed0 con 0x7f05900cb700 2023-08-11T15:51:59.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.142+0000 7f058f7fe640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f057006a050 0x7f057006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.143+0000 7f058f7fe640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f057006a050 0x7f057006c510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f05800012e0 tx=0x7f0580009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:59.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.146+0000 7f0587fff640 1 -- 172.21.15.27:0/3081523757 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f05880541f0 con 0x7f05900cb700 2023-08-11T15:51:59.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.338+0000 7f0595b42640 1 -- 172.21.15.27:0/3081523757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f059006c020 con 0x7f05900cb700 2023-08-11T15:51:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.339+0000 7f0587fff640 1 -- 172.21.15.27:0/3081523757 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0588057810 con 0x7f05900cb700 2023-08-11T15:51:59.342 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:51:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.342+0000 7f0595b42640 1 -- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f057006a050 msgr2=0x7f057006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.342+0000 7f0595b42640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f057006a050 0x7f057006c510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f05800012e0 tx=0x7f0580009040 comp rx=0 tx=0).stop 2023-08-11T15:51:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.342+0000 7f0595b42640 1 -- 172.21.15.27:0/3081523757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 msgr2=0x7f05900701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.342+0000 7f0595b42640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 0x7f05900701d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f058800d800 tx=0x7f058800dcc0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.343+0000 7f0595b42640 1 -- 172.21.15.27:0/3081523757 shutdown_connections 2023-08-11T15:51:59.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.343+0000 7f0595b42640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f057006a050 0x7f057006c510 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.343+0000 7f0595b42640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05900ac160 0x7f05900b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.343+0000 7f0595b42640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05900cb700 0x7f05900701d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.343+0000 7f0595b42640 1 --2- 172.21.15.27:0/3081523757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05900abd80 0x7f05901545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.343+0000 7f0595b42640 1 -- 172.21.15.27:0/3081523757 >> 172.21.15.27:0/3081523757 conn(0x7f0590069bf0 msgr2=0x7f05900cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:59.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.343+0000 7f0595b42640 1 -- 172.21.15.27:0/3081523757 shutdown_connections 2023-08-11T15:51:59.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.343+0000 7f0595b42640 1 -- 172.21.15.27:0/3081523757 wait complete. 2023-08-11T15:51:59.365 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:51:59.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.603+0000 7f6807da4640 1 Processor -- start 2023-08-11T15:51:59.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.603+0000 7f6807da4640 1 -- start start 2023-08-11T15:51:59.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.605+0000 7f6807da4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f68000cb700 0x7f680015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.605+0000 7f6807da4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6800071ad0 con 0x7f68000abd80 2023-08-11T15:51:59.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.605+0000 7f6807da4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6800160190 con 0x7f68000ac160 2023-08-11T15:51:59.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.605+0000 7f6807da4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68000c9720 con 0x7f68000cb700 2023-08-11T15:51:59.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.605+0000 7f6805b40640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f68000abd80 0x7f68000caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51754/0 (socket says 172.21.15.27:51754) 2023-08-11T15:51:59.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.605+0000 7f6805b40640 1 -- 172.21.15.27:0/2993293076 learned_addr learned my addr 172.21.15.27:0/2993293076 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:51:59.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.605+0000 7f680533f640 1 --2- 172.21.15.27:0/2993293076 >> v2:172.21.15.27:3300/0 conn(0x7f68000cb700 0x7f680015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.606+0000 7f680533f640 1 -- 172.21.15.27:0/2993293076 >> v1:172.21.15.119:6789/0 conn(0x7f68000ac160 legacy=0x7f680015d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:59.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.606+0000 7f6804b3e640 1 -- 172.21.15.27:0/2993293076 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789744408 0 0) 0x7f6800071ad0 con 0x7f68000abd80 2023-08-11T15:51:59.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.606+0000 7f680533f640 1 -- 172.21.15.27:0/2993293076 >> v1:172.21.15.27:6789/0 conn(0x7f68000abd80 legacy=0x7f68000caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.606+0000 7f680533f640 1 -- 172.21.15.27:0/2993293076 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680015d940 con 0x7f68000cb700 2023-08-11T15:51:59.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.607+0000 7f680533f640 1 --2- 172.21.15.27:0/2993293076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000cb700 0x7f680015cba0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67f00097d0 tx=0x7f67f002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b711f2689a3e54ae server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:59.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.607+0000 7f6804b3e640 1 -- 172.21.15.27:0/2993293076 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0031780 con 0x7f68000cb700 2023-08-11T15:51:59.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.608+0000 7f6804b3e640 1 -- 172.21.15.27:0/2993293076 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f00318e0 con 0x7f68000cb700 2023-08-11T15:51:59.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.608+0000 7f6804b3e640 1 -- 172.21.15.27:0/2993293076 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0031c70 con 0x7f68000cb700 2023-08-11T15:51:59.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.608+0000 7f6807da4640 1 -- 172.21.15.27:0/2993293076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000cb700 msgr2=0x7f680015cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.608+0000 7f6807da4640 1 --2- 172.21.15.27:0/2993293076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000cb700 0x7f680015cba0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67f00097d0 tx=0x7f67f002b820 comp rx=0 tx=0).stop 2023-08-11T15:51:59.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.608+0000 7f6807da4640 1 -- 172.21.15.27:0/2993293076 shutdown_connections 2023-08-11T15:51:59.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.608+0000 7f6807da4640 1 --2- 172.21.15.27:0/2993293076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000cb700 0x7f680015cba0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.608+0000 7f6807da4640 1 -- 172.21.15.27:0/2993293076 >> 172.21.15.27:0/2993293076 conn(0x7f6800069bf0 msgr2=0x7f680006a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:59.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.608+0000 7f6807da4640 1 -- 172.21.15.27:0/2993293076 shutdown_connections 2023-08-11T15:51:59.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.608+0000 7f6807da4640 1 -- 172.21.15.27:0/2993293076 wait complete. 2023-08-11T15:51:59.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.610+0000 7f6807da4640 1 Processor -- start 2023-08-11T15:51:59.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.610+0000 7f6807da4640 1 -- start start 2023-08-11T15:51:59.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.611+0000 7f6807da4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000abd80 0x7f68001f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.611+0000 7f6807da4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68000ac160 0x7f68000af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.611+0000 7f6807da4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68000cb700 0x7f68000cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.611+0000 7f6807da4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68000aefc0 con 0x7f68000abd80 2023-08-11T15:51:59.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.611+0000 7f6807da4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68000b35a0 con 0x7f68000ac160 2023-08-11T15:51:59.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.611+0000 7f6807da4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68000ae990 con 0x7f68000cb700 2023-08-11T15:51:59.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.611+0000 7f6805b40640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000abd80 0x7f68001f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.612+0000 7f6805b40640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000abd80 0x7f68001f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57076/0 (socket says 172.21.15.27:57076) 2023-08-11T15:51:59.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.612+0000 7f6805b40640 1 -- 172.21.15.27:0/1773617120 learned_addr learned my addr 172.21.15.27:0/1773617120 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:51:59.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.612+0000 7f680533f640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68000ac160 0x7f68000af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.612+0000 7f6806341640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68000cb700 0x7f68000cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.612+0000 7f6805b40640 1 -- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68000cb700 msgr2=0x7f68000cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.612+0000 7f6805b40640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68000cb700 0x7f68000cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.612+0000 7f6805b40640 1 -- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68000ac160 msgr2=0x7f68000af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.612+0000 7f6805b40640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68000ac160 0x7f68000af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.612+0000 7f6805b40640 1 -- 172.21.15.27:0/1773617120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0009400 con 0x7f68000abd80 2023-08-11T15:51:59.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.613+0000 7f6805b40640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000abd80 0x7f68001f1950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c4d0 tx=0x7f67fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:59.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.613+0000 7f67f67fc640 1 -- 172.21.15.27:0/1773617120 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0074e0 con 0x7f68000abd80 2023-08-11T15:51:59.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.613+0000 7f67f67fc640 1 -- 172.21.15.27:0/1773617120 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc007660 con 0x7f68000abd80 2023-08-11T15:51:59.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.613+0000 7f6807da4640 1 -- 172.21.15.27:0/1773617120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68001f1e90 con 0x7f68000abd80 2023-08-11T15:51:59.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.613+0000 7f67f67fc640 1 -- 172.21.15.27:0/1773617120 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc004a20 con 0x7f68000abd80 2023-08-11T15:51:59.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.613+0000 7f6807da4640 1 -- 172.21.15.27:0/1773617120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68000ce880 con 0x7f68000abd80 2023-08-11T15:51:59.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.614+0000 7f6807da4640 1 -- 172.21.15.27:0/1773617120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680004e420 con 0x7f68000abd80 2023-08-11T15:51:59.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.616+0000 7f67f67fc640 1 -- 172.21.15.27:0/1773617120 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f67fc004b80 con 0x7f68000abd80 2023-08-11T15:51:59.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.617+0000 7f67f67fc640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67e406a050 0x7f67e406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:59.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.617+0000 7f67f67fc640 1 -- 172.21.15.27:0/1773617120 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f67fc087d40 con 0x7f68000abd80 2023-08-11T15:51:59.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.618+0000 7f680533f640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67e406a050 0x7f67e406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:59.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.619+0000 7f680533f640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67e406a050 0x7f67e406c510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f67f0036000 tx=0x7f67f003e040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.622+0000 7f67f67fc640 1 -- 172.21.15.27:0/1773617120 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f67fc0531a0 con 0x7f68000abd80 2023-08-11T15:51:59.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.816+0000 7f6807da4640 1 -- 172.21.15.27:0/1773617120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f680006c020 con 0x7f68000abd80 2023-08-11T15:51:59.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.817+0000 7f67f67fc640 1 -- 172.21.15.27:0/1773617120 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f67fc0180a0 con 0x7f68000abd80 2023-08-11T15:51:59.820 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:51:59.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 -- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67e406a050 msgr2=0x7f67e406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67e406a050 0x7f67e406c510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f67f0036000 tx=0x7f67f003e040 comp rx=0 tx=0).stop 2023-08-11T15:51:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 -- 172.21.15.27:0/1773617120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000abd80 msgr2=0x7f68001f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000abd80 0x7f68001f1950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c4d0 tx=0x7f67fc00c990 comp rx=0 tx=0).stop 2023-08-11T15:51:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 -- 172.21.15.27:0/1773617120 shutdown_connections 2023-08-11T15:51:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f67e406a050 0x7f67e406c510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68000cb700 0x7f68000cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68000ac160 0x7f68000af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 --2- 172.21.15.27:0/1773617120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68000abd80 0x7f68001f1950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 -- 172.21.15.27:0/1773617120 >> 172.21.15.27:0/1773617120 conn(0x7f6800069bf0 msgr2=0x7f68000702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 -- 172.21.15.27:0/1773617120 shutdown_connections 2023-08-11T15:51:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:51:59.820+0000 7f6807da4640 1 -- 172.21.15.27:0/1773617120 wait complete. 2023-08-11T15:51:59.843 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.072+0000 7f311b463640 1 Processor -- start 2023-08-11T15:52:00.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.072+0000 7f311b463640 1 -- start start 2023-08-11T15:52:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.073+0000 7f311b463640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a0f0 0x7f31141435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.073+0000 7f311b463640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f311406f5d0 0x7f311406fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.074+0000 7f311b463640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311406c500 con 0x7f311415a4d0 2023-08-11T15:52:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.074+0000 7f311b463640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311406ff80 con 0x7f311406f5d0 2023-08-11T15:52:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.074+0000 7f311b463640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311406b8f0 con 0x7f311415a0f0 2023-08-11T15:52:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.074+0000 7f311a461640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a0f0 0x7f31141435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.074+0000 7f311ac62640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f311406f5d0 0x7f311406fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.074+0000 7f311a461640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a0f0 0x7f31141435e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:34590/0 (socket says 172.21.15.27:34590) 2023-08-11T15:52:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.074+0000 7f311a461640 1 -- 172.21.15.27:0/1797422823 learned_addr learned my addr 172.21.15.27:0/1797422823 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.075+0000 7f311ac62640 1 -- 172.21.15.27:0/1797422823 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a0f0 msgr2=0x7f31141435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.075+0000 7f311ac62640 1 --2- 172.21.15.27:0/1797422823 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a0f0 0x7f31141435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.075+0000 7f311ac62640 1 -- 172.21.15.27:0/1797422823 >> v1:172.21.15.119:6789/0 conn(0x7f311415a4d0 legacy=0x7f3114071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.076+0000 7f311ac62640 1 -- 172.21.15.27:0/1797422823 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311406bbd0 con 0x7f311406f5d0 2023-08-11T15:52:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.076+0000 7f311a461640 1 --2- 172.21.15.27:0/1797422823 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a0f0 0x7f31141435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.077+0000 7f311ac62640 1 --2- 172.21.15.27:0/1797422823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f311406f5d0 0x7f311406fa40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f310c009d90 tx=0x7f310c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11e76a8398666887 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.078+0000 7f311945f640 1 -- 172.21.15.27:0/1797422823 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c02cc50 con 0x7f311406f5d0 2023-08-11T15:52:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.078+0000 7f311945f640 1 -- 172.21.15.27:0/1797422823 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c02cdb0 con 0x7f311406f5d0 2023-08-11T15:52:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.078+0000 7f311945f640 1 -- 172.21.15.27:0/1797422823 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c034660 con 0x7f311406f5d0 2023-08-11T15:52:00.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.079+0000 7f311b463640 1 -- 172.21.15.27:0/1797422823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f311406f5d0 msgr2=0x7f311406fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.079+0000 7f311b463640 1 --2- 172.21.15.27:0/1797422823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f311406f5d0 0x7f311406fa40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f310c009d90 tx=0x7f310c02bea0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.079+0000 7f311b463640 1 -- 172.21.15.27:0/1797422823 shutdown_connections 2023-08-11T15:52:00.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.079+0000 7f311b463640 1 --2- 172.21.15.27:0/1797422823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f311406f5d0 0x7f311406fa40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.079+0000 7f311b463640 1 --2- 172.21.15.27:0/1797422823 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a0f0 0x7f31141435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.079+0000 7f311b463640 1 -- 172.21.15.27:0/1797422823 >> 172.21.15.27:0/1797422823 conn(0x7f31140caab0 msgr2=0x7f31140cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:00.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.079+0000 7f311b463640 1 -- 172.21.15.27:0/1797422823 shutdown_connections 2023-08-11T15:52:00.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.079+0000 7f311b463640 1 -- 172.21.15.27:0/1797422823 wait complete. 2023-08-11T15:52:00.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.081+0000 7f311b463640 1 Processor -- start 2023-08-11T15:52:00.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.081+0000 7f311b463640 1 -- start start 2023-08-11T15:52:00.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.082+0000 7f311b463640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f311406f5d0 0x7f31140af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.082+0000 7f311b463640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f311415a0f0 0x7f311414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.082+0000 7f311b463640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a4d0 0x7f31140ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.082+0000 7f311b463640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311414ebd0 con 0x7f311415a0f0 2023-08-11T15:52:00.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.082+0000 7f311b463640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31140ae120 con 0x7f311406f5d0 2023-08-11T15:52:00.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.082+0000 7f311b463640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31140ae400 con 0x7f311415a4d0 2023-08-11T15:52:00.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.082+0000 7f3119c60640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f311415a0f0 0x7f311414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.082+0000 7f3119c60640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f311415a0f0 0x7f311414e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57088/0 (socket says 172.21.15.27:57088) 2023-08-11T15:52:00.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.082+0000 7f3119c60640 1 -- 172.21.15.27:0/1010932204 learned_addr learned my addr 172.21.15.27:0/1010932204 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:00.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f311a461640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f311406f5d0 0x7f31140af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f311ac62640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a4d0 0x7f31140ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f3119c60640 1 -- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a4d0 msgr2=0x7f31140ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f3119c60640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a4d0 0x7f31140ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f3119c60640 1 -- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f311406f5d0 msgr2=0x7f31140af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f3119c60640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f311406f5d0 0x7f31140af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f3119c60640 1 -- 172.21.15.27:0/1010932204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31100094d0 con 0x7f311415a0f0 2023-08-11T15:52:00.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f3119c60640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f311415a0f0 0x7f311414e690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f310400c3c0 tx=0x7f310400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:00.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f310affd640 1 -- 172.21.15.27:0/1010932204 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31040052d0 con 0x7f311415a0f0 2023-08-11T15:52:00.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f310affd640 1 -- 172.21.15.27:0/1010932204 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3104005450 con 0x7f311415a0f0 2023-08-11T15:52:00.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f310affd640 1 -- 172.21.15.27:0/1010932204 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104007260 con 0x7f311415a0f0 2023-08-11T15:52:00.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.083+0000 7f311b463640 1 -- 172.21.15.27:0/1010932204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c009a70 con 0x7f311415a0f0 2023-08-11T15:52:00.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.084+0000 7f311b463640 1 -- 172.21.15.27:0/1010932204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31140ae9c0 con 0x7f311415a0f0 2023-08-11T15:52:00.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.085+0000 7f311b463640 1 -- 172.21.15.27:0/1010932204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311404e420 con 0x7f311415a0f0 2023-08-11T15:52:00.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.086+0000 7f310affd640 1 -- 172.21.15.27:0/1010932204 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f31040073c0 con 0x7f311415a0f0 2023-08-11T15:52:00.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.087+0000 7f310affd640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f30f806a050 0x7f30f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.087+0000 7f310affd640 1 -- 172.21.15.27:0/1010932204 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3104087c60 con 0x7f311415a0f0 2023-08-11T15:52:00.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.088+0000 7f311a461640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f30f806a050 0x7f30f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.089+0000 7f311a461640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f30f806a050 0x7f30f806c510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3110009cf0 tx=0x7f3110008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:00.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.091+0000 7f310affd640 1 -- 172.21.15.27:0/1010932204 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f3104053010 con 0x7f311415a0f0 2023-08-11T15:52:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.288+0000 7f311b463640 1 -- 172.21.15.27:0/1010932204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31140b3fb0 con 0x7f311415a0f0 2023-08-11T15:52:00.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.289+0000 7f310affd640 1 -- 172.21.15.27:0/1010932204 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3104056630 con 0x7f311415a0f0 2023-08-11T15:52:00.291 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 -- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f30f806a050 msgr2=0x7f30f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f30f806a050 0x7f30f806c510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3110009cf0 tx=0x7f3110008040 comp rx=0 tx=0).stop 2023-08-11T15:52:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 -- 172.21.15.27:0/1010932204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f311415a0f0 msgr2=0x7f311414e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f311415a0f0 0x7f311414e690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f310400c3c0 tx=0x7f310400c880 comp rx=0 tx=0).stop 2023-08-11T15:52:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 -- 172.21.15.27:0/1010932204 shutdown_connections 2023-08-11T15:52:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f30f806a050 0x7f30f806c510 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f311415a4d0 0x7f31140ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f311415a0f0 0x7f311414e690 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 --2- 172.21.15.27:0/1010932204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f311406f5d0 0x7f31140af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 -- 172.21.15.27:0/1010932204 >> 172.21.15.27:0/1010932204 conn(0x7f31140caab0 msgr2=0x7f31140b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 -- 172.21.15.27:0/1010932204 shutdown_connections 2023-08-11T15:52:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.291+0000 7f311b463640 1 -- 172.21.15.27:0/1010932204 wait complete. 2023-08-11T15:52:00.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:00.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.542+0000 7ff624d4e640 1 Processor -- start 2023-08-11T15:52:00.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.542+0000 7ff624d4e640 1 -- start start 2023-08-11T15:52:00.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.544+0000 7ff624d4e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff620151600 0x7ff62015df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.544+0000 7ff624d4e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff62015e4b0 0x7ff6200ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.544+0000 7ff624d4e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62006c850 con 0x7ff620151220 2023-08-11T15:52:00.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.544+0000 7ff624d4e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62015e940 con 0x7ff62015e4b0 2023-08-11T15:52:00.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.544+0000 7ff624d4e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62006bfc0 con 0x7ff620151600 2023-08-11T15:52:00.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.544+0000 7ff61ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff62015e4b0 0x7ff6200ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.544+0000 7ff61ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff62015e4b0 0x7ff6200ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57096/0 (socket says 172.21.15.27:57096) 2023-08-11T15:52:00.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.544+0000 7ff61ed9d640 1 -- 172.21.15.27:0/2883881813 learned_addr learned my addr 172.21.15.27:0/2883881813 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:00.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.544+0000 7ff61e59c640 1 --1- 172.21.15.27:0/2883881813 >> v1:172.21.15.27:6789/0 conn(0x7ff620151220 0x7ff62006b8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51768/0 (socket says 172.21.15.27:51768) 2023-08-11T15:52:00.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.545+0000 7ff61dd9b640 1 --2- 172.21.15.27:0/2883881813 >> v2:172.21.15.119:3300/0 conn(0x7ff620151600 0x7ff62015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.545+0000 7ff61ed9d640 1 -- 172.21.15.27:0/2883881813 >> v2:172.21.15.119:3300/0 conn(0x7ff620151600 msgr2=0x7ff62015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:00.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.545+0000 7ff61ed9d640 1 --2- 172.21.15.27:0/2883881813 >> v2:172.21.15.119:3300/0 conn(0x7ff620151600 0x7ff62015df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.545+0000 7ff61ed9d640 1 -- 172.21.15.27:0/2883881813 >> v1:172.21.15.27:6789/0 conn(0x7ff620151220 legacy=0x7ff62006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.545+0000 7ff61ed9d640 1 -- 172.21.15.27:0/2883881813 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62006c290 con 0x7ff62015e4b0 2023-08-11T15:52:00.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.546+0000 7ff61ed9d640 1 --2- 172.21.15.27:0/2883881813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff62015e4b0 0x7ff6200ca850 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff6200b0ca0 tx=0x7ff61802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77755f2797d6d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:00.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.546+0000 7ff61d59a640 1 -- 172.21.15.27:0/2883881813 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6180317e0 con 0x7ff62015e4b0 2023-08-11T15:52:00.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.546+0000 7ff61d59a640 1 -- 172.21.15.27:0/2883881813 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff618031940 con 0x7ff62015e4b0 2023-08-11T15:52:00.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.546+0000 7ff61d59a640 1 -- 172.21.15.27:0/2883881813 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618031cd0 con 0x7ff62015e4b0 2023-08-11T15:52:00.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.547+0000 7ff624d4e640 1 -- 172.21.15.27:0/2883881813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff62015e4b0 msgr2=0x7ff6200ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.547+0000 7ff624d4e640 1 --2- 172.21.15.27:0/2883881813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff62015e4b0 0x7ff6200ca850 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff6200b0ca0 tx=0x7ff61802b880 comp rx=0 tx=0).stop 2023-08-11T15:52:00.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.547+0000 7ff624d4e640 1 -- 172.21.15.27:0/2883881813 shutdown_connections 2023-08-11T15:52:00.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.547+0000 7ff624d4e640 1 --2- 172.21.15.27:0/2883881813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff62015e4b0 0x7ff6200ca850 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.547+0000 7ff624d4e640 1 --2- 172.21.15.27:0/2883881813 >> v2:172.21.15.119:3300/0 conn(0x7ff620151600 0x7ff62015df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.547+0000 7ff624d4e640 1 -- 172.21.15.27:0/2883881813 >> 172.21.15.27:0/2883881813 conn(0x7ff6200afbe0 msgr2=0x7ff6200affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:00.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.547+0000 7ff624d4e640 1 -- 172.21.15.27:0/2883881813 shutdown_connections 2023-08-11T15:52:00.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.547+0000 7ff624d4e640 1 -- 172.21.15.27:0/2883881813 wait complete. 2023-08-11T15:52:00.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.549+0000 7ff624d4e640 1 Processor -- start 2023-08-11T15:52:00.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.549+0000 7ff624d4e640 1 -- start start 2023-08-11T15:52:00.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.550+0000 7ff624d4e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff620151220 0x7ff6200a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.550+0000 7ff624d4e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff620151600 0x7ff6200c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.550+0000 7ff624d4e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015e4b0 0x7ff62015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.550+0000 7ff624d4e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200c7e30 con 0x7ff620151220 2023-08-11T15:52:00.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.550+0000 7ff624d4e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200c8040 con 0x7ff62015e4b0 2023-08-11T15:52:00.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.550+0000 7ff624d4e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200c8320 con 0x7ff620151600 2023-08-11T15:52:00.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.550+0000 7ff61e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff620151220 0x7ff6200a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.550+0000 7ff61e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff620151220 0x7ff6200a74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57108/0 (socket says 172.21.15.27:57108) 2023-08-11T15:52:00.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.550+0000 7ff61e59c640 1 -- 172.21.15.27:0/4140795887 learned_addr learned my addr 172.21.15.27:0/4140795887 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:00.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff61dd9b640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff620151600 0x7ff6200c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff61e59c640 1 -- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff620151600 msgr2=0x7ff6200c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff61ed9d640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015e4b0 0x7ff62015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff61e59c640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff620151600 0x7ff6200c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff61e59c640 1 -- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015e4b0 msgr2=0x7ff62015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff61e59c640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015e4b0 0x7ff62015c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff61e59c640 1 -- 172.21.15.27:0/4140795887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6180094d0 con 0x7ff620151220 2023-08-11T15:52:00.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff61e59c640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff620151220 0x7ff6200a74d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff61000c800 tx=0x7ff61000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff60effd640 1 -- 172.21.15.27:0/4140795887 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000a640 con 0x7ff620151220 2023-08-11T15:52:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff60effd640 1 -- 172.21.15.27:0/4140795887 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61000a7a0 con 0x7ff620151220 2023-08-11T15:52:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff60effd640 1 -- 172.21.15.27:0/4140795887 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000ab50 con 0x7ff620151220 2023-08-11T15:52:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.551+0000 7ff624d4e640 1 -- 172.21.15.27:0/4140795887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6200c8600 con 0x7ff620151220 2023-08-11T15:52:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.552+0000 7ff624d4e640 1 -- 172.21.15.27:0/4140795887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6200c8bc0 con 0x7ff620151220 2023-08-11T15:52:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.553+0000 7ff624d4e640 1 -- 172.21.15.27:0/4140795887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62004e420 con 0x7ff620151220 2023-08-11T15:52:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.554+0000 7ff60effd640 1 -- 172.21.15.27:0/4140795887 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff61000acb0 con 0x7ff620151220 2023-08-11T15:52:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.555+0000 7ff60effd640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff60006a050 0x7ff60006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.555+0000 7ff60effd640 1 -- 172.21.15.27:0/4140795887 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff610088140 con 0x7ff620151220 2023-08-11T15:52:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.555+0000 7ff61dd9b640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff60006a050 0x7ff60006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.556+0000 7ff61dd9b640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff60006a050 0x7ff60006c510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff6200ad9a0 tx=0x7ff614008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.559+0000 7ff60effd640 1 -- 172.21.15.27:0/4140795887 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ff610057030 con 0x7ff620151220 2023-08-11T15:52:00.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.755+0000 7ff624d4e640 1 -- 172.21.15.27:0/4140795887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6200cef20 con 0x7ff620151220 2023-08-11T15:52:00.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.756+0000 7ff60effd640 1 -- 172.21.15.27:0/4140795887 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6100018c0 con 0x7ff620151220 2023-08-11T15:52:00.758 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:00.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.758+0000 7ff624d4e640 1 -- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff60006a050 msgr2=0x7ff60006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.758+0000 7ff624d4e640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff60006a050 0x7ff60006c510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff6200ad9a0 tx=0x7ff614008040 comp rx=0 tx=0).stop 2023-08-11T15:52:00.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.758+0000 7ff624d4e640 1 -- 172.21.15.27:0/4140795887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff620151220 msgr2=0x7ff6200a74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:00.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.758+0000 7ff624d4e640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff620151220 0x7ff6200a74d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff61000c800 tx=0x7ff61000ccc0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.758+0000 7ff624d4e640 1 -- 172.21.15.27:0/4140795887 shutdown_connections 2023-08-11T15:52:00.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.758+0000 7ff624d4e640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff60006a050 0x7ff60006c510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.758+0000 7ff624d4e640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff620151600 0x7ff6200c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.758+0000 7ff624d4e640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015e4b0 0x7ff62015c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.758+0000 7ff624d4e640 1 --2- 172.21.15.27:0/4140795887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff620151220 0x7ff6200a74d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:00.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.758+0000 7ff624d4e640 1 -- 172.21.15.27:0/4140795887 >> 172.21.15.27:0/4140795887 conn(0x7ff6200afbe0 msgr2=0x7ff6200cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:00.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.759+0000 7ff624d4e640 1 -- 172.21.15.27:0/4140795887 shutdown_connections 2023-08-11T15:52:00.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:00.759+0000 7ff624d4e640 1 -- 172.21.15.27:0/4140795887 wait complete. 2023-08-11T15:52:00.780 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:01.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.016+0000 7f895c94e640 1 Processor -- start 2023-08-11T15:52:01.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.016+0000 7f895c94e640 1 -- start start 2023-08-11T15:52:01.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.017+0000 7f895c94e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89580b3090 0x7f89580d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.017+0000 7f895c94e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8958161210 0x7f89580b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.017+0000 7f895c94e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89580af250 con 0x7f89580b38f0 2023-08-11T15:52:01.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.017+0000 7f895c94e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89580b0190 con 0x7f89580b3090 2023-08-11T15:52:01.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.017+0000 7f895c94e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89580af510 con 0x7f8958161210 2023-08-11T15:52:01.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.018+0000 7f8955d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89580b3090 0x7f89580d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.018+0000 7f8955d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89580b3090 0x7f89580d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:43176/0 (socket says 172.21.15.27:43176) 2023-08-11T15:52:01.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.018+0000 7f8955d9b640 1 -- 172.21.15.27:0/1197384925 learned_addr learned my addr 172.21.15.27:0/1197384925 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:01.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.018+0000 7f8956d9d640 1 --2- 172.21.15.27:0/1197384925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8958161210 0x7f89580b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.019+0000 7f8955d9b640 1 -- 172.21.15.27:0/1197384925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8958161210 msgr2=0x7f89580b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:01.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.019+0000 7f8955d9b640 1 --2- 172.21.15.27:0/1197384925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8958161210 0x7f89580b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.019+0000 7f8955d9b640 1 -- 172.21.15.27:0/1197384925 >> v1:172.21.15.119:6789/0 conn(0x7f89580b38f0 legacy=0x7f8958160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.019+0000 7f8955d9b640 1 -- 172.21.15.27:0/1197384925 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89580af7f0 con 0x7f89580b3090 2023-08-11T15:52:01.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.020+0000 7f8955d9b640 1 --2- 172.21.15.27:0/1197384925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580b3090 0x7f89580d30e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8940009870 tx=0x7f894002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fa367b2834d5752 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.020+0000 7f895559a640 1 -- 172.21.15.27:0/1197384925 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940030840 con 0x7f89580b3090 2023-08-11T15:52:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.020+0000 7f895559a640 1 -- 172.21.15.27:0/1197384925 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89400309a0 con 0x7f89580b3090 2023-08-11T15:52:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.021+0000 7f895559a640 1 -- 172.21.15.27:0/1197384925 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940030d30 con 0x7f89580b3090 2023-08-11T15:52:01.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.021+0000 7f895c94e640 1 -- 172.21.15.27:0/1197384925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580b3090 msgr2=0x7f89580d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.021+0000 7f895c94e640 1 --2- 172.21.15.27:0/1197384925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580b3090 0x7f89580d30e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8940009870 tx=0x7f894002b980 comp rx=0 tx=0).stop 2023-08-11T15:52:01.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.021+0000 7f895c94e640 1 -- 172.21.15.27:0/1197384925 shutdown_connections 2023-08-11T15:52:01.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.021+0000 7f895c94e640 1 --2- 172.21.15.27:0/1197384925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8958161210 0x7f89580b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.021+0000 7f895c94e640 1 --2- 172.21.15.27:0/1197384925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580b3090 0x7f89580d30e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.021+0000 7f895c94e640 1 -- 172.21.15.27:0/1197384925 >> 172.21.15.27:0/1197384925 conn(0x7f895809b960 msgr2=0x7f89580d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:01.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.021+0000 7f895c94e640 1 -- 172.21.15.27:0/1197384925 shutdown_connections 2023-08-11T15:52:01.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.021+0000 7f895c94e640 1 -- 172.21.15.27:0/1197384925 wait complete. 2023-08-11T15:52:01.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.023+0000 7f895c94e640 1 Processor -- start 2023-08-11T15:52:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.023+0000 7f895c94e640 1 -- start start 2023-08-11T15:52:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.024+0000 7f895c94e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f89580b3090 0x7f89581f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.024+0000 7f895c94e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580b38f0 0x7f895815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.024+0000 7f895c94e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8958161210 0x7f89580cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.024+0000 7f895c94e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89581594c0 con 0x7f8958161210 2023-08-11T15:52:01.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.024+0000 7f895c94e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895815aaf0 con 0x7f89580b38f0 2023-08-11T15:52:01.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.024+0000 7f895c94e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8958158bd0 con 0x7f89580b3090 2023-08-11T15:52:01.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f8956d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8958161210 0x7f89580cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f8956d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8958161210 0x7f89580cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57124/0 (socket says 172.21.15.27:57124) 2023-08-11T15:52:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f8956d9d640 1 -- 172.21.15.27:0/3296151292 learned_addr learned my addr 172.21.15.27:0/3296151292 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f8955d9b640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580b38f0 0x7f895815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f895659c640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f89580b3090 0x7f89581f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f8956d9d640 1 -- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f89580b3090 msgr2=0x7f89581f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f8956d9d640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f89580b3090 0x7f89581f2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f8956d9d640 1 -- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580b38f0 msgr2=0x7f895815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f8956d9d640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580b38f0 0x7f895815a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f8956d9d640 1 -- 172.21.15.27:0/3296151292 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89400094d0 con 0x7f8958161210 2023-08-11T15:52:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.025+0000 7f895659c640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f89580b3090 0x7f89581f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.026+0000 7f8956d9d640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8958161210 0x7f89580cd3b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f895000e7c0 tx=0x7f895000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.026+0000 7f894effd640 1 -- 172.21.15.27:0/3296151292 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89500094b0 con 0x7f8958161210 2023-08-11T15:52:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.026+0000 7f894effd640 1 -- 172.21.15.27:0/3296151292 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895000c730 con 0x7f8958161210 2023-08-11T15:52:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.026+0000 7f894effd640 1 -- 172.21.15.27:0/3296151292 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000cae0 con 0x7f8958161210 2023-08-11T15:52:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.026+0000 7f895c94e640 1 -- 172.21.15.27:0/3296151292 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895815a850 con 0x7f8958161210 2023-08-11T15:52:01.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.026+0000 7f895c94e640 1 -- 172.21.15.27:0/3296151292 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8958159150 con 0x7f8958161210 2023-08-11T15:52:01.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.027+0000 7f895c94e640 1 -- 172.21.15.27:0/3296151292 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8958159630 con 0x7f8958161210 2023-08-11T15:52:01.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.028+0000 7f894effd640 1 -- 172.21.15.27:0/3296151292 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f895000cc40 con 0x7f8958161210 2023-08-11T15:52:01.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.029+0000 7f894effd640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f893806a080 0x7f893806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.029+0000 7f894effd640 1 -- 172.21.15.27:0/3296151292 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8950088bd0 con 0x7f8958161210 2023-08-11T15:52:01.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.030+0000 7f895659c640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f893806a080 0x7f893806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.031+0000 7f895659c640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f893806a080 0x7f893806c540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8948001330 tx=0x7f8948009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:01.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.034+0000 7f894effd640 1 -- 172.21.15.27:0/3296151292 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f8950054030 con 0x7f8958161210 2023-08-11T15:52:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.230+0000 7f895c94e640 1 -- 172.21.15.27:0/3296151292 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f895804e420 con 0x7f8958161210 2023-08-11T15:52:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.231+0000 7f894effd640 1 -- 172.21.15.27:0/3296151292 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8950057650 con 0x7f8958161210 2023-08-11T15:52:01.234 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:01.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.233+0000 7f895c94e640 1 -- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f893806a080 msgr2=0x7f893806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.233+0000 7f895c94e640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f893806a080 0x7f893806c540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8948001330 tx=0x7f8948009040 comp rx=0 tx=0).stop 2023-08-11T15:52:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.233+0000 7f895c94e640 1 -- 172.21.15.27:0/3296151292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8958161210 msgr2=0x7f89580cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.233+0000 7f895c94e640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8958161210 0x7f89580cd3b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f895000e7c0 tx=0x7f895000ec80 comp rx=0 tx=0).stop 2023-08-11T15:52:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.233+0000 7f895c94e640 1 -- 172.21.15.27:0/3296151292 shutdown_connections 2023-08-11T15:52:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.233+0000 7f895c94e640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f893806a080 0x7f893806c540 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.233+0000 7f895c94e640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8958161210 0x7f89580cd3b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.233+0000 7f895c94e640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580b38f0 0x7f895815a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.233+0000 7f895c94e640 1 --2- 172.21.15.27:0/3296151292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f89580b3090 0x7f89581f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.233+0000 7f895c94e640 1 -- 172.21.15.27:0/3296151292 >> 172.21.15.27:0/3296151292 conn(0x7f895809b960 msgr2=0x7f89580d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.234+0000 7f895c94e640 1 -- 172.21.15.27:0/3296151292 shutdown_connections 2023-08-11T15:52:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.234+0000 7f895c94e640 1 -- 172.21.15.27:0/3296151292 wait complete. 2023-08-11T15:52:01.256 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.484+0000 7f6f7e2aa640 1 Processor -- start 2023-08-11T15:52:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.484+0000 7f6f7e2aa640 1 -- start start 2023-08-11T15:52:01.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7e2aa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f780d1b90 0x7f6f780b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7e2aa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7815c970 con 0x7f6f780cd0e0 2023-08-11T15:52:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7e2aa640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f780d0250 con 0x7f6f7815c590 2023-08-11T15:52:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7e2aa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f780d0ed0 con 0x7f6f780d1b90 2023-08-11T15:52:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7caa7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f780d1b90 0x7f6f780b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7caa7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f780d1b90 0x7f6f780b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57130/0 (socket says 172.21.15.27:57130) 2023-08-11T15:52:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7caa7640 1 -- 172.21.15.27:0/3225283189 learned_addr learned my addr 172.21.15.27:0/3225283189 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7daa9640 1 --1- 172.21.15.27:0/3225283189 >> v1:172.21.15.27:6789/0 conn(0x7f6f780cd0e0 0x7f6f78160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51782/0 (socket says 172.21.15.27:51782) 2023-08-11T15:52:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7caa7640 1 -- 172.21.15.27:0/3225283189 >> v1:172.21.15.119:6789/0 conn(0x7f6f7815c590 legacy=0x7f6f780d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7caa7640 1 -- 172.21.15.27:0/3225283189 >> v1:172.21.15.27:6789/0 conn(0x7f6f780cd0e0 legacy=0x7f6f78160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.486+0000 7f6f7caa7640 1 -- 172.21.15.27:0/3225283189 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f780d0530 con 0x7f6f780d1b90 2023-08-11T15:52:01.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.487+0000 7f6f7caa7640 1 --2- 172.21.15.27:0/3225283189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780d1b90 0x7f6f780b5440 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009870 tx=0x7f6f6802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d163504158ca1eb4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:01.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.488+0000 7f6f6ffff640 1 -- 172.21.15.27:0/3225283189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f680317e0 con 0x7f6f780d1b90 2023-08-11T15:52:01.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.488+0000 7f6f6ffff640 1 -- 172.21.15.27:0/3225283189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f68031940 con 0x7f6f780d1b90 2023-08-11T15:52:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.488+0000 7f6f6ffff640 1 -- 172.21.15.27:0/3225283189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68031cd0 con 0x7f6f780d1b90 2023-08-11T15:52:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.488+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/3225283189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780d1b90 msgr2=0x7f6f780b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.488+0000 7f6f7e2aa640 1 --2- 172.21.15.27:0/3225283189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780d1b90 0x7f6f780b5440 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009870 tx=0x7f6f6802b880 comp rx=0 tx=0).stop 2023-08-11T15:52:01.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.488+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/3225283189 shutdown_connections 2023-08-11T15:52:01.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.488+0000 7f6f7e2aa640 1 --2- 172.21.15.27:0/3225283189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780d1b90 0x7f6f780b5440 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.488+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/3225283189 >> 172.21.15.27:0/3225283189 conn(0x7f6f7806a250 msgr2=0x7f6f7806a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:01.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.489+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/3225283189 shutdown_connections 2023-08-11T15:52:01.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.489+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/3225283189 wait complete. 2023-08-11T15:52:01.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.490+0000 7f6f7e2aa640 1 Processor -- start 2023-08-11T15:52:01.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.491+0000 7f6f7e2aa640 1 -- start start 2023-08-11T15:52:01.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.491+0000 7f6f7e2aa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd0e0 0x7f6f781557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.491+0000 7f6f7e2aa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f780d1b90 0x7f6f7815f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.491+0000 7f6f7e2aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f7815c590 0x7f6f7815f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.491+0000 7f6f7e2aa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f781560e0 con 0x7f6f780cd0e0 2023-08-11T15:52:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7e2aa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78156380 con 0x7f6f7815c590 2023-08-11T15:52:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7e2aa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78156660 con 0x7f6f780d1b90 2023-08-11T15:52:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7d2a8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd0e0 0x7f6f781557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7d2a8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd0e0 0x7f6f781557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57142/0 (socket says 172.21.15.27:57142) 2023-08-11T15:52:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7d2a8640 1 -- 172.21.15.27:0/4139818656 learned_addr learned my addr 172.21.15.27:0/4139818656 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7caa7640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f780d1b90 0x7f6f7815f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7daa9640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f7815c590 0x7f6f7815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7d2a8640 1 -- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f780d1b90 msgr2=0x7f6f7815f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7d2a8640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f780d1b90 0x7f6f7815f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7d2a8640 1 -- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f7815c590 msgr2=0x7f6f7815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7d2a8640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f7815c590 0x7f6f7815f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7d2a8640 1 -- 172.21.15.27:0/4139818656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680094d0 con 0x7f6f780cd0e0 2023-08-11T15:52:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.492+0000 7f6f7daa9640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f7815c590 0x7f6f7815f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.493+0000 7f6f7d2a8640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd0e0 0x7f6f781557c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400b3f0 tx=0x7f6f7400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.493+0000 7f6f6dffb640 1 -- 172.21.15.27:0/4139818656 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f74011830 con 0x7f6f780cd0e0 2023-08-11T15:52:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.493+0000 7f6f6dffb640 1 -- 172.21.15.27:0/4139818656 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f74011990 con 0x7f6f780cd0e0 2023-08-11T15:52:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.493+0000 7f6f6dffb640 1 -- 172.21.15.27:0/4139818656 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f74011d40 con 0x7f6f780cd0e0 2023-08-11T15:52:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.493+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/4139818656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f78156940 con 0x7f6f780cd0e0 2023-08-11T15:52:01.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.493+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/4139818656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f78155d00 con 0x7f6f780cd0e0 2023-08-11T15:52:01.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.494+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/4139818656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f780cf670 con 0x7f6f780cd0e0 2023-08-11T15:52:01.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.495+0000 7f6f6dffb640 1 -- 172.21.15.27:0/4139818656 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6f7401a400 con 0x7f6f780cd0e0 2023-08-11T15:52:01.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.496+0000 7f6f6dffb640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f5c06a050 0x7f6f5c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.496+0000 7f6f6dffb640 1 -- 172.21.15.27:0/4139818656 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6f74088740 con 0x7f6f780cd0e0 2023-08-11T15:52:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.497+0000 7f6f7caa7640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f5c06a050 0x7f6f5c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.498+0000 7f6f7caa7640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f5c06a050 0x7f6f5c06c510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6f68036000 tx=0x7f6f6803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.500+0000 7f6f6dffb640 1 -- 172.21.15.27:0/4139818656 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6f74053af0 con 0x7f6f780cd0e0 2023-08-11T15:52:01.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.697+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/4139818656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f7804e420 con 0x7f6f780cd0e0 2023-08-11T15:52:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.700+0000 7f6f6dffb640 1 -- 172.21.15.27:0/4139818656 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f6f74057110 con 0x7f6f780cd0e0 2023-08-11T15:52:01.703 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:01.703 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.703+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f5c06a050 msgr2=0x7f6f5c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.703+0000 7f6f7e2aa640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f5c06a050 0x7f6f5c06c510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6f68036000 tx=0x7f6f6803d040 comp rx=0 tx=0).stop 2023-08-11T15:52:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.703+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/4139818656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd0e0 msgr2=0x7f6f781557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.703+0000 7f6f7e2aa640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd0e0 0x7f6f781557c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400b3f0 tx=0x7f6f7400b8b0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.703+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/4139818656 shutdown_connections 2023-08-11T15:52:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.703+0000 7f6f7e2aa640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f5c06a050 0x7f6f5c06c510 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.703+0000 7f6f7e2aa640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f780d1b90 0x7f6f7815f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.703+0000 7f6f7e2aa640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f7815c590 0x7f6f7815f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.703+0000 7f6f7e2aa640 1 --2- 172.21.15.27:0/4139818656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd0e0 0x7f6f781557c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.703+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/4139818656 >> 172.21.15.27:0/4139818656 conn(0x7f6f7806a250 msgr2=0x7f6f7806a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:01.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.704+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/4139818656 shutdown_connections 2023-08-11T15:52:01.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.704+0000 7f6f7e2aa640 1 -- 172.21.15.27:0/4139818656 wait complete. 2023-08-11T15:52:01.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:01.727 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.945+0000 7f264450f640 1 Processor -- start 2023-08-11T15:52:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.945+0000 7f264450f640 1 -- start start 2023-08-11T15:52:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.947+0000 7f264450f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f263c0cb700 0x7f263c13fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.947+0000 7f264450f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f263c15b6b0 0x7f263c1401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.947+0000 7f264450f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c140ba0 con 0x7f263c154ac0 2023-08-11T15:52:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.947+0000 7f264450f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263c071b10 con 0x7f263c0cb700 2023-08-11T15:52:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.947+0000 7f264450f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263c15b220 con 0x7f263c15b6b0 2023-08-11T15:52:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.948+0000 7f2641aaa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f263c0cb700 0x7f263c13fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.948+0000 7f2641aaa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f263c0cb700 0x7f263c13fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57144/0 (socket says 172.21.15.27:57144) 2023-08-11T15:52:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.948+0000 7f2641aaa640 1 -- 172.21.15.27:0/1260094867 learned_addr learned my addr 172.21.15.27:0/1260094867 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.948+0000 7f2642aac640 1 --2- 172.21.15.27:0/1260094867 >> v2:172.21.15.119:3300/0 conn(0x7f263c15b6b0 0x7f263c1401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.948+0000 7f2641aaa640 1 -- 172.21.15.27:0/1260094867 >> v2:172.21.15.119:3300/0 conn(0x7f263c15b6b0 msgr2=0x7f263c1401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.948+0000 7f2641aaa640 1 --2- 172.21.15.27:0/1260094867 >> v2:172.21.15.119:3300/0 conn(0x7f263c15b6b0 0x7f263c1401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.948+0000 7f2641aaa640 1 -- 172.21.15.27:0/1260094867 >> v1:172.21.15.119:6789/0 conn(0x7f263c154ac0 legacy=0x7f263c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:01.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.948+0000 7f2641aaa640 1 -- 172.21.15.27:0/1260094867 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c1408c0 con 0x7f263c0cb700 2023-08-11T15:52:01.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.949+0000 7f2641aaa640 1 --2- 172.21.15.27:0/1260094867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c0cb700 0x7f263c13fc90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f263c158600 tx=0x7f262c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1ae7a4c45384e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:01.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.950+0000 7f26412a9640 1 -- 172.21.15.27:0/1260094867 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0317e0 con 0x7f263c0cb700 2023-08-11T15:52:01.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.950+0000 7f26412a9640 1 -- 172.21.15.27:0/1260094867 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c031940 con 0x7f263c0cb700 2023-08-11T15:52:01.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.950+0000 7f26412a9640 1 -- 172.21.15.27:0/1260094867 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c031cd0 con 0x7f263c0cb700 2023-08-11T15:52:01.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.950+0000 7f264450f640 1 -- 172.21.15.27:0/1260094867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c0cb700 msgr2=0x7f263c13fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.950+0000 7f264450f640 1 --2- 172.21.15.27:0/1260094867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c0cb700 0x7f263c13fc90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f263c158600 tx=0x7f262c02b880 comp rx=0 tx=0).stop 2023-08-11T15:52:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.950+0000 7f264450f640 1 -- 172.21.15.27:0/1260094867 shutdown_connections 2023-08-11T15:52:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.951+0000 7f264450f640 1 --2- 172.21.15.27:0/1260094867 >> v2:172.21.15.119:3300/0 conn(0x7f263c15b6b0 0x7f263c1401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.951+0000 7f264450f640 1 --2- 172.21.15.27:0/1260094867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c0cb700 0x7f263c13fc90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.951+0000 7f264450f640 1 -- 172.21.15.27:0/1260094867 >> 172.21.15.27:0/1260094867 conn(0x7f263c069bf0 msgr2=0x7f263c06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.951+0000 7f264450f640 1 -- 172.21.15.27:0/1260094867 shutdown_connections 2023-08-11T15:52:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.951+0000 7f264450f640 1 -- 172.21.15.27:0/1260094867 wait complete. 2023-08-11T15:52:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.952+0000 7f264450f640 1 Processor -- start 2023-08-11T15:52:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.953+0000 7f264450f640 1 -- start start 2023-08-11T15:52:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.953+0000 7f264450f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f263c0cb700 0x7f263c0af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.953+0000 7f264450f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f263c154ac0 0x7f263c0c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f264450f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c15b6b0 0x7f263c0c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f264450f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0c7ee0 con 0x7f263c15b6b0 2023-08-11T15:52:01.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f264450f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c159460 con 0x7f263c154ac0 2023-08-11T15:52:01.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f264450f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0c6bf0 con 0x7f263c0cb700 2023-08-11T15:52:01.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f2642aac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c15b6b0 0x7f263c0c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f2642aac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c15b6b0 0x7f263c0c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57152/0 (socket says 172.21.15.27:57152) 2023-08-11T15:52:01.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f2642aac640 1 -- 172.21.15.27:0/1522622984 learned_addr learned my addr 172.21.15.27:0/1522622984 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f26422ab640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f263c0cb700 0x7f263c0af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f2641aaa640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f263c154ac0 0x7f263c0c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f2642aac640 1 -- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f263c0cb700 msgr2=0x7f263c0af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f2642aac640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f263c0cb700 0x7f263c0af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f2642aac640 1 -- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f263c154ac0 msgr2=0x7f263c0c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:01.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f2642aac640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f263c154ac0 0x7f263c0c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:01.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f2642aac640 1 -- 172.21.15.27:0/1522622984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0094d0 con 0x7f263c15b6b0 2023-08-11T15:52:01.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f26422ab640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f263c0cb700 0x7f263c0af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:01.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.954+0000 7f2641aaa640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f263c154ac0 0x7f263c0c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:01.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.955+0000 7f2642aac640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c15b6b0 0x7f263c0c4140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f263400cd70 tx=0x7f263400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:01.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.955+0000 7f2632ffd640 1 -- 172.21.15.27:0/1522622984 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634003b90 con 0x7f263c15b6b0 2023-08-11T15:52:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.955+0000 7f2632ffd640 1 -- 172.21.15.27:0/1522622984 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2634003cf0 con 0x7f263c15b6b0 2023-08-11T15:52:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.955+0000 7f2632ffd640 1 -- 172.21.15.27:0/1522622984 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634004240 con 0x7f263c15b6b0 2023-08-11T15:52:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.955+0000 7f264450f640 1 -- 172.21.15.27:0/1522622984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c0c6e40 con 0x7f263c15b6b0 2023-08-11T15:52:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.955+0000 7f264450f640 1 -- 172.21.15.27:0/1522622984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c0c7400 con 0x7f263c15b6b0 2023-08-11T15:52:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.956+0000 7f264450f640 1 -- 172.21.15.27:0/1522622984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263c04e420 con 0x7f263c15b6b0 2023-08-11T15:52:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.957+0000 7f2632ffd640 1 -- 172.21.15.27:0/1522622984 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f26340043a0 con 0x7f263c15b6b0 2023-08-11T15:52:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.958+0000 7f2632ffd640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f262006a050 0x7f262006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.958+0000 7f2632ffd640 1 -- 172.21.15.27:0/1522622984 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2634012070 con 0x7f263c15b6b0 2023-08-11T15:52:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.959+0000 7f26422ab640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f262006a050 0x7f262006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:01.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.960+0000 7f26422ab640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f262006a050 0x7f262006c510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f263c0b1e20 tx=0x7f2638009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:01.963+0000 7f2632ffd640 1 -- 172.21.15.27:0/1522622984 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2634054690 con 0x7f263c15b6b0 2023-08-11T15:52:02.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.157+0000 7f264450f640 1 -- 172.21.15.27:0/1522622984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f263c0c4f50 con 0x7f263c15b6b0 2023-08-11T15:52:02.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.158+0000 7f2632ffd640 1 -- 172.21.15.27:0/1522622984 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2634057cb0 con 0x7f263c15b6b0 2023-08-11T15:52:02.161 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:02.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 -- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f262006a050 msgr2=0x7f262006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f262006a050 0x7f262006c510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f263c0b1e20 tx=0x7f2638009040 comp rx=0 tx=0).stop 2023-08-11T15:52:02.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 -- 172.21.15.27:0/1522622984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c15b6b0 msgr2=0x7f263c0c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c15b6b0 0x7f263c0c4140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f263400cd70 tx=0x7f263400a660 comp rx=0 tx=0).stop 2023-08-11T15:52:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 -- 172.21.15.27:0/1522622984 shutdown_connections 2023-08-11T15:52:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f262006a050 0x7f262006c510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f263c15b6b0 0x7f263c0c4140 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f263c154ac0 0x7f263c0c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 --2- 172.21.15.27:0/1522622984 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f263c0cb700 0x7f263c0af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 -- 172.21.15.27:0/1522622984 >> 172.21.15.27:0/1522622984 conn(0x7f263c069bf0 msgr2=0x7f263c0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 -- 172.21.15.27:0/1522622984 shutdown_connections 2023-08-11T15:52:02.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.162+0000 7f264450f640 1 -- 172.21.15.27:0/1522622984 wait complete. 2023-08-11T15:52:02.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:02.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.402+0000 7f85ed124640 1 Processor -- start 2023-08-11T15:52:02.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.402+0000 7f85ed124640 1 -- start start 2023-08-11T15:52:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.403+0000 7f85ed124640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85e81403a0 0x7f85e80b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.403+0000 7f85ed124640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e8140ba0 con 0x7f85e8154ac0 2023-08-11T15:52:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.403+0000 7f85ed124640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e815b220 con 0x7f85e80cb700 2023-08-11T15:52:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.403+0000 7f85ed124640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e8071b10 con 0x7f85e81403a0 2023-08-11T15:52:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.403+0000 7f85e6d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f85e8154ac0 0x7f85e80caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51788/0 (socket says 172.21.15.27:51788) 2023-08-11T15:52:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.404+0000 7f85e6d9d640 1 -- 172.21.15.27:0/4016236518 learned_addr learned my addr 172.21.15.27:0/4016236518 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.404+0000 7f85e759e640 1 --2- 172.21.15.27:0/4016236518 >> v2:172.21.15.119:3300/0 conn(0x7f85e81403a0 0x7f85e80b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.404+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192506309 0 0) 0x7f85e8140ba0 con 0x7f85e8154ac0 2023-08-11T15:52:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.404+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85d4003520 con 0x7f85e8154ac0 2023-08-11T15:52:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.404+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3882517711 0 0) 0x7f85d4003520 con 0x7f85e8154ac0 2023-08-11T15:52:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.404+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 >> v2:172.21.15.119:3300/0 conn(0x7f85e81403a0 msgr2=0x7f85e80b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.404+0000 7f85e5d9b640 1 --2- 172.21.15.27:0/4016236518 >> v2:172.21.15.119:3300/0 conn(0x7f85e81403a0 0x7f85e80b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.404+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 >> v1:172.21.15.119:6789/0 conn(0x7f85e80cb700 legacy=0x7f85e813fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.404+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e81408c0 con 0x7f85e8154ac0 2023-08-11T15:52:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f85d0001670 con 0x7f85e8154ac0 2023-08-11T15:52:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 >> v1:172.21.15.27:6789/0 conn(0x7f85e8154ac0 legacy=0x7f85e80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e5d9b640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85d40039c0 0x7f85d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e5d9b640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d40047c0 0x7f85d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e5d9b640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85d40051d0 0x7f85d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e8071b10 con 0x7f85d40051d0 2023-08-11T15:52:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d400c120 con 0x7f85d40047c0 2023-08-11T15:52:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d400c2e0 con 0x7f85d40039c0 2023-08-11T15:52:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e759e640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85d40051d0 0x7f85d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e6d9d640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85d40039c0 0x7f85d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e659c640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d40047c0 0x7f85d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e759e640 1 -- 172.21.15.27:0/4016236518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85d40039c0 msgr2=0x7f85d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e759e640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85d40039c0 0x7f85d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e759e640 1 -- 172.21.15.27:0/4016236518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d40047c0 msgr2=0x7f85d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e759e640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d40047c0 0x7f85d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e759e640 1 -- 172.21.15.27:0/4016236518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d4003520 con 0x7f85d40051d0 2023-08-11T15:52:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.405+0000 7f85e659c640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d40047c0 0x7f85d4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.406+0000 7f85e6d9d640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85d40039c0 0x7f85d4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:02.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.406+0000 7f85e759e640 1 -- 172.21.15.27:0/4016236518 reap_dead start 2023-08-11T15:52:02.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.407+0000 7f85e759e640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85d40051d0 0x7f85d400b980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f85e81407f0 tx=0x7f85d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c204f9caa14b8e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.407+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0004020 con 0x7f85d40051d0 2023-08-11T15:52:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.407+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0004180 con 0x7f85d40051d0 2023-08-11T15:52:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.407+0000 7f85e5d9b640 1 -- 172.21.15.27:0/4016236518 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e002c680 con 0x7f85d40051d0 2023-08-11T15:52:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.407+0000 7f85ed124640 1 -- 172.21.15.27:0/4016236518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85d40051d0 msgr2=0x7f85d400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.407+0000 7f85ed124640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85d40051d0 0x7f85d400b980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f85e81407f0 tx=0x7f85d40057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.408+0000 7f85ed124640 1 -- 172.21.15.27:0/4016236518 shutdown_connections 2023-08-11T15:52:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.408+0000 7f85ed124640 1 --2- 172.21.15.27:0/4016236518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85d40051d0 0x7f85d400b980 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.408+0000 7f85ed124640 1 -- 172.21.15.27:0/4016236518 >> 172.21.15.27:0/4016236518 conn(0x7f85e8069bf0 msgr2=0x7f85e806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:02.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.408+0000 7f85ed124640 1 -- 172.21.15.27:0/4016236518 shutdown_connections 2023-08-11T15:52:02.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.408+0000 7f85ed124640 1 -- 172.21.15.27:0/4016236518 wait complete. 2023-08-11T15:52:02.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.409+0000 7f85ed124640 1 Processor -- start 2023-08-11T15:52:02.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.410+0000 7f85ed124640 1 -- start start 2023-08-11T15:52:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85ed124640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d40051d0 0x7f85e80b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85ed124640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e80c3a30 0x7f85e8155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85ed124640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85e8156400 0x7f85e80ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85ed124640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80b3d20 con 0x7f85e80c3a30 2023-08-11T15:52:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85ed124640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80b3350 con 0x7f85d40051d0 2023-08-11T15:52:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85ed124640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80b2460 con 0x7f85e8156400 2023-08-11T15:52:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85e659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e80c3a30 0x7f85e8155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85e659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e80c3a30 0x7f85e8155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57168/0 (socket says 172.21.15.27:57168) 2023-08-11T15:52:02.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85e659c640 1 -- 172.21.15.27:0/2421469283 learned_addr learned my addr 172.21.15.27:0/2421469283 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:02.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85e6d9d640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d40051d0 0x7f85e80b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85e759e640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85e8156400 0x7f85e80ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85e659c640 1 -- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85e8156400 msgr2=0x7f85e80ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85e659c640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85e8156400 0x7f85e80ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85e659c640 1 -- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d40051d0 msgr2=0x7f85e80b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.411+0000 7f85e659c640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d40051d0 0x7f85e80b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.412+0000 7f85e659c640 1 -- 172.21.15.27:0/2421469283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e80b2740 con 0x7f85e80c3a30 2023-08-11T15:52:02.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.412+0000 7f85e759e640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85e8156400 0x7f85e80ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:02.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.412+0000 7f85e659c640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e80c3a30 0x7f85e8155ec0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f85d800d8e0 tx=0x7f85d800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:02.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.412+0000 7f85df7fe640 1 -- 172.21.15.27:0/2421469283 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8003da0 con 0x7f85e80c3a30 2023-08-11T15:52:02.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.412+0000 7f85df7fe640 1 -- 172.21.15.27:0/2421469283 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d8005070 con 0x7f85e80c3a30 2023-08-11T15:52:02.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.412+0000 7f85df7fe640 1 -- 172.21.15.27:0/2421469283 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8005440 con 0x7f85e80c3a30 2023-08-11T15:52:02.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.412+0000 7f85ed124640 1 -- 172.21.15.27:0/2421469283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e80b2a20 con 0x7f85e80c3a30 2023-08-11T15:52:02.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.412+0000 7f85ed124640 1 -- 172.21.15.27:0/2421469283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e80b2fe0 con 0x7f85e80c3a30 2023-08-11T15:52:02.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.413+0000 7f85ed124640 1 -- 172.21.15.27:0/2421469283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e80b3170 con 0x7f85e80c3a30 2023-08-11T15:52:02.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.414+0000 7f85df7fe640 1 -- 172.21.15.27:0/2421469283 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f85d80055a0 con 0x7f85e80c3a30 2023-08-11T15:52:02.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.416+0000 7f85df7fe640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85c806a050 0x7f85c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.416+0000 7f85df7fe640 1 -- 172.21.15.27:0/2421469283 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f85d8088c60 con 0x7f85e80c3a30 2023-08-11T15:52:02.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.416+0000 7f85e6d9d640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85c806a050 0x7f85c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.417+0000 7f85e6d9d640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85c806a050 0x7f85c806c510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85d0000c00 tx=0x7f85e815b190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:02.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.420+0000 7f85df7fe640 1 -- 172.21.15.27:0/2421469283 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f85d8053f80 con 0x7f85e80c3a30 2023-08-11T15:52:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.618+0000 7f85ed124640 1 -- 172.21.15.27:0/2421469283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f85e804e420 con 0x7f85e80c3a30 2023-08-11T15:52:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.619+0000 7f85df7fe640 1 -- 172.21.15.27:0/2421469283 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f85d80575a0 con 0x7f85e80c3a30 2023-08-11T15:52:02.620 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:02.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 -- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85c806a050 msgr2=0x7f85c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85c806a050 0x7f85c806c510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85d0000c00 tx=0x7f85e815b190 comp rx=0 tx=0).stop 2023-08-11T15:52:02.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 -- 172.21.15.27:0/2421469283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e80c3a30 msgr2=0x7f85e8155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e80c3a30 0x7f85e8155ec0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f85d800d8e0 tx=0x7f85d800dda0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 -- 172.21.15.27:0/2421469283 shutdown_connections 2023-08-11T15:52:02.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f85c806a050 0x7f85c806c510 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85e8156400 0x7f85e80ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e80c3a30 0x7f85e8155ec0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 --2- 172.21.15.27:0/2421469283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d40051d0 0x7f85e80b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 -- 172.21.15.27:0/2421469283 >> 172.21.15.27:0/2421469283 conn(0x7f85e8069bf0 msgr2=0x7f85e80c9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:02.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 -- 172.21.15.27:0/2421469283 shutdown_connections 2023-08-11T15:52:02.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.622+0000 7f85ed124640 1 -- 172.21.15.27:0/2421469283 wait complete. 2023-08-11T15:52:02.641 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:02.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.887+0000 7f35bb4ec640 1 Processor -- start 2023-08-11T15:52:02.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.887+0000 7f35bb4ec640 1 -- start start 2023-08-11T15:52:02.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.889+0000 7f35bb4ec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35b415a310 0x7f35b406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.889+0000 7f35bb4ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35b406cf00 0x7f35b40c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.889+0000 7f35bb4ec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b40ccb00 con 0x7f35b40ca690 2023-08-11T15:52:02.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.889+0000 7f35bb4ec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b406d370 con 0x7f35b415a310 2023-08-11T15:52:02.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.889+0000 7f35bb4ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40ccde0 con 0x7f35b406cf00 2023-08-11T15:52:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.889+0000 7f35b8a87640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35b415a310 0x7f35b406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.889+0000 7f35b8a87640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35b415a310 0x7f35b406c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57176/0 (socket says 172.21.15.27:57176) 2023-08-11T15:52:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.889+0000 7f35b8a87640 1 -- 172.21.15.27:0/680728245 learned_addr learned my addr 172.21.15.27:0/680728245 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.890+0000 7f35b9a89640 1 --2- 172.21.15.27:0/680728245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35b406cf00 0x7f35b40c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.890+0000 7f35b8a87640 1 -- 172.21.15.27:0/680728245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35b406cf00 msgr2=0x7f35b40c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.890+0000 7f35b8a87640 1 --2- 172.21.15.27:0/680728245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35b406cf00 0x7f35b40c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.890+0000 7f35b8a87640 1 -- 172.21.15.27:0/680728245 >> v1:172.21.15.119:6789/0 conn(0x7f35b40ca690 legacy=0x7f35b40cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.890+0000 7f35b8a87640 1 -- 172.21.15.27:0/680728245 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b40cd0c0 con 0x7f35b415a310 2023-08-11T15:52:02.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.891+0000 7f35b8a87640 1 --2- 172.21.15.27:0/680728245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 0x7f35b406c9c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009870 tx=0x7f35a402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9c2784015ec8dd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:02.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.892+0000 7f35abfff640 1 -- 172.21.15.27:0/680728245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4030840 con 0x7f35b415a310 2023-08-11T15:52:02.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.892+0000 7f35abfff640 1 -- 172.21.15.27:0/680728245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a40309a0 con 0x7f35b415a310 2023-08-11T15:52:02.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.892+0000 7f35abfff640 1 -- 172.21.15.27:0/680728245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4030d30 con 0x7f35b415a310 2023-08-11T15:52:02.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.892+0000 7f35bb4ec640 1 -- 172.21.15.27:0/680728245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 msgr2=0x7f35b406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.892+0000 7f35bb4ec640 1 --2- 172.21.15.27:0/680728245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 0x7f35b406c9c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009870 tx=0x7f35a402b980 comp rx=0 tx=0).stop 2023-08-11T15:52:02.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.893+0000 7f35bb4ec640 1 -- 172.21.15.27:0/680728245 shutdown_connections 2023-08-11T15:52:02.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.893+0000 7f35bb4ec640 1 --2- 172.21.15.27:0/680728245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35b406cf00 0x7f35b40c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.893+0000 7f35bb4ec640 1 --2- 172.21.15.27:0/680728245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 0x7f35b406c9c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.893+0000 7f35bb4ec640 1 -- 172.21.15.27:0/680728245 >> 172.21.15.27:0/680728245 conn(0x7f35b40afbe0 msgr2=0x7f35b40affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:02.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.893+0000 7f35bb4ec640 1 -- 172.21.15.27:0/680728245 shutdown_connections 2023-08-11T15:52:02.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.893+0000 7f35bb4ec640 1 -- 172.21.15.27:0/680728245 wait complete. 2023-08-11T15:52:02.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.895+0000 7f35bb4ec640 1 Processor -- start 2023-08-11T15:52:02.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.895+0000 7f35bb4ec640 1 -- start start 2023-08-11T15:52:02.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.896+0000 7f35bb4ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b406cf00 0x7f35b40adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.896+0000 7f35bb4ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35b40ca690 0x7f35b4156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.896+0000 7f35bb4ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 0x7f35b4150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.896+0000 7f35bb4ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b41547a0 con 0x7f35b415a310 2023-08-11T15:52:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.896+0000 7f35bb4ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b41591f0 con 0x7f35b406cf00 2023-08-11T15:52:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.896+0000 7f35bb4ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b4154a60 con 0x7f35b40ca690 2023-08-11T15:52:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.896+0000 7f35b9a89640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 0x7f35b4150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.896+0000 7f35b9a89640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 0x7f35b4150db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57182/0 (socket says 172.21.15.27:57182) 2023-08-11T15:52:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.896+0000 7f35b9a89640 1 -- 172.21.15.27:0/718185746 learned_addr learned my addr 172.21.15.27:0/718185746 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.897+0000 7f35b9288640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b406cf00 0x7f35b40adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.897+0000 7f35b8a87640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35b40ca690 0x7f35b4156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.897+0000 7f35b9a89640 1 -- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35b40ca690 msgr2=0x7f35b4156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.897+0000 7f35b9a89640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35b40ca690 0x7f35b4156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.897+0000 7f35b9a89640 1 -- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b406cf00 msgr2=0x7f35b40adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.897+0000 7f35b9a89640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b406cf00 0x7f35b40adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.897+0000 7f35b9a89640 1 -- 172.21.15.27:0/718185746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40094d0 con 0x7f35b415a310 2023-08-11T15:52:02.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.897+0000 7f35b9a89640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 0x7f35b4150db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f35ac007e60 tx=0x7f35ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:02.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.898+0000 7f35a9ffb640 1 -- 172.21.15.27:0/718185746 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac014420 con 0x7f35b415a310 2023-08-11T15:52:02.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.898+0000 7f35a9ffb640 1 -- 172.21.15.27:0/718185746 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ac0145a0 con 0x7f35b415a310 2023-08-11T15:52:02.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.898+0000 7f35a9ffb640 1 -- 172.21.15.27:0/718185746 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac014950 con 0x7f35b415a310 2023-08-11T15:52:02.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.898+0000 7f35bb4ec640 1 -- 172.21.15.27:0/718185746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b40ae4f0 con 0x7f35b415a310 2023-08-11T15:52:02.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.898+0000 7f35bb4ec640 1 -- 172.21.15.27:0/718185746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b4158c30 con 0x7f35b415a310 2023-08-11T15:52:02.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.899+0000 7f35bb4ec640 1 -- 172.21.15.27:0/718185746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b404e420 con 0x7f35b415a310 2023-08-11T15:52:02.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.900+0000 7f35a9ffb640 1 -- 172.21.15.27:0/718185746 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f35ac014ab0 con 0x7f35b415a310 2023-08-11T15:52:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.901+0000 7f35a9ffb640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f359806a080 0x7f359806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.901+0000 7f35a9ffb640 1 -- 172.21.15.27:0/718185746 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f35ac0891c0 con 0x7f35b415a310 2023-08-11T15:52:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.902+0000 7f35b9288640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f359806a080 0x7f359806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.903+0000 7f35b9288640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f359806a080 0x7f359806c540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f35b0000f60 tx=0x7f35b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:02.905+0000 7f35a9ffb640 1 -- 172.21.15.27:0/718185746 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f35ac054570 con 0x7f35b415a310 2023-08-11T15:52:03.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.098+0000 7f35bb4ec640 1 -- 172.21.15.27:0/718185746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35b40cc760 con 0x7f35b415a310 2023-08-11T15:52:03.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.099+0000 7f35a9ffb640 1 -- 172.21.15.27:0/718185746 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f35ac057b90 con 0x7f35b415a310 2023-08-11T15:52:03.101 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:03.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.102+0000 7f35bb4ec640 1 -- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f359806a080 msgr2=0x7f359806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.102+0000 7f35bb4ec640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f359806a080 0x7f359806c540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f35b0000f60 tx=0x7f35b0009040 comp rx=0 tx=0).stop 2023-08-11T15:52:03.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.102+0000 7f35bb4ec640 1 -- 172.21.15.27:0/718185746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 msgr2=0x7f35b4150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.102+0000 7f35bb4ec640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 0x7f35b4150db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f35ac007e60 tx=0x7f35ac00e4b0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.102+0000 7f35bb4ec640 1 -- 172.21.15.27:0/718185746 shutdown_connections 2023-08-11T15:52:03.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.102+0000 7f35bb4ec640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f359806a080 0x7f359806c540 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.102+0000 7f35bb4ec640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35b40ca690 0x7f35b4156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.102+0000 7f35bb4ec640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35b415a310 0x7f35b4150db0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.102+0000 7f35bb4ec640 1 --2- 172.21.15.27:0/718185746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b406cf00 0x7f35b40adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.102+0000 7f35bb4ec640 1 -- 172.21.15.27:0/718185746 >> 172.21.15.27:0/718185746 conn(0x7f35b40afbe0 msgr2=0x7f35b4151ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:03.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.103+0000 7f35bb4ec640 1 -- 172.21.15.27:0/718185746 shutdown_connections 2023-08-11T15:52:03.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.103+0000 7f35bb4ec640 1 -- 172.21.15.27:0/718185746 wait complete. 2023-08-11T15:52:03.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:03.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.363+0000 7f2c4d057640 1 Processor -- start 2023-08-11T15:52:03.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.363+0000 7f2c4d057640 1 -- start start 2023-08-11T15:52:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.365+0000 7f2c4d057640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2c48151600 0x7f2c4815df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.365+0000 7f2c4d057640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2c4815e4b0 0x7f2c480ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.365+0000 7f2c4d057640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4806c850 con 0x7f2c48151220 2023-08-11T15:52:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.365+0000 7f2c4d057640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4815e940 con 0x7f2c4815e4b0 2023-08-11T15:52:03.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.365+0000 7f2c4d057640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4806bfc0 con 0x7f2c48151600 2023-08-11T15:52:03.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.365+0000 7f2c4759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2c4815e4b0 0x7f2c480ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.365+0000 7f2c4759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2c4815e4b0 0x7f2c480ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57190/0 (socket says 172.21.15.27:57190) 2023-08-11T15:52:03.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.365+0000 7f2c4759e640 1 -- 172.21.15.27:0/2551702771 learned_addr learned my addr 172.21.15.27:0/2551702771 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:03.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.365+0000 7f2c4659c640 1 --2- 172.21.15.27:0/2551702771 >> v2:172.21.15.119:3300/0 conn(0x7f2c48151600 0x7f2c4815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.366+0000 7f2c4759e640 1 -- 172.21.15.27:0/2551702771 >> v2:172.21.15.119:3300/0 conn(0x7f2c48151600 msgr2=0x7f2c4815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:03.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.366+0000 7f2c4759e640 1 --2- 172.21.15.27:0/2551702771 >> v2:172.21.15.119:3300/0 conn(0x7f2c48151600 0x7f2c4815df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.366+0000 7f2c4759e640 1 -- 172.21.15.27:0/2551702771 >> v1:172.21.15.119:6789/0 conn(0x7f2c48151220 legacy=0x7f2c4806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:03.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.366+0000 7f2c4759e640 1 -- 172.21.15.27:0/2551702771 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4806c290 con 0x7f2c4815e4b0 2023-08-11T15:52:03.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.367+0000 7f2c4759e640 1 --2- 172.21.15.27:0/2551702771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 0x7f2c480ca850 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2c480b0ca0 tx=0x7f2c4002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83be0670523dcdf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.367+0000 7f2c45d9b640 1 -- 172.21.15.27:0/2551702771 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4002c960 con 0x7f2c4815e4b0 2023-08-11T15:52:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.367+0000 7f2c45d9b640 1 -- 172.21.15.27:0/2551702771 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4002cac0 con 0x7f2c4815e4b0 2023-08-11T15:52:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.367+0000 7f2c45d9b640 1 -- 172.21.15.27:0/2551702771 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4002ce50 con 0x7f2c4815e4b0 2023-08-11T15:52:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.368+0000 7f2c4d057640 1 -- 172.21.15.27:0/2551702771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 msgr2=0x7f2c480ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.368+0000 7f2c4d057640 1 --2- 172.21.15.27:0/2551702771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 0x7f2c480ca850 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2c480b0ca0 tx=0x7f2c4002bdc0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.368+0000 7f2c4d057640 1 -- 172.21.15.27:0/2551702771 shutdown_connections 2023-08-11T15:52:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.368+0000 7f2c4d057640 1 --2- 172.21.15.27:0/2551702771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 0x7f2c480ca850 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.368+0000 7f2c4d057640 1 --2- 172.21.15.27:0/2551702771 >> v2:172.21.15.119:3300/0 conn(0x7f2c48151600 0x7f2c4815df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.368+0000 7f2c4d057640 1 -- 172.21.15.27:0/2551702771 >> 172.21.15.27:0/2551702771 conn(0x7f2c480afbe0 msgr2=0x7f2c480affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.368+0000 7f2c4d057640 1 -- 172.21.15.27:0/2551702771 shutdown_connections 2023-08-11T15:52:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.368+0000 7f2c4d057640 1 -- 172.21.15.27:0/2551702771 wait complete. 2023-08-11T15:52:03.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.370+0000 7f2c4d057640 1 Processor -- start 2023-08-11T15:52:03.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.370+0000 7f2c4d057640 1 -- start start 2023-08-11T15:52:03.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.371+0000 7f2c4d057640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c48151220 0x7f2c48146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.371+0000 7f2c4d057640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2c48151600 0x7f2c480b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.371+0000 7f2c4d057640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 0x7f2c481578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.371+0000 7f2c4d057640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c48157430 con 0x7f2c4815e4b0 2023-08-11T15:52:03.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.371+0000 7f2c4d057640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c48156820 con 0x7f2c48151220 2023-08-11T15:52:03.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.371+0000 7f2c4d057640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c48156b00 con 0x7f2c48151600 2023-08-11T15:52:03.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.371+0000 7f2c4759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 0x7f2c481578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c4759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 0x7f2c481578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57200/0 (socket says 172.21.15.27:57200) 2023-08-11T15:52:03.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c4759e640 1 -- 172.21.15.27:0/200433789 learned_addr learned my addr 172.21.15.27:0/200433789 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:03.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c4659c640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2c48151600 0x7f2c480b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c46d9d640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c48151220 0x7f2c48146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c4759e640 1 -- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2c48151600 msgr2=0x7f2c480b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c4759e640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2c48151600 0x7f2c480b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c4759e640 1 -- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c48151220 msgr2=0x7f2c48146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c4759e640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c48151220 0x7f2c48146fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c4759e640 1 -- 172.21.15.27:0/200433789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c40009b30 con 0x7f2c4815e4b0 2023-08-11T15:52:03.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c4659c640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2c48151600 0x7f2c480b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:03.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c46d9d640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c48151220 0x7f2c48146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:03.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.372+0000 7f2c4759e640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 0x7f2c481578c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2c40036040 tx=0x7f2c40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:03.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.373+0000 7f2c2f7fe640 1 -- 172.21.15.27:0/200433789 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40009510 con 0x7f2c4815e4b0 2023-08-11T15:52:03.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.373+0000 7f2c2f7fe640 1 -- 172.21.15.27:0/200433789 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4003e450 con 0x7f2c4815e4b0 2023-08-11T15:52:03.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.373+0000 7f2c2f7fe640 1 -- 172.21.15.27:0/200433789 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4003e800 con 0x7f2c4815e4b0 2023-08-11T15:52:03.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.373+0000 7f2c4d057640 1 -- 172.21.15.27:0/200433789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c48156de0 con 0x7f2c4815e4b0 2023-08-11T15:52:03.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.373+0000 7f2c4d057640 1 -- 172.21.15.27:0/200433789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c481564b0 con 0x7f2c4815e4b0 2023-08-11T15:52:03.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.374+0000 7f2c4d057640 1 -- 172.21.15.27:0/200433789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c481575e0 con 0x7f2c4815e4b0 2023-08-11T15:52:03.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.375+0000 7f2c2f7fe640 1 -- 172.21.15.27:0/200433789 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2c4003e960 con 0x7f2c4815e4b0 2023-08-11T15:52:03.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.376+0000 7f2c2f7fe640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2c3406a080 0x7f2c3406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.377+0000 7f2c2f7fe640 1 -- 172.21.15.27:0/200433789 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2c400abaa0 con 0x7f2c4815e4b0 2023-08-11T15:52:03.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.377+0000 7f2c46d9d640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2c3406a080 0x7f2c3406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.378+0000 7f2c46d9d640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2c3406a080 0x7f2c3406c540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800a830 tx=0x7f2c38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:03.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.379+0000 7f2c2f7fe640 1 -- 172.21.15.27:0/200433789 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2c40076ed0 con 0x7f2c4815e4b0 2023-08-11T15:52:03.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.576+0000 7f2c4d057640 1 -- 172.21.15.27:0/200433789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c48070bd0 con 0x7f2c4815e4b0 2023-08-11T15:52:03.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.577+0000 7f2c2f7fe640 1 -- 172.21.15.27:0/200433789 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c4007a4f0 con 0x7f2c4815e4b0 2023-08-11T15:52:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.579+0000 7f2c4d057640 1 -- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2c3406a080 msgr2=0x7f2c3406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.579+0000 7f2c4d057640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2c3406a080 0x7f2c3406c540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800a830 tx=0x7f2c38008040 comp rx=0 tx=0).stop 2023-08-11T15:52:03.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.579+0000 7f2c4d057640 1 -- 172.21.15.27:0/200433789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 msgr2=0x7f2c481578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.579+0000 7f2c4d057640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 0x7f2c481578c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2c40036040 tx=0x7f2c40004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.579+0000 7f2c4d057640 1 -- 172.21.15.27:0/200433789 shutdown_connections 2023-08-11T15:52:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.579+0000 7f2c4d057640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2c3406a080 0x7f2c3406c540 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.579+0000 7f2c4d057640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2c48151600 0x7f2c480b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.579+0000 7f2c4d057640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4815e4b0 0x7f2c481578c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.579+0000 7f2c4d057640 1 --2- 172.21.15.27:0/200433789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c48151220 0x7f2c48146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.579+0000 7f2c4d057640 1 -- 172.21.15.27:0/200433789 >> 172.21.15.27:0/200433789 conn(0x7f2c480afbe0 msgr2=0x7f2c480affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:03.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.580+0000 7f2c4d057640 1 -- 172.21.15.27:0/200433789 shutdown_connections 2023-08-11T15:52:03.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.580+0000 7f2c4d057640 1 -- 172.21.15.27:0/200433789 wait complete. 2023-08-11T15:52:03.601 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:03.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.840+0000 7f3ea7d42640 1 Processor -- start 2023-08-11T15:52:03.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.840+0000 7f3ea7d42640 1 -- start start 2023-08-11T15:52:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.842+0000 7f3ea7d42640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8640 0x7f3ea00ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.842+0000 7f3ea7d42640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3ea00c8a20 0x7f3ea01646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.842+0000 7f3ea7d42640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea0165460 con 0x7f3ea0164c10 2023-08-11T15:52:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.842+0000 7f3ea7d42640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea00c1980 con 0x7f3ea00c8a20 2023-08-11T15:52:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.842+0000 7f3ea7d42640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00cc940 con 0x7f3ea00c8640 2023-08-11T15:52:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.842+0000 7f3ea52dd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3ea00c8a20 0x7f3ea01646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.842+0000 7f3ea52dd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3ea00c8a20 0x7f3ea01646d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57216/0 (socket says 172.21.15.27:57216) 2023-08-11T15:52:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.843+0000 7f3ea52dd640 1 -- 172.21.15.27:0/5254954 learned_addr learned my addr 172.21.15.27:0/5254954 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.843+0000 7f3ea62df640 1 --1- 172.21.15.27:0/5254954 >> v1:172.21.15.27:6789/0 conn(0x7f3ea0164c10 0x7f3ea006f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51796/0 (socket says 172.21.15.27:51796) 2023-08-11T15:52:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.843+0000 7f3ea5ade640 1 --2- 172.21.15.27:0/5254954 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8640 0x7f3ea00ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.843+0000 7f3ea52dd640 1 -- 172.21.15.27:0/5254954 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8640 msgr2=0x7f3ea00ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.843+0000 7f3ea52dd640 1 --2- 172.21.15.27:0/5254954 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8640 0x7f3ea00ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.843+0000 7f3ea52dd640 1 -- 172.21.15.27:0/5254954 >> v1:172.21.15.27:6789/0 conn(0x7f3ea0164c10 legacy=0x7f3ea006f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.843+0000 7f3ea52dd640 1 -- 172.21.15.27:0/5254954 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00cc010 con 0x7f3ea00c8a20 2023-08-11T15:52:03.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.843+0000 7f3ea5ade640 1 --2- 172.21.15.27:0/5254954 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8640 0x7f3ea00ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.844+0000 7f3ea52dd640 1 --2- 172.21.15.27:0/5254954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8a20 0x7f3ea01646d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009870 tx=0x7f3e9002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87b8dd0ed4bab6d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea4adc640 1 -- 172.21.15.27:0/5254954 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90031860 con 0x7f3ea00c8a20 2023-08-11T15:52:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea4adc640 1 -- 172.21.15.27:0/5254954 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e900319c0 con 0x7f3ea00c8a20 2023-08-11T15:52:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea4adc640 1 -- 172.21.15.27:0/5254954 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90031d50 con 0x7f3ea00c8a20 2023-08-11T15:52:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea7d42640 1 -- 172.21.15.27:0/5254954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8a20 msgr2=0x7f3ea01646d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea7d42640 1 --2- 172.21.15.27:0/5254954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8a20 0x7f3ea01646d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009870 tx=0x7f3e9002b900 comp rx=0 tx=0).stop 2023-08-11T15:52:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea7d42640 1 -- 172.21.15.27:0/5254954 shutdown_connections 2023-08-11T15:52:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea7d42640 1 --2- 172.21.15.27:0/5254954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8a20 0x7f3ea01646d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea7d42640 1 --2- 172.21.15.27:0/5254954 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8640 0x7f3ea00ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea7d42640 1 -- 172.21.15.27:0/5254954 >> 172.21.15.27:0/5254954 conn(0x7f3ea00afbe0 msgr2=0x7f3ea00affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea7d42640 1 -- 172.21.15.27:0/5254954 shutdown_connections 2023-08-11T15:52:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.845+0000 7f3ea7d42640 1 -- 172.21.15.27:0/5254954 wait complete. 2023-08-11T15:52:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.847+0000 7f3ea7d42640 1 Processor -- start 2023-08-11T15:52:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.847+0000 7f3ea7d42640 1 -- start start 2023-08-11T15:52:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.848+0000 7f3ea7d42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8640 0x7f3ea01f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.848+0000 7f3ea7d42640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8a20 0x7f3ea01f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.848+0000 7f3ea7d42640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea0164c10 0x7f3ea00b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.848+0000 7f3ea7d42640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea015f180 con 0x7f3ea00c8640 2023-08-11T15:52:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.848+0000 7f3ea7d42640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01f9be0 con 0x7f3ea0164c10 2023-08-11T15:52:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.848+0000 7f3ea7d42640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01f9ec0 con 0x7f3ea00c8a20 2023-08-11T15:52:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea5ade640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8640 0x7f3ea01f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea5ade640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8640 0x7f3ea01f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57224/0 (socket says 172.21.15.27:57224) 2023-08-11T15:52:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea5ade640 1 -- 172.21.15.27:0/3600595964 learned_addr learned my addr 172.21.15.27:0/3600595964 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea52dd640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8a20 0x7f3ea01f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea5ade640 1 -- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8a20 msgr2=0x7f3ea01f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea62df640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea0164c10 0x7f3ea00b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea5ade640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8a20 0x7f3ea01f94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea5ade640 1 -- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea0164c10 msgr2=0x7f3ea00b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea5ade640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea0164c10 0x7f3ea00b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea5ade640 1 -- 172.21.15.27:0/3600595964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900094d0 con 0x7f3ea00c8640 2023-08-11T15:52:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea62df640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea0164c10 0x7f3ea00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.849+0000 7f3ea5ade640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8640 0x7f3ea01f5ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00c7f0 tx=0x7f3e9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.850+0000 7f3e967fc640 1 -- 172.21.15.27:0/3600595964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c007820 con 0x7f3ea00c8640 2023-08-11T15:52:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.850+0000 7f3e967fc640 1 -- 172.21.15.27:0/3600595964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c007980 con 0x7f3ea00c8640 2023-08-11T15:52:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.850+0000 7f3e967fc640 1 -- 172.21.15.27:0/3600595964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c007cf0 con 0x7f3ea00c8640 2023-08-11T15:52:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.850+0000 7f3ea7d42640 1 -- 172.21.15.27:0/3600595964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea01fa1a0 con 0x7f3ea00c8640 2023-08-11T15:52:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.850+0000 7f3ea7d42640 1 -- 172.21.15.27:0/3600595964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea00b3c90 con 0x7f3ea00c8640 2023-08-11T15:52:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.851+0000 7f3ea7d42640 1 -- 172.21.15.27:0/3600595964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea004e420 con 0x7f3ea00c8640 2023-08-11T15:52:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.852+0000 7f3e967fc640 1 -- 172.21.15.27:0/3600595964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3e9c0113f0 con 0x7f3ea00c8640 2023-08-11T15:52:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.853+0000 7f3e967fc640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e8406a050 0x7f3e8406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.853+0000 7f3e967fc640 1 -- 172.21.15.27:0/3600595964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3e9c088c30 con 0x7f3ea00c8640 2023-08-11T15:52:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.854+0000 7f3ea52dd640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e8406a050 0x7f3e8406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.855+0000 7f3ea52dd640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e8406a050 0x7f3e8406c510 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3e90036000 tx=0x7f3e9003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:03.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:03.858+0000 7f3e967fc640 1 -- 172.21.15.27:0/3600595964 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f3e9c053fe0 con 0x7f3ea00c8640 2023-08-11T15:52:04.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.051+0000 7f3ea7d42640 1 -- 172.21.15.27:0/3600595964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3ea006d560 con 0x7f3ea00c8640 2023-08-11T15:52:04.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.052+0000 7f3e967fc640 1 -- 172.21.15.27:0/3600595964 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3e9c057600 con 0x7f3ea00c8640 2023-08-11T15:52:04.055 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:04.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.054+0000 7f3ea7d42640 1 -- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e8406a050 msgr2=0x7f3e8406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.054+0000 7f3ea7d42640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e8406a050 0x7f3e8406c510 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3e90036000 tx=0x7f3e9003c040 comp rx=0 tx=0).stop 2023-08-11T15:52:04.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.054+0000 7f3ea7d42640 1 -- 172.21.15.27:0/3600595964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8640 msgr2=0x7f3ea01f5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.054+0000 7f3ea7d42640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8640 0x7f3ea01f5ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00c7f0 tx=0x7f3e9c00ccb0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.054+0000 7f3ea7d42640 1 -- 172.21.15.27:0/3600595964 shutdown_connections 2023-08-11T15:52:04.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.055+0000 7f3ea7d42640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e8406a050 0x7f3e8406c510 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.055+0000 7f3ea7d42640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ea00c8a20 0x7f3ea01f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.055+0000 7f3ea7d42640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ea0164c10 0x7f3ea00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.055+0000 7f3ea7d42640 1 --2- 172.21.15.27:0/3600595964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea00c8640 0x7f3ea01f5ef0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.055+0000 7f3ea7d42640 1 -- 172.21.15.27:0/3600595964 >> 172.21.15.27:0/3600595964 conn(0x7f3ea00afbe0 msgr2=0x7f3ea00c0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:04.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.055+0000 7f3ea7d42640 1 -- 172.21.15.27:0/3600595964 shutdown_connections 2023-08-11T15:52:04.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.055+0000 7f3ea7d42640 1 -- 172.21.15.27:0/3600595964 wait complete. 2023-08-11T15:52:04.077 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:04.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.319+0000 7f373d2ec640 1 Processor -- start 2023-08-11T15:52:04.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.319+0000 7f373d2ec640 1 -- start start 2023-08-11T15:52:04.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.320+0000 7f373d2ec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f373815c620 0x7f37380b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.320+0000 7f373d2ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f37380b82f0 0x7f37380cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f373d2ec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37380cda30 con 0x7f37380b55c0 2023-08-11T15:52:04.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f373d2ec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37380ccc10 con 0x7f373815c620 2023-08-11T15:52:04.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f373d2ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3738161110 con 0x7f37380b82f0 2023-08-11T15:52:04.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f3736ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f373815c620 0x7f37380b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f3736ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f373815c620 0x7f37380b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57234/0 (socket says 172.21.15.27:57234) 2023-08-11T15:52:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f3736ffd640 1 -- 172.21.15.27:0/3534955352 learned_addr learned my addr 172.21.15.27:0/3534955352 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f37367fc640 1 --2- 172.21.15.27:0/3534955352 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f37380b82f0 0x7f37380cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f3736ffd640 1 -- 172.21.15.27:0/3534955352 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f37380b82f0 msgr2=0x7f37380cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f3736ffd640 1 --2- 172.21.15.27:0/3534955352 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f37380b82f0 0x7f37380cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f3736ffd640 1 -- 172.21.15.27:0/3534955352 >> v1:172.21.15.119:6789/0 conn(0x7f37380b55c0 legacy=0x7f37380b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:04.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f3736ffd640 1 -- 172.21.15.27:0/3534955352 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37380cce20 con 0x7f373815c620 2023-08-11T15:52:04.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.321+0000 7f37367fc640 1 --2- 172.21.15.27:0/3534955352 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f37380b82f0 0x7f37380cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.323+0000 7f3736ffd640 1 --2- 172.21.15.27:0/3534955352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373815c620 0x7f37380b7db0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3728009e50 tx=0x7f372802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e8a1eb2a1dbd82 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.323+0000 7f3735ffb640 1 -- 172.21.15.27:0/3534955352 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372802ca80 con 0x7f373815c620 2023-08-11T15:52:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.323+0000 7f3735ffb640 1 -- 172.21.15.27:0/3534955352 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372802cbe0 con 0x7f373815c620 2023-08-11T15:52:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.323+0000 7f3735ffb640 1 -- 172.21.15.27:0/3534955352 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37280343f0 con 0x7f373815c620 2023-08-11T15:52:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.324+0000 7f373d2ec640 1 -- 172.21.15.27:0/3534955352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373815c620 msgr2=0x7f37380b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.324+0000 7f373d2ec640 1 --2- 172.21.15.27:0/3534955352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373815c620 0x7f37380b7db0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3728009e50 tx=0x7f372802bee0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.324+0000 7f373d2ec640 1 -- 172.21.15.27:0/3534955352 shutdown_connections 2023-08-11T15:52:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.324+0000 7f373d2ec640 1 --2- 172.21.15.27:0/3534955352 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f37380b82f0 0x7f37380cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.324+0000 7f373d2ec640 1 --2- 172.21.15.27:0/3534955352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373815c620 0x7f37380b7db0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.324+0000 7f373d2ec640 1 -- 172.21.15.27:0/3534955352 >> 172.21.15.27:0/3534955352 conn(0x7f3738071110 msgr2=0x7f37380714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.324+0000 7f373d2ec640 1 -- 172.21.15.27:0/3534955352 shutdown_connections 2023-08-11T15:52:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.324+0000 7f373d2ec640 1 -- 172.21.15.27:0/3534955352 wait complete. 2023-08-11T15:52:04.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.326+0000 7f373d2ec640 1 Processor -- start 2023-08-11T15:52:04.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.326+0000 7f373d2ec640 1 -- start start 2023-08-11T15:52:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.327+0000 7f373d2ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37380b55c0 0x7f37381f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.327+0000 7f373d2ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f37380b82f0 0x7f373815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.327+0000 7f373d2ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373815c620 0x7f3738150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.327+0000 7f373d2ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3738150bc0 con 0x7f37380b55c0 2023-08-11T15:52:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.327+0000 7f373d2ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37380c9bf0 con 0x7f373815c620 2023-08-11T15:52:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.327+0000 7f373d2ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373815d230 con 0x7f37380b82f0 2023-08-11T15:52:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.327+0000 7f3736ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37380b55c0 0x7f37381f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.327+0000 7f3736ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37380b55c0 0x7f37381f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57244/0 (socket says 172.21.15.27:57244) 2023-08-11T15:52:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.327+0000 7f3736ffd640 1 -- 172.21.15.27:0/3443129135 learned_addr learned my addr 172.21.15.27:0/3443129135 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f3736ffd640 1 -- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f37380b82f0 msgr2=0x7f373815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f3736ffd640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f37380b82f0 0x7f373815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f3736ffd640 1 -- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373815c620 msgr2=0x7f3738150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f37377fe640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373815c620 0x7f3738150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f3736ffd640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373815c620 0x7f3738150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f3736ffd640 1 -- 172.21.15.27:0/3443129135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3728009b30 con 0x7f37380b55c0 2023-08-11T15:52:04.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f3736ffd640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37380b55c0 0x7f37381f1b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3728009e50 tx=0x7f3728004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:04.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f371f7fe640 1 -- 172.21.15.27:0/3443129135 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372803e450 con 0x7f37380b55c0 2023-08-11T15:52:04.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f371f7fe640 1 -- 172.21.15.27:0/3443129135 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372803e5f0 con 0x7f37380b55c0 2023-08-11T15:52:04.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f373d2ec640 1 -- 172.21.15.27:0/3443129135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373815d510 con 0x7f37380b55c0 2023-08-11T15:52:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.328+0000 7f371f7fe640 1 -- 172.21.15.27:0/3443129135 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372803e9a0 con 0x7f37380b55c0 2023-08-11T15:52:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.329+0000 7f373d2ec640 1 -- 172.21.15.27:0/3443129135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37380c9eb0 con 0x7f37380b55c0 2023-08-11T15:52:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.330+0000 7f373d2ec640 1 -- 172.21.15.27:0/3443129135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3738073990 con 0x7f37380b55c0 2023-08-11T15:52:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.331+0000 7f371f7fe640 1 -- 172.21.15.27:0/3443129135 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f372802cd50 con 0x7f37380b55c0 2023-08-11T15:52:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.332+0000 7f371f7fe640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f372406a050 0x7f372406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.332+0000 7f371f7fe640 1 -- 172.21.15.27:0/3443129135 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f37280ab5a0 con 0x7f37380b55c0 2023-08-11T15:52:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.332+0000 7f37367fc640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f372406a050 0x7f372406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.333+0000 7f37367fc640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f372406a050 0x7f372406c510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f372c006fd0 tx=0x7f372c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:04.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.336+0000 7f371f7fe640 1 -- 172.21.15.27:0/3443129135 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f37280769e0 con 0x7f37380b55c0 2023-08-11T15:52:04.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.536+0000 7f373d2ec640 1 -- 172.21.15.27:0/3443129135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37380ca0a0 con 0x7f37380b55c0 2023-08-11T15:52:04.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.539+0000 7f371f7fe640 1 -- 172.21.15.27:0/3443129135 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f372807a000 con 0x7f37380b55c0 2023-08-11T15:52:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:04.541 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.541+0000 7f373d2ec640 1 -- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f372406a050 msgr2=0x7f372406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.541+0000 7f373d2ec640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f372406a050 0x7f372406c510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f372c006fd0 tx=0x7f372c008040 comp rx=0 tx=0).stop 2023-08-11T15:52:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.541+0000 7f373d2ec640 1 -- 172.21.15.27:0/3443129135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37380b55c0 msgr2=0x7f37381f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.541+0000 7f373d2ec640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37380b55c0 0x7f37381f1b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3728009e50 tx=0x7f3728004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.542+0000 7f373d2ec640 1 -- 172.21.15.27:0/3443129135 shutdown_connections 2023-08-11T15:52:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.542+0000 7f373d2ec640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f372406a050 0x7f372406c510 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.542+0000 7f373d2ec640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f37380b82f0 0x7f373815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.542+0000 7f373d2ec640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f373815c620 0x7f3738150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.542+0000 7f373d2ec640 1 --2- 172.21.15.27:0/3443129135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37380b55c0 0x7f37381f1b80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.542+0000 7f373d2ec640 1 -- 172.21.15.27:0/3443129135 >> 172.21.15.27:0/3443129135 conn(0x7f3738071110 msgr2=0x7f373815bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:04.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.542+0000 7f373d2ec640 1 -- 172.21.15.27:0/3443129135 shutdown_connections 2023-08-11T15:52:04.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.542+0000 7f373d2ec640 1 -- 172.21.15.27:0/3443129135 wait complete. 2023-08-11T15:52:04.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:04.562 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.801+0000 7f676b742640 1 Processor -- start 2023-08-11T15:52:04.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.801+0000 7f676b742640 1 -- start start 2023-08-11T15:52:04.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.803+0000 7f676b742640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6764154ac0 0x7f67640caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.803+0000 7f676b742640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f676415b6b0 0x7f67640cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.803+0000 7f676b742640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6764140ba0 con 0x7f676413fc90 2023-08-11T15:52:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.803+0000 7f676b742640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6764071b10 con 0x7f676415b6b0 2023-08-11T15:52:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.803+0000 7f676b742640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676415b220 con 0x7f6764154ac0 2023-08-11T15:52:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.803+0000 7f6768cdd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f676415b6b0 0x7f67640cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.803+0000 7f6768cdd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f676415b6b0 0x7f67640cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57250/0 (socket says 172.21.15.27:57250) 2023-08-11T15:52:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.803+0000 7f6768cdd640 1 -- 172.21.15.27:0/2107618849 learned_addr learned my addr 172.21.15.27:0/2107618849 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.803+0000 7f67694de640 1 --2- 172.21.15.27:0/2107618849 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6764154ac0 0x7f67640caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.804+0000 7f6768cdd640 1 -- 172.21.15.27:0/2107618849 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6764154ac0 msgr2=0x7f67640caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.804+0000 7f6768cdd640 1 --2- 172.21.15.27:0/2107618849 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6764154ac0 0x7f67640caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.804+0000 7f6768cdd640 1 -- 172.21.15.27:0/2107618849 >> v1:172.21.15.27:6789/0 conn(0x7f676413fc90 legacy=0x7f67640b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.804+0000 7f6768cdd640 1 -- 172.21.15.27:0/2107618849 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67641408c0 con 0x7f676415b6b0 2023-08-11T15:52:04.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.804+0000 7f675bfff640 1 -- 172.21.15.27:0/2107618849 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471348476 0 0) 0x7f6764140ba0 con 0x7f676413fc90 2023-08-11T15:52:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.805+0000 7f6768cdd640 1 --2- 172.21.15.27:0/2107618849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676415b6b0 0x7f67640cb530 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f67540098a0 tx=0x7f675402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ab0207a57bc7ea server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.805+0000 7f675bfff640 1 -- 172.21.15.27:0/2107618849 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754031640 con 0x7f676415b6b0 2023-08-11T15:52:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.805+0000 7f675bfff640 1 -- 172.21.15.27:0/2107618849 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67540317a0 con 0x7f676415b6b0 2023-08-11T15:52:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.805+0000 7f675bfff640 1 -- 172.21.15.27:0/2107618849 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754031b30 con 0x7f676415b6b0 2023-08-11T15:52:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.806+0000 7f676b742640 1 -- 172.21.15.27:0/2107618849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676415b6b0 msgr2=0x7f67640cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.806+0000 7f676b742640 1 --2- 172.21.15.27:0/2107618849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676415b6b0 0x7f67640cb530 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f67540098a0 tx=0x7f675402b8f0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.806+0000 7f676b742640 1 -- 172.21.15.27:0/2107618849 shutdown_connections 2023-08-11T15:52:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.806+0000 7f676b742640 1 --2- 172.21.15.27:0/2107618849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676415b6b0 0x7f67640cb530 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.806+0000 7f676b742640 1 --2- 172.21.15.27:0/2107618849 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6764154ac0 0x7f67640caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.806+0000 7f676b742640 1 -- 172.21.15.27:0/2107618849 >> 172.21.15.27:0/2107618849 conn(0x7f6764069bf0 msgr2=0x7f676406a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.806+0000 7f676b742640 1 -- 172.21.15.27:0/2107618849 shutdown_connections 2023-08-11T15:52:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.806+0000 7f676b742640 1 -- 172.21.15.27:0/2107618849 wait complete. 2023-08-11T15:52:04.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.808+0000 7f676b742640 1 Processor -- start 2023-08-11T15:52:04.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.808+0000 7f676b742640 1 -- start start 2023-08-11T15:52:04.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.809+0000 7f676b742640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676413fc90 0x7f676415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.809+0000 7f676b742640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6764154ac0 0x7f6764142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.809+0000 7f676b742640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f676415b6b0 0x7f6764143420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.809+0000 7f676b742640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67640cd9c0 con 0x7f676413fc90 2023-08-11T15:52:04.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.809+0000 7f676b742640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6764140d50 con 0x7f6764154ac0 2023-08-11T15:52:04.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.809+0000 7f676b742640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67640cdde0 con 0x7f676415b6b0 2023-08-11T15:52:04.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f67694de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676413fc90 0x7f676415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f67694de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676413fc90 0x7f676415ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57258/0 (socket says 172.21.15.27:57258) 2023-08-11T15:52:04.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f67694de640 1 -- 172.21.15.27:0/4143222140 learned_addr learned my addr 172.21.15.27:0/4143222140 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:04.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f6769cdf640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f676415b6b0 0x7f6764143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f6768cdd640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6764154ac0 0x7f6764142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f67694de640 1 -- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f676415b6b0 msgr2=0x7f6764143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f67694de640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f676415b6b0 0x7f6764143420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f67694de640 1 -- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6764154ac0 msgr2=0x7f6764142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f67694de640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6764154ac0 0x7f6764142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f67694de640 1 -- 172.21.15.27:0/4143222140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67540094d0 con 0x7f676413fc90 2023-08-11T15:52:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f6768cdd640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6764154ac0 0x7f6764142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f6769cdf640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f676415b6b0 0x7f6764143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.810+0000 7f67694de640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676413fc90 0x7f676415ce90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f676000d800 tx=0x7f676000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.811+0000 7f6759ffb640 1 -- 172.21.15.27:0/4143222140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676000b6c0 con 0x7f676413fc90 2023-08-11T15:52:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.811+0000 7f6759ffb640 1 -- 172.21.15.27:0/4143222140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676000b820 con 0x7f676413fc90 2023-08-11T15:52:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.811+0000 7f6759ffb640 1 -- 172.21.15.27:0/4143222140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676000bbb0 con 0x7f676413fc90 2023-08-11T15:52:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.811+0000 7f676b742640 1 -- 172.21.15.27:0/4143222140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67640ccd40 con 0x7f676413fc90 2023-08-11T15:52:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.811+0000 7f676b742640 1 -- 172.21.15.27:0/4143222140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67640cd300 con 0x7f676413fc90 2023-08-11T15:52:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.812+0000 7f676b742640 1 -- 172.21.15.27:0/4143222140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67640cdc30 con 0x7f676413fc90 2023-08-11T15:52:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.813+0000 7f6759ffb640 1 -- 172.21.15.27:0/4143222140 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f676000bd10 con 0x7f676413fc90 2023-08-11T15:52:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.814+0000 7f6759ffb640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f674806a050 0x7f674806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.814+0000 7f6759ffb640 1 -- 172.21.15.27:0/4143222140 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6760088ef0 con 0x7f676413fc90 2023-08-11T15:52:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.815+0000 7f6768cdd640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f674806a050 0x7f674806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:04.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.816+0000 7f6768cdd640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f674806a050 0x7f674806c510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6754036000 tx=0x7f675403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:04.818+0000 7f6759ffb640 1 -- 172.21.15.27:0/4143222140 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6760054210 con 0x7f676413fc90 2023-08-11T15:52:05.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.010+0000 7f676b742640 1 -- 172.21.15.27:0/4143222140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f676404e420 con 0x7f676413fc90 2023-08-11T15:52:05.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.011+0000 7f6759ffb640 1 -- 172.21.15.27:0/4143222140 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6760057830 con 0x7f676413fc90 2023-08-11T15:52:05.012 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:05.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.013+0000 7f676b742640 1 -- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f674806a050 msgr2=0x7f674806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.013+0000 7f676b742640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f674806a050 0x7f674806c510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6754036000 tx=0x7f675403c040 comp rx=0 tx=0).stop 2023-08-11T15:52:05.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.013+0000 7f676b742640 1 -- 172.21.15.27:0/4143222140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676413fc90 msgr2=0x7f676415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.013+0000 7f676b742640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676413fc90 0x7f676415ce90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f676000d800 tx=0x7f676000dcc0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.014+0000 7f676b742640 1 -- 172.21.15.27:0/4143222140 shutdown_connections 2023-08-11T15:52:05.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.014+0000 7f676b742640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f674806a050 0x7f674806c510 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.014+0000 7f676b742640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f676415b6b0 0x7f6764143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.014+0000 7f676b742640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6764154ac0 0x7f6764142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.014+0000 7f676b742640 1 --2- 172.21.15.27:0/4143222140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f676413fc90 0x7f676415ce90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.014+0000 7f676b742640 1 -- 172.21.15.27:0/4143222140 >> 172.21.15.27:0/4143222140 conn(0x7f6764069bf0 msgr2=0x7f676415a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:05.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.014+0000 7f676b742640 1 -- 172.21.15.27:0/4143222140 shutdown_connections 2023-08-11T15:52:05.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.014+0000 7f676b742640 1 -- 172.21.15.27:0/4143222140 wait complete. 2023-08-11T15:52:05.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:05.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.275+0000 7fbd4f56d640 1 Processor -- start 2023-08-11T15:52:05.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.275+0000 7fbd4f56d640 1 -- start start 2023-08-11T15:52:05.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.277+0000 7fbd4f56d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd480ced00 0x7fbd480d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.277+0000 7fbd4f56d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd480cba20 con 0x7fbd4815a320 2023-08-11T15:52:05.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.277+0000 7fbd4f56d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd480cabb0 con 0x7fbd480cddf0 2023-08-11T15:52:05.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.277+0000 7fbd4f56d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480cae50 con 0x7fbd480ced00 2023-08-11T15:52:05.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.277+0000 7fbd4e56b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbd4815a320 0x7fbd480ce5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51808/0 (socket says 172.21.15.27:51808) 2023-08-11T15:52:05.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.277+0000 7fbd4e56b640 1 -- 172.21.15.27:0/1827113689 learned_addr learned my addr 172.21.15.27:0/1827113689 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:05.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.278+0000 7fbd4dd6a640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd480ced00 0x7fbd480d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.278+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047885437 0 0) 0x7fbd480cba20 con 0x7fbd4815a320 2023-08-11T15:52:05.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.278+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd34003520 con 0x7fbd4815a320 2023-08-11T15:52:05.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.278+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3806123965 0 0) 0x7fbd34003520 con 0x7fbd4815a320 2023-08-11T15:52:05.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.278+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd480ced00 msgr2=0x7fbd480d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:05.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.278+0000 7fbd4d569640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd480ced00 0x7fbd480d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.278+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 >> v1:172.21.15.119:6789/0 conn(0x7fbd480cddf0 legacy=0x7fbd480b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.278+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480b0980 con 0x7fbd4815a320 2023-08-11T15:52:05.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fbd44001670 con 0x7fbd4815a320 2023-08-11T15:52:05.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 >> v1:172.21.15.27:6789/0 conn(0x7fbd4815a320 legacy=0x7fbd480ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4d569640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd340039a0 0x7fbd34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4d569640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd480ced00 0x7fbd340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4d569640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd34004d00 0x7fbd34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480cae50 con 0x7fbd340039a0 2023-08-11T15:52:05.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3400ba30 con 0x7fbd34004d00 2023-08-11T15:52:05.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3400bbf0 con 0x7fbd480ced00 2023-08-11T15:52:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4e56b640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd340039a0 0x7fbd34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4dd6a640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd480ced00 0x7fbd340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4ed6c640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd34004d00 0x7fbd34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4e56b640 1 -- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd480ced00 msgr2=0x7fbd340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4e56b640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd480ced00 0x7fbd340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4e56b640 1 -- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd34004d00 msgr2=0x7fbd34005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4e56b640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd34004d00 0x7fbd34005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.279+0000 7fbd4e56b640 1 -- 172.21.15.27:0/1827113689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34003520 con 0x7fbd340039a0 2023-08-11T15:52:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.280+0000 7fbd4ed6c640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd34004d00 0x7fbd34005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.281+0000 7fbd4e56b640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd340039a0 0x7fbd34003dc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400c210 tx=0x7fbd340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da262cbe9813f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.281+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44005ad0 con 0x7fbd340039a0 2023-08-11T15:52:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.281+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44005c30 con 0x7fbd340039a0 2023-08-11T15:52:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.281+0000 7fbd4d569640 1 -- 172.21.15.27:0/1827113689 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4402c890 con 0x7fbd340039a0 2023-08-11T15:52:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.281+0000 7fbd4f56d640 1 -- 172.21.15.27:0/1827113689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd340039a0 msgr2=0x7fbd34003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.281+0000 7fbd4f56d640 1 --2- 172.21.15.27:0/1827113689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd340039a0 0x7fbd34003dc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400c210 tx=0x7fbd340057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.281+0000 7fbd4ed6c640 1 -- 172.21.15.27:0/1827113689 reap_dead start 2023-08-11T15:52:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.282+0000 7fbd4f56d640 1 -- 172.21.15.27:0/1827113689 shutdown_connections 2023-08-11T15:52:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.282+0000 7fbd4f56d640 1 -- 172.21.15.27:0/1827113689 >> 172.21.15.27:0/1827113689 conn(0x7fbd480d0010 msgr2=0x7fbd480d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.282+0000 7fbd4f56d640 1 -- 172.21.15.27:0/1827113689 shutdown_connections 2023-08-11T15:52:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.282+0000 7fbd4f56d640 1 -- 172.21.15.27:0/1827113689 wait complete. 2023-08-11T15:52:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.284+0000 7fbd4f56d640 1 Processor -- start 2023-08-11T15:52:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.284+0000 7fbd4f56d640 1 -- start start 2023-08-11T15:52:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4f56d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd340039a0 0x7fbd480c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4f56d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd480c8f20 0x7fbd480c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4f56d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd4815bdc0 0x7fbd4815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4f56d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4815c790 con 0x7fbd4815bdc0 2023-08-11T15:52:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4f56d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd48158e70 con 0x7fbd480c8f20 2023-08-11T15:52:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4f56d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480c43a0 con 0x7fbd340039a0 2023-08-11T15:52:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4ed6c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd4815bdc0 0x7fbd4815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4ed6c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd4815bdc0 0x7fbd4815c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57278/0 (socket says 172.21.15.27:57278) 2023-08-11T15:52:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4ed6c640 1 -- 172.21.15.27:0/852518317 learned_addr learned my addr 172.21.15.27:0/852518317 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4e56b640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd340039a0 0x7fbd480c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.285+0000 7fbd4dd6a640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd480c8f20 0x7fbd480c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd4ed6c640 1 -- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd340039a0 msgr2=0x7fbd480c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd4ed6c640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd340039a0 0x7fbd480c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd4ed6c640 1 -- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd480c8f20 msgr2=0x7fbd480c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd4ed6c640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd480c8f20 0x7fbd480c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd4ed6c640 1 -- 172.21.15.27:0/852518317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480c93a0 con 0x7fbd4815bdc0 2023-08-11T15:52:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd4ed6c640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd4815bdc0 0x7fbd4815c250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbd480b2b50 tx=0x7fbd4000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd3effd640 1 -- 172.21.15.27:0/852518317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40006d50 con 0x7fbd4815bdc0 2023-08-11T15:52:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd3effd640 1 -- 172.21.15.27:0/852518317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd40003970 con 0x7fbd4815bdc0 2023-08-11T15:52:05.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd3effd640 1 -- 172.21.15.27:0/852518317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40003d60 con 0x7fbd4815bdc0 2023-08-11T15:52:05.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd4f56d640 1 -- 172.21.15.27:0/852518317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd480c6ac0 con 0x7fbd4815bdc0 2023-08-11T15:52:05.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.286+0000 7fbd4f56d640 1 -- 172.21.15.27:0/852518317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd480c84d0 con 0x7fbd4815bdc0 2023-08-11T15:52:05.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.288+0000 7fbd4f56d640 1 -- 172.21.15.27:0/852518317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd480c4570 con 0x7fbd4815bdc0 2023-08-11T15:52:05.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.289+0000 7fbd3effd640 1 -- 172.21.15.27:0/852518317 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbd40005070 con 0x7fbd4815bdc0 2023-08-11T15:52:05.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.290+0000 7fbd3effd640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbd2c06a050 0x7fbd2c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.290+0000 7fbd3effd640 1 -- 172.21.15.27:0/852518317 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbd400887c0 con 0x7fbd4815bdc0 2023-08-11T15:52:05.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.290+0000 7fbd4e56b640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbd2c06a050 0x7fbd2c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.291+0000 7fbd4e56b640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbd2c06a050 0x7fbd2c06c510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd44034000 tx=0x7fbd44009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.294+0000 7fbd3effd640 1 -- 172.21.15.27:0/852518317 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fbd480c4570 con 0x7fbd4815bdc0 2023-08-11T15:52:05.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.487+0000 7fbd4f56d640 1 -- 172.21.15.27:0/852518317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd480c4570 con 0x7fbd4815bdc0 2023-08-11T15:52:05.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.488+0000 7fbd3effd640 1 -- 172.21.15.27:0/852518317 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd480c4570 con 0x7fbd4815bdc0 2023-08-11T15:52:05.490 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:05.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.491+0000 7fbd4f56d640 1 -- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbd2c06a050 msgr2=0x7fbd2c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.491+0000 7fbd4f56d640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbd2c06a050 0x7fbd2c06c510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd44034000 tx=0x7fbd44009030 comp rx=0 tx=0).stop 2023-08-11T15:52:05.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.491+0000 7fbd4f56d640 1 -- 172.21.15.27:0/852518317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd4815bdc0 msgr2=0x7fbd4815c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.491+0000 7fbd4f56d640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd4815bdc0 0x7fbd4815c250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbd480b2b50 tx=0x7fbd4000dc00 comp rx=0 tx=0).stop 2023-08-11T15:52:05.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.491+0000 7fbd4f56d640 1 -- 172.21.15.27:0/852518317 shutdown_connections 2023-08-11T15:52:05.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.491+0000 7fbd4f56d640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbd2c06a050 0x7fbd2c06c510 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.491+0000 7fbd4f56d640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd4815bdc0 0x7fbd4815c250 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.491+0000 7fbd4f56d640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd480c8f20 0x7fbd480c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.491+0000 7fbd4f56d640 1 --2- 172.21.15.27:0/852518317 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd340039a0 0x7fbd480c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.491+0000 7fbd4f56d640 1 -- 172.21.15.27:0/852518317 >> 172.21.15.27:0/852518317 conn(0x7fbd480d0010 msgr2=0x7fbd480cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:05.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.492+0000 7fbd4f56d640 1 -- 172.21.15.27:0/852518317 shutdown_connections 2023-08-11T15:52:05.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.492+0000 7fbd4f56d640 1 -- 172.21.15.27:0/852518317 wait complete. 2023-08-11T15:52:05.511 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.734+0000 7f35730aa640 1 Processor -- start 2023-08-11T15:52:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.735+0000 7f35730aa640 1 -- start start 2023-08-11T15:52:05.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.736+0000 7f35730aa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f356c15a0f0 0x7f356c1435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.736+0000 7f35730aa640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f356c15a4d0 0x7f356c071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.736+0000 7f35730aa640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356c06c500 con 0x7f356c06f5d0 2023-08-11T15:52:05.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.736+0000 7f35730aa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c06b8f0 con 0x7f356c15a0f0 2023-08-11T15:52:05.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.736+0000 7f35730aa640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c06bbd0 con 0x7f356c15a4d0 2023-08-11T15:52:05.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.737+0000 7f35720a8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f356c15a0f0 0x7f356c1435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.737+0000 7f35720a8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f356c15a0f0 0x7f356c1435e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57284/0 (socket says 172.21.15.27:57284) 2023-08-11T15:52:05.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.737+0000 7f35720a8640 1 -- 172.21.15.27:0/2993590446 learned_addr learned my addr 172.21.15.27:0/2993590446 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:05.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.737+0000 7f35718a7640 1 --2- 172.21.15.27:0/2993590446 >> v2:172.21.15.119:3300/0 conn(0x7f356c15a4d0 0x7f356c071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.737+0000 7f35720a8640 1 -- 172.21.15.27:0/2993590446 >> v2:172.21.15.119:3300/0 conn(0x7f356c15a4d0 msgr2=0x7f356c071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:05.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.737+0000 7f35720a8640 1 --2- 172.21.15.27:0/2993590446 >> v2:172.21.15.119:3300/0 conn(0x7f356c15a4d0 0x7f356c071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.737+0000 7f35720a8640 1 -- 172.21.15.27:0/2993590446 >> v1:172.21.15.119:6789/0 conn(0x7f356c06f5d0 legacy=0x7f356c15de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:05.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.737+0000 7f35720a8640 1 -- 172.21.15.27:0/2993590446 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c06beb0 con 0x7f356c15a0f0 2023-08-11T15:52:05.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.739+0000 7f35720a8640 1 --2- 172.21.15.27:0/2993590446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c15a0f0 0x7f356c1435e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f356c06f3b0 tx=0x7f356802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e648fbd7ebcbf9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:05.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.739+0000 7f35710a6640 1 -- 172.21.15.27:0/2993590446 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568031870 con 0x7f356c15a0f0 2023-08-11T15:52:05.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.739+0000 7f35710a6640 1 -- 172.21.15.27:0/2993590446 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35680319d0 con 0x7f356c15a0f0 2023-08-11T15:52:05.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.739+0000 7f35710a6640 1 -- 172.21.15.27:0/2993590446 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568031d60 con 0x7f356c15a0f0 2023-08-11T15:52:05.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.740+0000 7f35730aa640 1 -- 172.21.15.27:0/2993590446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c15a0f0 msgr2=0x7f356c1435e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.740+0000 7f35730aa640 1 --2- 172.21.15.27:0/2993590446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c15a0f0 0x7f356c1435e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f356c06f3b0 tx=0x7f356802b910 comp rx=0 tx=0).stop 2023-08-11T15:52:05.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.740+0000 7f35730aa640 1 -- 172.21.15.27:0/2993590446 shutdown_connections 2023-08-11T15:52:05.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.740+0000 7f35730aa640 1 --2- 172.21.15.27:0/2993590446 >> v2:172.21.15.119:3300/0 conn(0x7f356c15a4d0 0x7f356c071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.740+0000 7f35730aa640 1 --2- 172.21.15.27:0/2993590446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c15a0f0 0x7f356c1435e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.740+0000 7f35730aa640 1 -- 172.21.15.27:0/2993590446 >> 172.21.15.27:0/2993590446 conn(0x7f356c0caab0 msgr2=0x7f356c0cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:05.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.740+0000 7f35730aa640 1 -- 172.21.15.27:0/2993590446 shutdown_connections 2023-08-11T15:52:05.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.740+0000 7f35730aa640 1 -- 172.21.15.27:0/2993590446 wait complete. 2023-08-11T15:52:05.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.742+0000 7f35730aa640 1 Processor -- start 2023-08-11T15:52:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.742+0000 7f35730aa640 1 -- start start 2023-08-11T15:52:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.743+0000 7f35730aa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c06f5d0 0x7f356c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.743+0000 7f35730aa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f356c15a0f0 0x7f356c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.743+0000 7f35730aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f356c15a4d0 0x7f356c157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.743+0000 7f35730aa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0b3bd0 con 0x7f356c06f5d0 2023-08-11T15:52:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.743+0000 7f35730aa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c157730 con 0x7f356c15a4d0 2023-08-11T15:52:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.743+0000 7f35730aa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c157a10 con 0x7f356c15a0f0 2023-08-11T15:52:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.744+0000 7f35720a8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c06f5d0 0x7f356c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.744+0000 7f35720a8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c06f5d0 0x7f356c0b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57298/0 (socket says 172.21.15.27:57298) 2023-08-11T15:52:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.744+0000 7f35720a8640 1 -- 172.21.15.27:0/3060079679 learned_addr learned my addr 172.21.15.27:0/3060079679 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:05.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.744+0000 7f35720a8640 1 -- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f356c15a0f0 msgr2=0x7f356c0b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.744+0000 7f35720a8640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f356c15a0f0 0x7f356c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.744+0000 7f35720a8640 1 -- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f356c15a4d0 msgr2=0x7f356c157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.744+0000 7f35720a8640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f356c15a4d0 0x7f356c157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.744+0000 7f35720a8640 1 -- 172.21.15.27:0/3060079679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35680094d0 con 0x7f356c06f5d0 2023-08-11T15:52:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.745+0000 7f35720a8640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c06f5d0 0x7f356c0b3690 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3568000c00 tx=0x7f35680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.745+0000 7f3562ffd640 1 -- 172.21.15.27:0/3060079679 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356803abe0 con 0x7f356c06f5d0 2023-08-11T15:52:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.745+0000 7f3562ffd640 1 -- 172.21.15.27:0/3060079679 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356803ad40 con 0x7f356c06f5d0 2023-08-11T15:52:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.745+0000 7f3562ffd640 1 -- 172.21.15.27:0/3060079679 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568044630 con 0x7f356c06f5d0 2023-08-11T15:52:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.745+0000 7f35730aa640 1 -- 172.21.15.27:0/3060079679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c0b1920 con 0x7f356c06f5d0 2023-08-11T15:52:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.745+0000 7f35730aa640 1 -- 172.21.15.27:0/3060079679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356c0b1620 con 0x7f356c06f5d0 2023-08-11T15:52:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.747+0000 7f35730aa640 1 -- 172.21.15.27:0/3060079679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356c0b1e40 con 0x7f356c06f5d0 2023-08-11T15:52:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.748+0000 7f3562ffd640 1 -- 172.21.15.27:0/3060079679 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f356804d450 con 0x7f356c06f5d0 2023-08-11T15:52:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.749+0000 7f3562ffd640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355006a080 0x7f355006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.749+0000 7f3562ffd640 1 -- 172.21.15.27:0/3060079679 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f356802d040 con 0x7f356c06f5d0 2023-08-11T15:52:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.750+0000 7f35718a7640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355006a080 0x7f355006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.751+0000 7f35718a7640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355006a080 0x7f355006c540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f355c006fd0 tx=0x7f355c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.753+0000 7f3562ffd640 1 -- 172.21.15.27:0/3060079679 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f356c0b1e40 con 0x7f356c06f5d0 2023-08-11T15:52:05.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.949+0000 7f35730aa640 1 -- 172.21.15.27:0/3060079679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f356c0b1e40 con 0x7f356c06f5d0 2023-08-11T15:52:05.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.950+0000 7f3562ffd640 1 -- 172.21.15.27:0/3060079679 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f356c0b1e40 con 0x7f356c06f5d0 2023-08-11T15:52:05.952 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 -- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355006a080 msgr2=0x7f355006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355006a080 0x7f355006c540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f355c006fd0 tx=0x7f355c008040 comp rx=0 tx=0).stop 2023-08-11T15:52:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 -- 172.21.15.27:0/3060079679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c06f5d0 msgr2=0x7f356c0b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c06f5d0 0x7f356c0b3690 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3568000c00 tx=0x7f35680046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 -- 172.21.15.27:0/3060079679 shutdown_connections 2023-08-11T15:52:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355006a080 0x7f355006c540 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f356c15a0f0 0x7f356c0b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f356c15a4d0 0x7f356c157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 --2- 172.21.15.27:0/3060079679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f356c06f5d0 0x7f356c0b3690 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:05.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 -- 172.21.15.27:0/3060079679 >> 172.21.15.27:0/3060079679 conn(0x7f356c0caab0 msgr2=0x7f356c0cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:05.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 -- 172.21.15.27:0/3060079679 shutdown_connections 2023-08-11T15:52:05.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:05.953+0000 7f35730aa640 1 -- 172.21.15.27:0/3060079679 wait complete. 2023-08-11T15:52:05.974 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.214+0000 7f5231c6a640 1 Processor -- start 2023-08-11T15:52:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.214+0000 7f5231c6a640 1 -- start start 2023-08-11T15:52:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.216+0000 7f5231c6a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f522c0b1d10 0x7f522c0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.216+0000 7f5231c6a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f522c0cada0 0x7f522c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.216+0000 7f5231c6a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f522c0cf1f0 0x7f522c0cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.216+0000 7f5231c6a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522c15a8b0 con 0x7f522c0cf1f0 2023-08-11T15:52:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.216+0000 7f5231c6a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522c0b2630 con 0x7f522c0b1d10 2023-08-11T15:52:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.216+0000 7f5231c6a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0cfb20 con 0x7f522c0cada0 2023-08-11T15:52:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.217+0000 7f522bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f522c0cf1f0 0x7f522c0cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.217+0000 7f522bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f522c0cf1f0 0x7f522c0cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57306/0 (socket says 172.21.15.27:57306) 2023-08-11T15:52:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.217+0000 7f522bfff640 1 -- 172.21.15.27:0/3826028619 learned_addr learned my addr 172.21.15.27:0/3826028619 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.217+0000 7f522affd640 1 --2- 172.21.15.27:0/3826028619 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f522c0cada0 0x7f522c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.217+0000 7f522b7fe640 1 --2- 172.21.15.27:0/3826028619 >> v2:172.21.15.119:3300/0 conn(0x7f522c0b1d10 0x7f522c0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.217+0000 7f522bfff640 1 -- 172.21.15.27:0/3826028619 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f522c0cada0 msgr2=0x7f522c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.217+0000 7f522bfff640 1 --2- 172.21.15.27:0/3826028619 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f522c0cada0 0x7f522c15b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.217+0000 7f522bfff640 1 -- 172.21.15.27:0/3826028619 >> v2:172.21.15.119:3300/0 conn(0x7f522c0b1d10 msgr2=0x7f522c0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.217+0000 7f522bfff640 1 --2- 172.21.15.27:0/3826028619 >> v2:172.21.15.119:3300/0 conn(0x7f522c0b1d10 0x7f522c0b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.217+0000 7f522bfff640 1 -- 172.21.15.27:0/3826028619 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c0cfe00 con 0x7f522c0cf1f0 2023-08-11T15:52:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.218+0000 7f522bfff640 1 --2- 172.21.15.27:0/3826028619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cf1f0 0x7f522c0cde60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f522c0d01f0 tx=0x7f522402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa3c78af4774511 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.219+0000 7f522a7fc640 1 -- 172.21.15.27:0/3826028619 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240317e0 con 0x7f522c0cf1f0 2023-08-11T15:52:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.219+0000 7f522a7fc640 1 -- 172.21.15.27:0/3826028619 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5224031940 con 0x7f522c0cf1f0 2023-08-11T15:52:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.219+0000 7f522a7fc640 1 -- 172.21.15.27:0/3826028619 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224031cd0 con 0x7f522c0cf1f0 2023-08-11T15:52:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.219+0000 7f5231c6a640 1 -- 172.21.15.27:0/3826028619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cf1f0 msgr2=0x7f522c0cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.220+0000 7f5231c6a640 1 --2- 172.21.15.27:0/3826028619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cf1f0 0x7f522c0cde60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f522c0d01f0 tx=0x7f522402b880 comp rx=0 tx=0).stop 2023-08-11T15:52:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.220+0000 7f5231c6a640 1 -- 172.21.15.27:0/3826028619 shutdown_connections 2023-08-11T15:52:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.220+0000 7f5231c6a640 1 --2- 172.21.15.27:0/3826028619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cf1f0 0x7f522c0cde60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.220+0000 7f5231c6a640 1 --2- 172.21.15.27:0/3826028619 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f522c0cada0 0x7f522c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.220+0000 7f5231c6a640 1 --2- 172.21.15.27:0/3826028619 >> v2:172.21.15.119:3300/0 conn(0x7f522c0b1d10 0x7f522c0b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.220+0000 7f5231c6a640 1 -- 172.21.15.27:0/3826028619 >> 172.21.15.27:0/3826028619 conn(0x7f522c0ccd10 msgr2=0x7f522c0cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.220+0000 7f5231c6a640 1 -- 172.21.15.27:0/3826028619 shutdown_connections 2023-08-11T15:52:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.220+0000 7f5231c6a640 1 -- 172.21.15.27:0/3826028619 wait complete. 2023-08-11T15:52:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.222+0000 7f5231c6a640 1 Processor -- start 2023-08-11T15:52:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.222+0000 7f5231c6a640 1 -- start start 2023-08-11T15:52:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f5231c6a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f522c0b1d10 0x7f522c1ecd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f5231c6a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cada0 0x7f522c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f5231c6a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f522c0cf1f0 0x7f522c0b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f5231c6a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0ca150 con 0x7f522c0cada0 2023-08-11T15:52:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f5231c6a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0b3f40 con 0x7f522c0cf1f0 2023-08-11T15:52:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f5231c6a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0b4220 con 0x7f522c0b1d10 2023-08-11T15:52:06.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f522affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cada0 0x7f522c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f522affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cada0 0x7f522c0b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57314/0 (socket says 172.21.15.27:57314) 2023-08-11T15:52:06.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f522affd640 1 -- 172.21.15.27:0/2785775648 learned_addr learned my addr 172.21.15.27:0/2785775648 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:06.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f522b7fe640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f522c0b1d10 0x7f522c1ecd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f522bfff640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f522c0cf1f0 0x7f522c0b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f522affd640 1 -- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f522c0b1d10 msgr2=0x7f522c1ecd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f522affd640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f522c0b1d10 0x7f522c1ecd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f522affd640 1 -- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f522c0cf1f0 msgr2=0x7f522c0b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.223+0000 7f522affd640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f522c0cf1f0 0x7f522c0b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.224+0000 7f522affd640 1 -- 172.21.15.27:0/2785775648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240094d0 con 0x7f522c0cada0 2023-08-11T15:52:06.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.224+0000 7f522bfff640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f522c0cf1f0 0x7f522c0b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:06.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.224+0000 7f522affd640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cada0 0x7f522c0b2e90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f522000d5f0 tx=0x7f522000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.224+0000 7f5213fff640 1 -- 172.21.15.27:0/2785775648 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200125f0 con 0x7f522c0cada0 2023-08-11T15:52:06.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.224+0000 7f5213fff640 1 -- 172.21.15.27:0/2785775648 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5220012750 con 0x7f522c0cada0 2023-08-11T15:52:06.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.224+0000 7f5213fff640 1 -- 172.21.15.27:0/2785775648 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220012b00 con 0x7f522c0cada0 2023-08-11T15:52:06.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.224+0000 7f5231c6a640 1 -- 172.21.15.27:0/2785775648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c0b4500 con 0x7f522c0cada0 2023-08-11T15:52:06.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.224+0000 7f5231c6a640 1 -- 172.21.15.27:0/2785775648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c1ed290 con 0x7f522c0cada0 2023-08-11T15:52:06.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.225+0000 7f5231c6a640 1 -- 172.21.15.27:0/2785775648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c04e420 con 0x7f522c0cada0 2023-08-11T15:52:06.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.226+0000 7f5213fff640 1 -- 172.21.15.27:0/2785775648 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5220012c60 con 0x7f522c0cada0 2023-08-11T15:52:06.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.228+0000 7f5213fff640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f521806a050 0x7f521806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.228+0000 7f5213fff640 1 -- 172.21.15.27:0/2785775648 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5220088db0 con 0x7f522c0cada0 2023-08-11T15:52:06.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.229+0000 7f522b7fe640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f521806a050 0x7f521806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.230+0000 7f522b7fe640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f521806a050 0x7f521806c510 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f521c0099c0 tx=0x7f521c003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.233+0000 7f5213fff640 1 -- 172.21.15.27:0/2785775648 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f522008d070 con 0x7f522c0cada0 2023-08-11T15:52:06.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.425+0000 7f5231c6a640 1 -- 172.21.15.27:0/2785775648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f522c0ca600 con 0x7f522c0cada0 2023-08-11T15:52:06.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.426+0000 7f5213fff640 1 -- 172.21.15.27:0/2785775648 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52200540d0 con 0x7f522c0cada0 2023-08-11T15:52:06.428 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:06.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 -- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f521806a050 msgr2=0x7f521806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f521806a050 0x7f521806c510 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f521c0099c0 tx=0x7f521c003990 comp rx=0 tx=0).stop 2023-08-11T15:52:06.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 -- 172.21.15.27:0/2785775648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cada0 msgr2=0x7f522c0b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cada0 0x7f522c0b2e90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f522000d5f0 tx=0x7f522000dab0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 -- 172.21.15.27:0/2785775648 shutdown_connections 2023-08-11T15:52:06.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f521806a050 0x7f521806c510 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f522c0cf1f0 0x7f522c0b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f522c0cada0 0x7f522c0b2e90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 --2- 172.21.15.27:0/2785775648 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f522c0b1d10 0x7f522c1ecd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 -- 172.21.15.27:0/2785775648 >> 172.21.15.27:0/2785775648 conn(0x7f522c0ccd10 msgr2=0x7f522c0cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:06.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 -- 172.21.15.27:0/2785775648 shutdown_connections 2023-08-11T15:52:06.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.429+0000 7f5231c6a640 1 -- 172.21.15.27:0/2785775648 wait complete. 2023-08-11T15:52:06.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.690+0000 7f7499298640 1 Processor -- start 2023-08-11T15:52:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.690+0000 7f7499298640 1 -- start start 2023-08-11T15:52:06.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.691+0000 7f7499298640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f74941592e0 0x7f749415d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.691+0000 7f7499298640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f74940ccbc0 0x7f74940afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.691+0000 7f7499298640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74940b0420 con 0x7f749406c960 2023-08-11T15:52:06.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.691+0000 7f7499298640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74941596c0 con 0x7f74941592e0 2023-08-11T15:52:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.691+0000 7f7499298640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74940b09e0 con 0x7f74940ccbc0 2023-08-11T15:52:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.692+0000 7f7493fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f74941592e0 0x7f749415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.692+0000 7f7493fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f74941592e0 0x7f749415d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57316/0 (socket says 172.21.15.27:57316) 2023-08-11T15:52:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.692+0000 7f7493fff640 1 -- 172.21.15.27:0/1851059545 learned_addr learned my addr 172.21.15.27:0/1851059545 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.692+0000 7f74937fe640 1 --2- 172.21.15.27:0/1851059545 >> v2:172.21.15.119:3300/0 conn(0x7f74940ccbc0 0x7f74940afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.692+0000 7f7493fff640 1 -- 172.21.15.27:0/1851059545 >> v2:172.21.15.119:3300/0 conn(0x7f74940ccbc0 msgr2=0x7f74940afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.692+0000 7f7493fff640 1 --2- 172.21.15.27:0/1851059545 >> v2:172.21.15.119:3300/0 conn(0x7f74940ccbc0 0x7f74940afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.692+0000 7f7493fff640 1 -- 172.21.15.27:0/1851059545 >> v1:172.21.15.119:6789/0 conn(0x7f749406c960 legacy=0x7f74940703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.692+0000 7f7493fff640 1 -- 172.21.15.27:0/1851059545 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940b0700 con 0x7f74941592e0 2023-08-11T15:52:06.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.692+0000 7f74937fe640 1 --2- 172.21.15.27:0/1851059545 >> v2:172.21.15.119:3300/0 conn(0x7f74940ccbc0 0x7f74940afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:06.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.694+0000 7f7493fff640 1 --2- 172.21.15.27:0/1851059545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 0x7f749415d730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f74940c9700 tx=0x7f748402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f564ac4d76241f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.694+0000 7f7492ffd640 1 -- 172.21.15.27:0/1851059545 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74840317e0 con 0x7f74941592e0 2023-08-11T15:52:06.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.694+0000 7f7492ffd640 1 -- 172.21.15.27:0/1851059545 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7484031940 con 0x7f74941592e0 2023-08-11T15:52:06.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.694+0000 7f7492ffd640 1 -- 172.21.15.27:0/1851059545 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484031cd0 con 0x7f74941592e0 2023-08-11T15:52:06.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.695+0000 7f7499298640 1 -- 172.21.15.27:0/1851059545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 msgr2=0x7f749415d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.695+0000 7f7499298640 1 --2- 172.21.15.27:0/1851059545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 0x7f749415d730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f74940c9700 tx=0x7f748402b880 comp rx=0 tx=0).stop 2023-08-11T15:52:06.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.695+0000 7f7499298640 1 -- 172.21.15.27:0/1851059545 shutdown_connections 2023-08-11T15:52:06.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.695+0000 7f7499298640 1 --2- 172.21.15.27:0/1851059545 >> v2:172.21.15.119:3300/0 conn(0x7f74940ccbc0 0x7f74940afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.695+0000 7f7499298640 1 --2- 172.21.15.27:0/1851059545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 0x7f749415d730 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.695+0000 7f7499298640 1 -- 172.21.15.27:0/1851059545 >> 172.21.15.27:0/1851059545 conn(0x7f74940caab0 msgr2=0x7f74940cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:06.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.695+0000 7f7499298640 1 -- 172.21.15.27:0/1851059545 shutdown_connections 2023-08-11T15:52:06.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.695+0000 7f7499298640 1 -- 172.21.15.27:0/1851059545 wait complete. 2023-08-11T15:52:06.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.697+0000 7f7499298640 1 Processor -- start 2023-08-11T15:52:06.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.697+0000 7f7499298640 1 -- start start 2023-08-11T15:52:06.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.698+0000 7f7499298640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f749406c960 0x7f749415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.698+0000 7f7499298640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74940ccbc0 0x7f749414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.698+0000 7f7499298640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 0x7f74940b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.698+0000 7f7499298640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749415c2f0 con 0x7f74941592e0 2023-08-11T15:52:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.698+0000 7f7499298640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494159940 con 0x7f74940ccbc0 2023-08-11T15:52:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.698+0000 7f7499298640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7494159c20 con 0x7f749406c960 2023-08-11T15:52:06.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f7498a97640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 0x7f74940b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f7498a97640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 0x7f74940b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57328/0 (socket says 172.21.15.27:57328) 2023-08-11T15:52:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f7498a97640 1 -- 172.21.15.27:0/1219420201 learned_addr learned my addr 172.21.15.27:0/1219420201 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f74937fe640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74940ccbc0 0x7f749414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f7493fff640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f749406c960 0x7f749415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f7498a97640 1 -- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f749406c960 msgr2=0x7f749415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f7498a97640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f749406c960 0x7f749415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f7498a97640 1 -- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74940ccbc0 msgr2=0x7f749414fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f7498a97640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74940ccbc0 0x7f749414fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f7498a97640 1 -- 172.21.15.27:0/1219420201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840094d0 con 0x7f74941592e0 2023-08-11T15:52:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.699+0000 7f7493fff640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f749406c960 0x7f749415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.700+0000 7f7498a97640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 0x7f74940b13d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c780 tx=0x7f748c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.700+0000 7f7490ff9640 1 -- 172.21.15.27:0/1219420201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c004b60 con 0x7f74941592e0 2023-08-11T15:52:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.700+0000 7f7490ff9640 1 -- 172.21.15.27:0/1219420201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c00ce90 con 0x7f74941592e0 2023-08-11T15:52:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.700+0000 7f7490ff9640 1 -- 172.21.15.27:0/1219420201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00a850 con 0x7f74941592e0 2023-08-11T15:52:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.700+0000 7f7499298640 1 -- 172.21.15.27:0/1219420201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7494159f00 con 0x7f74941592e0 2023-08-11T15:52:06.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.700+0000 7f7499298640 1 -- 172.21.15.27:0/1219420201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7494150990 con 0x7f74941592e0 2023-08-11T15:52:06.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.701+0000 7f7499298640 1 -- 172.21.15.27:0/1219420201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749404e420 con 0x7f74941592e0 2023-08-11T15:52:06.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.702+0000 7f7490ff9640 1 -- 172.21.15.27:0/1219420201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f748c00a9b0 con 0x7f74941592e0 2023-08-11T15:52:06.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.703+0000 7f7490ff9640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f748006a050 0x7f748006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.703+0000 7f7490ff9640 1 -- 172.21.15.27:0/1219420201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f748c088aa0 con 0x7f74941592e0 2023-08-11T15:52:06.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.704+0000 7f7493fff640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f748006a050 0x7f748006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.705+0000 7f7493fff640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f748006a050 0x7f748006c510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7484036000 tx=0x7f748403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.708+0000 7f7490ff9640 1 -- 172.21.15.27:0/1219420201 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f748c053e50 con 0x7f74941592e0 2023-08-11T15:52:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.902+0000 7f7499298640 1 -- 172.21.15.27:0/1219420201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f74940cc1f0 con 0x7f74941592e0 2023-08-11T15:52:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.903+0000 7f7490ff9640 1 -- 172.21.15.27:0/1219420201 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f748c057470 con 0x7f74941592e0 2023-08-11T15:52:06.905 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:06.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 -- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f748006a050 msgr2=0x7f748006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f748006a050 0x7f748006c510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7484036000 tx=0x7f748403d040 comp rx=0 tx=0).stop 2023-08-11T15:52:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 -- 172.21.15.27:0/1219420201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 msgr2=0x7f74940b13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 0x7f74940b13d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c780 tx=0x7f748c00cc40 comp rx=0 tx=0).stop 2023-08-11T15:52:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 -- 172.21.15.27:0/1219420201 shutdown_connections 2023-08-11T15:52:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f748006a050 0x7f748006c510 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74941592e0 0x7f74940b13d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74940ccbc0 0x7f749414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 --2- 172.21.15.27:0/1219420201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f749406c960 0x7f749415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 -- 172.21.15.27:0/1219420201 >> 172.21.15.27:0/1219420201 conn(0x7f74940caab0 msgr2=0x7f7494071240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 -- 172.21.15.27:0/1219420201 shutdown_connections 2023-08-11T15:52:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:06.906+0000 7f7499298640 1 -- 172.21.15.27:0/1219420201 wait complete. 2023-08-11T15:52:06.925 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.138+0000 7f16a2316640 1 Processor -- start 2023-08-11T15:52:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.138+0000 7f16a2316640 1 -- start start 2023-08-11T15:52:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.139+0000 7f16a2316640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f169c0cd7f0 0x7f169c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.139+0000 7f16a2316640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169c15c970 con 0x7f169c0d1b90 2023-08-11T15:52:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.140+0000 7f16a2316640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169c0d0250 con 0x7f169c15c590 2023-08-11T15:52:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.140+0000 7f16a2316640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c0d0ed0 con 0x7f169c0cd7f0 2023-08-11T15:52:07.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.140+0000 7f16a0b13640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f169c0d1b90 0x7f169c0cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51810/0 (socket says 172.21.15.27:51810) 2023-08-11T15:52:07.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.140+0000 7f16a0b13640 1 -- 172.21.15.27:0/2035254833 learned_addr learned my addr 172.21.15.27:0/2035254833 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:07.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.140+0000 7f16a1b15640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f169c0cd7f0 0x7f169c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.140+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340162852 0 0) 0x7f169c15c970 con 0x7f169c0d1b90 2023-08-11T15:52:07.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.140+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1684003520 con 0x7f169c0d1b90 2023-08-11T15:52:07.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3636494339 0 0) 0x7f1684003520 con 0x7f169c0d1b90 2023-08-11T15:52:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f169c0cd7f0 msgr2=0x7f169c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f169c0cd7f0 0x7f169c0b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 >> v1:172.21.15.119:6789/0 conn(0x7f169c15c590 legacy=0x7f169c0d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c0d0530 con 0x7f169c0d1b90 2023-08-11T15:52:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f1690001670 con 0x7f169c0d1b90 2023-08-11T15:52:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 >> v1:172.21.15.27:6789/0 conn(0x7f169c0d1b90 legacy=0x7f169c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16840039a0 0x7f1684003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16840047c0 0x7f1684004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f169c0cd7f0 0x7f1684005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c0d0ed0 con 0x7f16840047c0 2023-08-11T15:52:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168400bb40 con 0x7f16840039a0 2023-08-11T15:52:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168400bd00 con 0x7f169c0cd7f0 2023-08-11T15:52:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f16a0b13640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16840047c0 0x7f1684004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.141+0000 7f16a1314640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16840039a0 0x7f1684003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.142+0000 7f16a0b13640 1 -- 172.21.15.27:0/2035254833 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f169c0cd7f0 msgr2=0x7f1684005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:07.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.142+0000 7f16a0b13640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f169c0cd7f0 0x7f1684005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.142+0000 7f16a0b13640 1 -- 172.21.15.27:0/2035254833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16840039a0 msgr2=0x7f1684003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:07.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.142+0000 7f16a0b13640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16840039a0 0x7f1684003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.142+0000 7f16a0b13640 1 -- 172.21.15.27:0/2035254833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684003520 con 0x7f16840047c0 2023-08-11T15:52:07.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.143+0000 7f16a0b13640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16840047c0 0x7f1684004c40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f169000c210 tx=0x7f16840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213d37cc6f402a5c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.143+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690005ad0 con 0x7f16840047c0 2023-08-11T15:52:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.143+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1690005c30 con 0x7f16840047c0 2023-08-11T15:52:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.143+0000 7f169bfff640 1 -- 172.21.15.27:0/2035254833 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169002c880 con 0x7f16840047c0 2023-08-11T15:52:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.143+0000 7f16a2316640 1 -- 172.21.15.27:0/2035254833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16840047c0 msgr2=0x7f1684004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.143+0000 7f16a2316640 1 --2- 172.21.15.27:0/2035254833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16840047c0 0x7f1684004c40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f169000c210 tx=0x7f16840057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.143+0000 7f16a1b15640 1 -- 172.21.15.27:0/2035254833 reap_dead start 2023-08-11T15:52:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.143+0000 7f16a2316640 1 -- 172.21.15.27:0/2035254833 shutdown_connections 2023-08-11T15:52:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.143+0000 7f16a2316640 1 -- 172.21.15.27:0/2035254833 >> 172.21.15.27:0/2035254833 conn(0x7f169c06a250 msgr2=0x7f169c06a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.144+0000 7f16a2316640 1 -- 172.21.15.27:0/2035254833 shutdown_connections 2023-08-11T15:52:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.144+0000 7f16a2316640 1 -- 172.21.15.27:0/2035254833 wait complete. 2023-08-11T15:52:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.145+0000 7f16a2316640 1 Processor -- start 2023-08-11T15:52:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.145+0000 7f16a2316640 1 -- start start 2023-08-11T15:52:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.146+0000 7f16a2316640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16840047c0 0x7f169c0c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.146+0000 7f16a2316640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c0b2cc0 0x7f169c14d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.146+0000 7f16a2316640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f169c0ceb10 0x7f169c0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.146+0000 7f16a2316640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c0c5d20 con 0x7f169c0b2cc0 2023-08-11T15:52:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.146+0000 7f16a2316640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c0cf4d0 con 0x7f169c0ceb10 2023-08-11T15:52:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.146+0000 7f16a2316640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c14b110 con 0x7f16840047c0 2023-08-11T15:52:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a0b13640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c0b2cc0 0x7f169c14d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a0b13640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c0b2cc0 0x7f169c14d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57346/0 (socket says 172.21.15.27:57346) 2023-08-11T15:52:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a0b13640 1 -- 172.21.15.27:0/3458677894 learned_addr learned my addr 172.21.15.27:0/3458677894 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a1314640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16840047c0 0x7f169c0c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a1b15640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f169c0ceb10 0x7f169c0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a0b13640 1 -- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16840047c0 msgr2=0x7f169c0c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a0b13640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16840047c0 0x7f169c0c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a0b13640 1 -- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f169c0ceb10 msgr2=0x7f169c0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a0b13640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f169c0ceb10 0x7f169c0cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a0b13640 1 -- 172.21.15.27:0/3458677894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c0c5fc0 con 0x7f169c0b2cc0 2023-08-11T15:52:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a1314640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16840047c0 0x7f169c0c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.147+0000 7f16a0b13640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c0b2cc0 0x7f169c14d960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f169002c420 tx=0x7f1690001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:07.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.148+0000 7f1699ffb640 1 -- 172.21.15.27:0/3458677894 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690039030 con 0x7f169c0b2cc0 2023-08-11T15:52:07.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.148+0000 7f1699ffb640 1 -- 172.21.15.27:0/3458677894 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169002cc90 con 0x7f169c0b2cc0 2023-08-11T15:52:07.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.148+0000 7f1699ffb640 1 -- 172.21.15.27:0/3458677894 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169003c040 con 0x7f169c0b2cc0 2023-08-11T15:52:07.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.148+0000 7f16a2316640 1 -- 172.21.15.27:0/3458677894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169c0b34b0 con 0x7f169c0b2cc0 2023-08-11T15:52:07.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.148+0000 7f16a2316640 1 -- 172.21.15.27:0/3458677894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c0b3a70 con 0x7f169c0b2cc0 2023-08-11T15:52:07.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.149+0000 7f16a2316640 1 -- 172.21.15.27:0/3458677894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169c04e420 con 0x7f169c0b2cc0 2023-08-11T15:52:07.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.150+0000 7f1699ffb640 1 -- 172.21.15.27:0/3458677894 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1690032070 con 0x7f169c0b2cc0 2023-08-11T15:52:07.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.151+0000 7f1699ffb640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f168806a050 0x7f168806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.151+0000 7f1699ffb640 1 -- 172.21.15.27:0/3458677894 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f16900a9c40 con 0x7f169c0b2cc0 2023-08-11T15:52:07.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.151+0000 7f16a1314640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f168806a050 0x7f168806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.152+0000 7f16a1314640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f168806a050 0x7f168806c510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f168c0015f0 tx=0x7f168c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:07.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.155+0000 7f1699ffb640 1 -- 172.21.15.27:0/3458677894 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f1690074ff0 con 0x7f169c0b2cc0 2023-08-11T15:52:07.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.342+0000 7f16a2316640 1 -- 172.21.15.27:0/3458677894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f169c0cff90 con 0x7f169c0b2cc0 2023-08-11T15:52:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.344+0000 7f1699ffb640 1 -- 172.21.15.27:0/3458677894 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f1690001630 con 0x7f169c0b2cc0 2023-08-11T15:52:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:07.347 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:07.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 -- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f168806a050 msgr2=0x7f168806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f168806a050 0x7f168806c510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f168c0015f0 tx=0x7f168c009040 comp rx=0 tx=0).stop 2023-08-11T15:52:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 -- 172.21.15.27:0/3458677894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c0b2cc0 msgr2=0x7f169c14d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c0b2cc0 0x7f169c14d960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f169002c420 tx=0x7f1690001a00 comp rx=0 tx=0).stop 2023-08-11T15:52:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 -- 172.21.15.27:0/3458677894 shutdown_connections 2023-08-11T15:52:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f168806a050 0x7f168806c510 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f169c0ceb10 0x7f169c0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f169c0b2cc0 0x7f169c14d960 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 --2- 172.21.15.27:0/3458677894 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16840047c0 0x7f169c0c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 -- 172.21.15.27:0/3458677894 >> 172.21.15.27:0/3458677894 conn(0x7f169c06a250 msgr2=0x7f169c0ca940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 -- 172.21.15.27:0/3458677894 shutdown_connections 2023-08-11T15:52:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.347+0000 7f16a2316640 1 -- 172.21.15.27:0/3458677894 wait complete. 2023-08-11T15:52:07.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:07.369 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.605+0000 7f745190f640 1 Processor -- start 2023-08-11T15:52:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.605+0000 7f745190f640 1 -- start start 2023-08-11T15:52:07.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.606+0000 7f745190f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c15c760 0x7f744c0d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.606+0000 7f745190f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744c06bc10 con 0x7f744c06b670 2023-08-11T15:52:07.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.606+0000 7f745190f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744c06bef0 con 0x7f744c0d2a20 2023-08-11T15:52:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.606+0000 7f745190f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744c06c790 con 0x7f744c15c760 2023-08-11T15:52:07.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.607+0000 7f744a7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f744c06b670 0x7f744c0d2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51818/0 (socket says 172.21.15.27:51818) 2023-08-11T15:52:07.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.607+0000 7f744a7fc640 1 -- 172.21.15.27:0/3638256669 learned_addr learned my addr 172.21.15.27:0/3638256669 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:07.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.607+0000 7f744affd640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c15c760 0x7f744c0d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.607+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599895226 0 0) 0x7f744c06bc10 con 0x7f744c06b670 2023-08-11T15:52:07.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.607+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7438003520 con 0x7f744c06b670 2023-08-11T15:52:07.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1940657513 0 0) 0x7f7438003520 con 0x7f744c06b670 2023-08-11T15:52:07.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c15c760 msgr2=0x7f744c0d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:07.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c15c760 0x7f744c0d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 >> v1:172.21.15.119:6789/0 conn(0x7f744c0d2a20 legacy=0x7f744c0af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c06c1d0 con 0x7f744c06b670 2023-08-11T15:52:07.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f7434001670 con 0x7f744c06b670 2023-08-11T15:52:07.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 >> v1:172.21.15.27:6789/0 conn(0x7f744c06b670 legacy=0x7f744c0d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 0x7f7438003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74380047c0 0x7f7438004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c15c760 0x7f74380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743800ba80 con 0x7f74380039a0 2023-08-11T15:52:07.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743800bbf0 con 0x7f74380047c0 2023-08-11T15:52:07.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743800bdb0 con 0x7f744c15c760 2023-08-11T15:52:07.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.608+0000 7f744affd640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 0x7f7438003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.609+0000 7f744a7fc640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74380047c0 0x7f7438004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.609+0000 7f744affd640 1 -- 172.21.15.27:0/3638256669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c15c760 msgr2=0x7f74380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.609+0000 7f744affd640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c15c760 0x7f74380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.609+0000 7f744affd640 1 -- 172.21.15.27:0/3638256669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74380047c0 msgr2=0x7f7438004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.609+0000 7f744affd640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74380047c0 0x7f7438004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.609+0000 7f744affd640 1 -- 172.21.15.27:0/3638256669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438003520 con 0x7f74380039a0 2023-08-11T15:52:07.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.610+0000 7f744affd640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 0x7f7438003dc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f743c009fc0 tx=0x7f74380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ef21a39559bf48 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:07.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.610+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c02cc60 con 0x7f74380039a0 2023-08-11T15:52:07.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.610+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c00f040 con 0x7f74380039a0 2023-08-11T15:52:07.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.610+0000 7f7449ffb640 1 -- 172.21.15.27:0/3638256669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c034440 con 0x7f74380039a0 2023-08-11T15:52:07.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.611+0000 7f745190f640 1 -- 172.21.15.27:0/3638256669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 msgr2=0x7f7438003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.611+0000 7f745190f640 1 --2- 172.21.15.27:0/3638256669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 0x7f7438003dc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f743c009fc0 tx=0x7f74380057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.611+0000 7f744b7fe640 1 -- 172.21.15.27:0/3638256669 reap_dead start 2023-08-11T15:52:07.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.611+0000 7f745190f640 1 -- 172.21.15.27:0/3638256669 shutdown_connections 2023-08-11T15:52:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.611+0000 7f745190f640 1 -- 172.21.15.27:0/3638256669 >> 172.21.15.27:0/3638256669 conn(0x7f744c089ff0 msgr2=0x7f744c09b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.611+0000 7f745190f640 1 -- 172.21.15.27:0/3638256669 shutdown_connections 2023-08-11T15:52:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.611+0000 7f745190f640 1 -- 172.21.15.27:0/3638256669 wait complete. 2023-08-11T15:52:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.613+0000 7f745190f640 1 Processor -- start 2023-08-11T15:52:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.613+0000 7f745190f640 1 -- start start 2023-08-11T15:52:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.614+0000 7f745190f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 0x7f744c0b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.614+0000 7f745190f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c158040 0x7f744c1f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.614+0000 7f745190f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f744c0ce4b0 0x7f744c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.614+0000 7f745190f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c0cee70 con 0x7f74380039a0 2023-08-11T15:52:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.614+0000 7f745190f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c0cc290 con 0x7f744c0ce4b0 2023-08-11T15:52:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.614+0000 7f745190f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744c1f3160 con 0x7f744c158040 2023-08-11T15:52:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.614+0000 7f744affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 0x7f744c0b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.614+0000 7f744affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 0x7f744c0b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57370/0 (socket says 172.21.15.27:57370) 2023-08-11T15:52:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.614+0000 7f744affd640 1 -- 172.21.15.27:0/2783814866 learned_addr learned my addr 172.21.15.27:0/2783814866 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:07.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.615+0000 7f744affd640 1 -- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c158040 msgr2=0x7f744c1f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.615+0000 7f744a7fc640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c158040 0x7f744c1f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.615+0000 7f744affd640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c158040 0x7f744c1f2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.615+0000 7f744b7fe640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f744c0ce4b0 0x7f744c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.615+0000 7f744affd640 1 -- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f744c0ce4b0 msgr2=0x7f744c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.615+0000 7f744affd640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f744c0ce4b0 0x7f744c0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.615+0000 7f744affd640 1 -- 172.21.15.27:0/2783814866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c1f3440 con 0x7f74380039a0 2023-08-11T15:52:07.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.615+0000 7f744affd640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 0x7f744c0b5290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f743c00e230 tx=0x7f743c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:07.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.615+0000 7f7443fff640 1 -- 172.21.15.27:0/2783814866 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c034bd0 con 0x7f74380039a0 2023-08-11T15:52:07.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.616+0000 7f7443fff640 1 -- 172.21.15.27:0/2783814866 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c034d30 con 0x7f74380039a0 2023-08-11T15:52:07.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.616+0000 7f745190f640 1 -- 172.21.15.27:0/2783814866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744c1f3720 con 0x7f74380039a0 2023-08-11T15:52:07.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.616+0000 7f7443fff640 1 -- 172.21.15.27:0/2783814866 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c034ea0 con 0x7f74380039a0 2023-08-11T15:52:07.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.616+0000 7f745190f640 1 -- 172.21.15.27:0/2783814866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c1f3ce0 con 0x7f74380039a0 2023-08-11T15:52:07.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.617+0000 7f745190f640 1 -- 172.21.15.27:0/2783814866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744c04e420 con 0x7f74380039a0 2023-08-11T15:52:07.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.618+0000 7f7443fff640 1 -- 172.21.15.27:0/2783814866 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f743c040450 con 0x7f74380039a0 2023-08-11T15:52:07.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.619+0000 7f7443fff640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f742c06a050 0x7f742c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:07.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.619+0000 7f7443fff640 1 -- 172.21.15.27:0/2783814866 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f743c0aafd0 con 0x7f74380039a0 2023-08-11T15:52:07.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.620+0000 7f744a7fc640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f742c06a050 0x7f742c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:07.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.621+0000 7f744a7fc640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f742c06a050 0x7f742c06c510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7434001200 tx=0x7f7434009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:07.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.623+0000 7f7443fff640 1 -- 172.21.15.27:0/2783814866 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f743c076390 con 0x7f74380039a0 2023-08-11T15:52:07.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.818+0000 7f745190f640 1 -- 172.21.15.27:0/2783814866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f744c0d34b0 con 0x7f74380039a0 2023-08-11T15:52:07.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.819+0000 7f7443fff640 1 -- 172.21.15.27:0/2783814866 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f743c0799b0 con 0x7f74380039a0 2023-08-11T15:52:07.820 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:07.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.821+0000 7f745190f640 1 -- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f742c06a050 msgr2=0x7f742c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.821+0000 7f745190f640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f742c06a050 0x7f742c06c510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7434001200 tx=0x7f7434009040 comp rx=0 tx=0).stop 2023-08-11T15:52:07.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.821+0000 7f745190f640 1 -- 172.21.15.27:0/2783814866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 msgr2=0x7f744c0b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:07.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.821+0000 7f745190f640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 0x7f744c0b5290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f743c00e230 tx=0x7f743c009750 comp rx=0 tx=0).stop 2023-08-11T15:52:07.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.822+0000 7f745190f640 1 -- 172.21.15.27:0/2783814866 shutdown_connections 2023-08-11T15:52:07.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.822+0000 7f745190f640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f742c06a050 0x7f742c06c510 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.822+0000 7f745190f640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f744c158040 0x7f744c1f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.822+0000 7f745190f640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f744c0ce4b0 0x7f744c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.822+0000 7f745190f640 1 --2- 172.21.15.27:0/2783814866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380039a0 0x7f744c0b5290 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.822+0000 7f745190f640 1 -- 172.21.15.27:0/2783814866 >> 172.21.15.27:0/2783814866 conn(0x7f744c089ff0 msgr2=0x7f744c0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.822+0000 7f745190f640 1 -- 172.21.15.27:0/2783814866 shutdown_connections 2023-08-11T15:52:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:07.822+0000 7f745190f640 1 -- 172.21.15.27:0/2783814866 wait complete. 2023-08-11T15:52:07.842 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.085+0000 7fe9deb09640 1 Processor -- start 2023-08-11T15:52:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.085+0000 7fe9deb09640 1 -- start start 2023-08-11T15:52:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.087+0000 7fe9deb09640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe9d813ff20 0x7fe9d80cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.087+0000 7fe9deb09640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d80cd470 con 0x7fe9d813ce80 2023-08-11T15:52:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.087+0000 7fe9deb09640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d80cd750 con 0x7fe9d813d260 2023-08-11T15:52:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.087+0000 7fe9deb09640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d8140670 con 0x7fe9d813ff20 2023-08-11T15:52:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.088+0000 7fe9dd306640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe9d813ff20 0x7fe9d80cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.088+0000 7fe9ddb07640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe9d813ce80 0x7fe9d813f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51834/0 (socket says 172.21.15.27:51834) 2023-08-11T15:52:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.088+0000 7fe9ddb07640 1 -- 172.21.15.27:0/2096407538 learned_addr learned my addr 172.21.15.27:0/2096407538 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.088+0000 7fe9dd306640 1 -- 172.21.15.27:0/2096407538 >> v1:172.21.15.119:6789/0 conn(0x7fe9d813d260 legacy=0x7fe9d80ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.088+0000 7fe9dd306640 1 -- 172.21.15.27:0/2096407538 >> v1:172.21.15.27:6789/0 conn(0x7fe9d813ce80 legacy=0x7fe9d813f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.088+0000 7fe9dd306640 1 -- 172.21.15.27:0/2096407538 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d80ad7d0 con 0x7fe9d813ff20 2023-08-11T15:52:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.088+0000 7fe9dcb05640 1 -- 172.21.15.27:0/2096407538 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189562705 0 0) 0x7fe9d80cd470 con 0x7fe9d813ce80 2023-08-11T15:52:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.090+0000 7fe9dd306640 1 --2- 172.21.15.27:0/2096407538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 0x7fe9d80cc720 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80097d0 tx=0x7fe9c802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606541eeae03c590 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.090+0000 7fe9dcb05640 1 -- 172.21.15.27:0/2096407538 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8030800 con 0x7fe9d813ff20 2023-08-11T15:52:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.090+0000 7fe9dcb05640 1 -- 172.21.15.27:0/2096407538 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c8030960 con 0x7fe9d813ff20 2023-08-11T15:52:08.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.090+0000 7fe9dcb05640 1 -- 172.21.15.27:0/2096407538 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8030cf0 con 0x7fe9d813ff20 2023-08-11T15:52:08.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.091+0000 7fe9deb09640 1 -- 172.21.15.27:0/2096407538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 msgr2=0x7fe9d80cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.091+0000 7fe9deb09640 1 --2- 172.21.15.27:0/2096407538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 0x7fe9d80cc720 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80097d0 tx=0x7fe9c802b940 comp rx=0 tx=0).stop 2023-08-11T15:52:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.091+0000 7fe9deb09640 1 -- 172.21.15.27:0/2096407538 shutdown_connections 2023-08-11T15:52:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.091+0000 7fe9deb09640 1 --2- 172.21.15.27:0/2096407538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 0x7fe9d80cc720 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.091+0000 7fe9deb09640 1 -- 172.21.15.27:0/2096407538 >> 172.21.15.27:0/2096407538 conn(0x7fe9d8069800 msgr2=0x7fe9d8069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.091+0000 7fe9deb09640 1 -- 172.21.15.27:0/2096407538 shutdown_connections 2023-08-11T15:52:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.091+0000 7fe9deb09640 1 -- 172.21.15.27:0/2096407538 wait complete. 2023-08-11T15:52:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.093+0000 7fe9deb09640 1 Processor -- start 2023-08-11T15:52:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.093+0000 7fe9deb09640 1 -- start start 2023-08-11T15:52:08.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.094+0000 7fe9deb09640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe9d813ce80 0x7fe9d8151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.094+0000 7fe9deb09640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d813d260 0x7fe9d813c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.094+0000 7fe9deb09640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 0x7fe9d81f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.094+0000 7fe9deb09640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d8159f20 con 0x7fe9d813ff20 2023-08-11T15:52:08.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.094+0000 7fe9deb09640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d813a710 con 0x7fe9d813d260 2023-08-11T15:52:08.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.094+0000 7fe9deb09640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80b2c00 con 0x7fe9d813ce80 2023-08-11T15:52:08.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.094+0000 7fe9de308640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 0x7fe9d81f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.095+0000 7fe9de308640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 0x7fe9d81f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57374/0 (socket says 172.21.15.27:57374) 2023-08-11T15:52:08.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.095+0000 7fe9de308640 1 -- 172.21.15.27:0/1436568706 learned_addr learned my addr 172.21.15.27:0/1436568706 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.095+0000 7fe9ddb07640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe9d813ce80 0x7fe9d8151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.095+0000 7fe9dd306640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d813d260 0x7fe9d813c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.095+0000 7fe9de308640 1 -- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe9d813ce80 msgr2=0x7fe9d8151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.095+0000 7fe9de308640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe9d813ce80 0x7fe9d8151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.095+0000 7fe9de308640 1 -- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d813d260 msgr2=0x7fe9d813c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.095+0000 7fe9de308640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d813d260 0x7fe9d813c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.095+0000 7fe9de308640 1 -- 172.21.15.27:0/1436568706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c8009400 con 0x7fe9d813ff20 2023-08-11T15:52:08.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.095+0000 7fe9de308640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 0x7fe9d81f5a60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000c7a0 tx=0x7fe9d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:08.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.096+0000 7fe9ce7fc640 1 -- 172.21.15.27:0/1436568706 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0003bc0 con 0x7fe9d813ff20 2023-08-11T15:52:08.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.096+0000 7fe9ce7fc640 1 -- 172.21.15.27:0/1436568706 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d0003d40 con 0x7fe9d813ff20 2023-08-11T15:52:08.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.096+0000 7fe9ce7fc640 1 -- 172.21.15.27:0/1436568706 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d000aa30 con 0x7fe9d813ff20 2023-08-11T15:52:08.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.096+0000 7fe9deb09640 1 -- 172.21.15.27:0/1436568706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d813c6e0 con 0x7fe9d813ff20 2023-08-11T15:52:08.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.096+0000 7fe9deb09640 1 -- 172.21.15.27:0/1436568706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d81f5fa0 con 0x7fe9d813ff20 2023-08-11T15:52:08.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.097+0000 7fe9deb09640 1 -- 172.21.15.27:0/1436568706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d813a850 con 0x7fe9d813ff20 2023-08-11T15:52:08.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.098+0000 7fe9ce7fc640 1 -- 172.21.15.27:0/1436568706 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe9d000ab90 con 0x7fe9d813ff20 2023-08-11T15:52:08.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.100+0000 7fe9ce7fc640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe9bc06a050 0x7fe9bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.100+0000 7fe9ce7fc640 1 -- 172.21.15.27:0/1436568706 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe9d0088760 con 0x7fe9d813ff20 2023-08-11T15:52:08.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.100+0000 7fe9ddb07640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe9bc06a050 0x7fe9bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.101+0000 7fe9ddb07640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe9bc06a050 0x7fe9bc06c510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4000f60 tx=0x7fe9d4007b90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.104+0000 7fe9ce7fc640 1 -- 172.21.15.27:0/1436568706 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe9d813a850 con 0x7fe9d813ff20 2023-08-11T15:52:08.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.299+0000 7fe9deb09640 1 -- 172.21.15.27:0/1436568706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe9d813a850 con 0x7fe9d813ff20 2023-08-11T15:52:08.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.299+0000 7fe9ce7fc640 1 -- 172.21.15.27:0/1436568706 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe9d813a850 con 0x7fe9d813ff20 2023-08-11T15:52:08.304 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 -- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe9bc06a050 msgr2=0x7fe9bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe9bc06a050 0x7fe9bc06c510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4000f60 tx=0x7fe9d4007b90 comp rx=0 tx=0).stop 2023-08-11T15:52:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 -- 172.21.15.27:0/1436568706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 msgr2=0x7fe9d81f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 0x7fe9d81f5a60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000c7a0 tx=0x7fe9d000cc60 comp rx=0 tx=0).stop 2023-08-11T15:52:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 -- 172.21.15.27:0/1436568706 shutdown_connections 2023-08-11T15:52:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe9bc06a050 0x7fe9bc06c510 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9d813ff20 0x7fe9d81f5a60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d813d260 0x7fe9d813c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 --2- 172.21.15.27:0/1436568706 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe9d813ce80 0x7fe9d8151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 -- 172.21.15.27:0/1436568706 >> 172.21.15.27:0/1436568706 conn(0x7fe9d8069800 msgr2=0x7fe9d80cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 -- 172.21.15.27:0/1436568706 shutdown_connections 2023-08-11T15:52:08.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.302+0000 7fe9deb09640 1 -- 172.21.15.27:0/1436568706 wait complete. 2023-08-11T15:52:08.323 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.563+0000 7f982b3c9640 1 Processor -- start 2023-08-11T15:52:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.564+0000 7f982b3c9640 1 -- start start 2023-08-11T15:52:08.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.565+0000 7f982b3c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f98240cb110 0x7f98240cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.566+0000 7f982b3c9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98241600d0 con 0x7f98240cc170 2023-08-11T15:52:08.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.566+0000 7f982b3c9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9824160320 con 0x7f9824159f80 2023-08-11T15:52:08.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.566+0000 7f982b3c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9824160600 con 0x7f98240cb110 2023-08-11T15:52:08.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.566+0000 7f9829966640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f98240cc170 0x7f982415cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51844/0 (socket says 172.21.15.27:51844) 2023-08-11T15:52:08.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.566+0000 7f9829966640 1 -- 172.21.15.27:0/3291718455 learned_addr learned my addr 172.21.15.27:0/3291718455 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:08.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.566+0000 7f9828964640 1 --2- 172.21.15.27:0/3291718455 >> v2:172.21.15.119:3300/0 conn(0x7f98240cb110 0x7f98240cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362439967 0 0) 0x7f98241600d0 con 0x7f98240cc170 2023-08-11T15:52:08.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9810003520 con 0x7f98240cc170 2023-08-11T15:52:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308398425 0 0) 0x7f9824160320 con 0x7f9824159f80 2023-08-11T15:52:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98241600d0 con 0x7f9824159f80 2023-08-11T15:52:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1615720852 0 0) 0x7f9810003520 con 0x7f98240cc170 2023-08-11T15:52:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 >> v2:172.21.15.119:3300/0 conn(0x7f98240cb110 msgr2=0x7f98240cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 --2- 172.21.15.27:0/3291718455 >> v2:172.21.15.119:3300/0 conn(0x7f98240cb110 0x7f98240cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 >> v1:172.21.15.119:6789/0 conn(0x7f9824159f80 legacy=0x7f98240caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98241608e0 con 0x7f98240cc170 2023-08-11T15:52:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f981c0019f0 con 0x7f98240cc170 2023-08-11T15:52:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 >> v1:172.21.15.27:6789/0 conn(0x7f98240cc170 legacy=0x7f982415cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9810003b00 0x7f9810003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98100047c0 0x7f9810004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9810005180 0x7f981000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9824160600 con 0x7f9810003b00 2023-08-11T15:52:08.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981000c000 con 0x7f98100047c0 2023-08-11T15:52:08.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.567+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981000c1c0 con 0x7f9810005180 2023-08-11T15:52:08.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.568+0000 7f9829165640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9810003b00 0x7f9810003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.568+0000 7f9829966640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9810005180 0x7f981000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.568+0000 7f9828964640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98100047c0 0x7f9810004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.568+0000 7f9829165640 1 -- 172.21.15.27:0/3291718455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9810005180 msgr2=0x7f981000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:08.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.568+0000 7f9829165640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9810005180 0x7f981000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.568+0000 7f9829165640 1 -- 172.21.15.27:0/3291718455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98100047c0 msgr2=0x7f9810004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:08.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.568+0000 7f9829165640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98100047c0 0x7f9810004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.568+0000 7f9829165640 1 -- 172.21.15.27:0/3291718455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9810003520 con 0x7f9810003b00 2023-08-11T15:52:08.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.568+0000 7f9829966640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9810005180 0x7f981000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.568+0000 7f9829966640 1 -- 172.21.15.27:0/3291718455 reap_dead start 2023-08-11T15:52:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.569+0000 7f9829165640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9810003b00 0x7f9810003f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f98200018d0 tx=0x7f98100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c730d6c7937c9b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.570+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98200052d0 con 0x7f9810003b00 2023-08-11T15:52:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.570+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9820005430 con 0x7f9810003b00 2023-08-11T15:52:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.570+0000 7f981bfff640 1 -- 172.21.15.27:0/3291718455 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820003810 con 0x7f9810003b00 2023-08-11T15:52:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.570+0000 7f982b3c9640 1 -- 172.21.15.27:0/3291718455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9810003b00 msgr2=0x7f9810003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.570+0000 7f982b3c9640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9810003b00 0x7f9810003f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f98200018d0 tx=0x7f98100057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.570+0000 7f982b3c9640 1 -- 172.21.15.27:0/3291718455 shutdown_connections 2023-08-11T15:52:08.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.570+0000 7f982b3c9640 1 --2- 172.21.15.27:0/3291718455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9810003b00 0x7f9810003f00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.570+0000 7f982b3c9640 1 -- 172.21.15.27:0/3291718455 >> 172.21.15.27:0/3291718455 conn(0x7f98240afbe0 msgr2=0x7f98240affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:08.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.571+0000 7f982b3c9640 1 -- 172.21.15.27:0/3291718455 shutdown_connections 2023-08-11T15:52:08.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.571+0000 7f982b3c9640 1 -- 172.21.15.27:0/3291718455 wait complete. 2023-08-11T15:52:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.573+0000 7f982b3c9640 1 Processor -- start 2023-08-11T15:52:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.573+0000 7f982b3c9640 1 -- start start 2023-08-11T15:52:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.574+0000 7f982b3c9640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9810003b00 0x7f98240c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.574+0000 7f982b3c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98241550a0 0x7f9824155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.574+0000 7f982b3c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824155a60 0x7f9824144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.574+0000 7f982b3c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9824143ef0 con 0x7f98241550a0 2023-08-11T15:52:08.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.574+0000 7f982b3c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98240b2450 con 0x7f9824155a60 2023-08-11T15:52:08.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.574+0000 7f982b3c9640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98240b2730 con 0x7f9810003b00 2023-08-11T15:52:08.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.574+0000 7f9828964640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98241550a0 0x7f9824155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.574+0000 7f9828964640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98241550a0 0x7f9824155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57396/0 (socket says 172.21.15.27:57396) 2023-08-11T15:52:08.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.574+0000 7f9828964640 1 -- 172.21.15.27:0/2645657025 learned_addr learned my addr 172.21.15.27:0/2645657025 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.574+0000 7f9829165640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9810003b00 0x7f98240c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f9829966640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824155a60 0x7f9824144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f9828964640 1 -- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9810003b00 msgr2=0x7f98240c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f9828964640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9810003b00 0x7f98240c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f9828964640 1 -- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824155a60 msgr2=0x7f9824144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f9828964640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824155a60 0x7f9824144380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f9828964640 1 -- 172.21.15.27:0/2645657025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240b2a10 con 0x7f98241550a0 2023-08-11T15:52:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f9828964640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98241550a0 0x7f9824155520 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f98140108e0 tx=0x7f9814010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f9819ffb640 1 -- 172.21.15.27:0/2645657025 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981400eb60 con 0x7f98241550a0 2023-08-11T15:52:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f9819ffb640 1 -- 172.21.15.27:0/2645657025 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981400ecc0 con 0x7f98241550a0 2023-08-11T15:52:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f9819ffb640 1 -- 172.21.15.27:0/2645657025 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981401c400 con 0x7f98241550a0 2023-08-11T15:52:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f982b3c9640 1 -- 172.21.15.27:0/2645657025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98240b2cf0 con 0x7f98241550a0 2023-08-11T15:52:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.575+0000 7f982b3c9640 1 -- 172.21.15.27:0/2645657025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98241448c0 con 0x7f98241550a0 2023-08-11T15:52:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.577+0000 7f982b3c9640 1 -- 172.21.15.27:0/2645657025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9824144060 con 0x7f98241550a0 2023-08-11T15:52:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.578+0000 7f9819ffb640 1 -- 172.21.15.27:0/2645657025 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f981401c560 con 0x7f98241550a0 2023-08-11T15:52:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.579+0000 7f9819ffb640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f980806a080 0x7f980806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:08.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.579+0000 7f9819ffb640 1 -- 172.21.15.27:0/2645657025 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9814088ff0 con 0x7f98241550a0 2023-08-11T15:52:08.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.580+0000 7f9829165640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f980806a080 0x7f980806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:08.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.581+0000 7f9829165640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f980806a080 0x7f980806c540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9820034000 tx=0x7f9820031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:08.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.584+0000 7f9819ffb640 1 -- 172.21.15.27:0/2645657025 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9814054420 con 0x7f98241550a0 2023-08-11T15:52:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.780+0000 7f982b3c9640 1 -- 172.21.15.27:0/2645657025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f982404e420 con 0x7f98241550a0 2023-08-11T15:52:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.781+0000 7f9819ffb640 1 -- 172.21.15.27:0/2645657025 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9814057a40 con 0x7f98241550a0 2023-08-11T15:52:08.782 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.783+0000 7f982b3c9640 1 -- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f980806a080 msgr2=0x7f980806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.783+0000 7f982b3c9640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f980806a080 0x7f980806c540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9820034000 tx=0x7f9820031000 comp rx=0 tx=0).stop 2023-08-11T15:52:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.783+0000 7f982b3c9640 1 -- 172.21.15.27:0/2645657025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98241550a0 msgr2=0x7f9824155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.783+0000 7f982b3c9640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98241550a0 0x7f9824155520 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f98140108e0 tx=0x7f9814010da0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.783+0000 7f982b3c9640 1 -- 172.21.15.27:0/2645657025 shutdown_connections 2023-08-11T15:52:08.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.783+0000 7f982b3c9640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f980806a080 0x7f980806c540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.783+0000 7f982b3c9640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824155a60 0x7f9824144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.783+0000 7f982b3c9640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98241550a0 0x7f9824155520 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.783+0000 7f982b3c9640 1 --2- 172.21.15.27:0/2645657025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9810003b00 0x7f98240c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:08.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.783+0000 7f982b3c9640 1 -- 172.21.15.27:0/2645657025 >> 172.21.15.27:0/2645657025 conn(0x7f98240afbe0 msgr2=0x7f98240709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.784+0000 7f982b3c9640 1 -- 172.21.15.27:0/2645657025 shutdown_connections 2023-08-11T15:52:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:08.784+0000 7f982b3c9640 1 -- 172.21.15.27:0/2645657025 wait complete. 2023-08-11T15:52:08.805 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.045+0000 7f02cc6ec640 1 Processor -- start 2023-08-11T15:52:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.045+0000 7f02cc6ec640 1 -- start start 2023-08-11T15:52:09.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.046+0000 7f02cc6ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f02c4151220 0x7f02c406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.047+0000 7f02cc6ec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c406c850 con 0x7f02c406bdf0 2023-08-11T15:52:09.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.047+0000 7f02cc6ec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c406c290 con 0x7f02c415e680 2023-08-11T15:52:09.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.047+0000 7f02cc6ec640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c406c570 con 0x7f02c4151220 2023-08-11T15:52:09.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.047+0000 7f02c9c87640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f02c406bdf0 0x7f02c415df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51856/0 (socket says 172.21.15.27:51856) 2023-08-11T15:52:09.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.047+0000 7f02c9c87640 1 -- 172.21.15.27:0/1616117504 learned_addr learned my addr 172.21.15.27:0/1616117504 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:09.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.047+0000 7f02ca488640 1 --2- 172.21.15.27:0/1616117504 >> v2:172.21.15.119:3300/0 conn(0x7f02c4151220 0x7f02c406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533324555 0 0) 0x7f02c406c850 con 0x7f02c406bdf0 2023-08-11T15:52:09.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02b0003520 con 0x7f02c406bdf0 2023-08-11T15:52:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372058540 0 0) 0x7f02b0003520 con 0x7f02c406bdf0 2023-08-11T15:52:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 >> v2:172.21.15.119:3300/0 conn(0x7f02c4151220 msgr2=0x7f02c406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 --2- 172.21.15.27:0/1616117504 >> v2:172.21.15.119:3300/0 conn(0x7f02c4151220 0x7f02c406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 >> v1:172.21.15.119:6789/0 conn(0x7f02c415e680 legacy=0x7f02c40ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c40ca050 con 0x7f02c406bdf0 2023-08-11T15:52:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f02b4001670 con 0x7f02c406bdf0 2023-08-11T15:52:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 >> v1:172.21.15.27:6789/0 conn(0x7f02c406bdf0 legacy=0x7f02c415df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02b00047c0 0x7f02b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b00052a0 0x7f02b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c406c570 con 0x7f02b0003970 2023-08-11T15:52:09.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b000c170 con 0x7f02b00052a0 2023-08-11T15:52:09.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.048+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b000c330 con 0x7f02b00047c0 2023-08-11T15:52:09.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.049+0000 7f02ca488640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.049+0000 7f02ca488640 1 -- 172.21.15.27:0/1616117504 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02b00047c0 msgr2=0x7f02b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.049+0000 7f02ca488640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02b00047c0 0x7f02b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.049+0000 7f02ca488640 1 -- 172.21.15.27:0/1616117504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b00052a0 msgr2=0x7f02b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.049+0000 7f02cac89640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b00052a0 0x7f02b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.049+0000 7f02ca488640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b00052a0 0x7f02b000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.049+0000 7f02ca488640 1 -- 172.21.15.27:0/1616117504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b0003520 con 0x7f02b0003970 2023-08-11T15:52:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.049+0000 7f02cac89640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b00052a0 0x7f02b000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.049+0000 7f02cac89640 1 -- 172.21.15.27:0/1616117504 reap_dead start 2023-08-11T15:52:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.050+0000 7f02ca488640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02b0003d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009f90 tx=0x7f02b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d12459ec0e9e307 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.051+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c002cbd0 con 0x7f02b0003970 2023-08-11T15:52:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.051+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c000f040 con 0x7f02b0003970 2023-08-11T15:52:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.051+0000 7f02c9486640 1 -- 172.21.15.27:0/1616117504 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0034440 con 0x7f02b0003970 2023-08-11T15:52:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.051+0000 7f02cc6ec640 1 -- 172.21.15.27:0/1616117504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 msgr2=0x7f02b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.051+0000 7f02cc6ec640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02b0003d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009f90 tx=0x7f02b00057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.051+0000 7f02cc6ec640 1 -- 172.21.15.27:0/1616117504 shutdown_connections 2023-08-11T15:52:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.051+0000 7f02cc6ec640 1 --2- 172.21.15.27:0/1616117504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02b0003d90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.051+0000 7f02cc6ec640 1 -- 172.21.15.27:0/1616117504 >> 172.21.15.27:0/1616117504 conn(0x7f02c40afbe0 msgr2=0x7f02c40affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.051+0000 7f02cc6ec640 1 -- 172.21.15.27:0/1616117504 shutdown_connections 2023-08-11T15:52:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.051+0000 7f02cc6ec640 1 -- 172.21.15.27:0/1616117504 wait complete. 2023-08-11T15:52:09.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.053+0000 7f02cc6ec640 1 Processor -- start 2023-08-11T15:52:09.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.053+0000 7f02cc6ec640 1 -- start start 2023-08-11T15:52:09.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.054+0000 7f02cc6ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02c41413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02cc6ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02c4159b50 0x7f02c4142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02cc6ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c41448d0 0x7f02c4144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02cc6ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c4141af0 con 0x7f02b0003970 2023-08-11T15:52:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02cc6ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c4141dd0 con 0x7f02c41448d0 2023-08-11T15:52:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02cc6ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41420b0 con 0x7f02c4159b50 2023-08-11T15:52:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02ca488640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02c41413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02ca488640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02c41413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57406/0 (socket says 172.21.15.27:57406) 2023-08-11T15:52:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02ca488640 1 -- 172.21.15.27:0/55223937 learned_addr learned my addr 172.21.15.27:0/55223937 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02c9c87640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02c4159b50 0x7f02c4142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02ca488640 1 -- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02c4159b50 msgr2=0x7f02c4142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02cac89640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c41448d0 0x7f02c4144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02ca488640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02c4159b50 0x7f02c4142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02ca488640 1 -- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c41448d0 msgr2=0x7f02c4144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02ca488640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c41448d0 0x7f02c4144d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02ca488640 1 -- 172.21.15.27:0/55223937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c4142390 con 0x7f02b0003970 2023-08-11T15:52:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.055+0000 7f02c9c87640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02c4159b50 0x7f02c4142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.056+0000 7f02ca488640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02c41413e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02c002c570 tx=0x7f02c0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.056+0000 7f02baffd640 1 -- 172.21.15.27:0/55223937 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0040450 con 0x7f02b0003970 2023-08-11T15:52:09.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.056+0000 7f02baffd640 1 -- 172.21.15.27:0/55223937 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c00405d0 con 0x7f02b0003970 2023-08-11T15:52:09.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.056+0000 7f02baffd640 1 -- 172.21.15.27:0/55223937 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0040980 con 0x7f02b0003970 2023-08-11T15:52:09.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.056+0000 7f02cc6ec640 1 -- 172.21.15.27:0/55223937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c4142670 con 0x7f02b0003970 2023-08-11T15:52:09.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.056+0000 7f02cc6ec640 1 -- 172.21.15.27:0/55223937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c41452c0 con 0x7f02b0003970 2023-08-11T15:52:09.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.057+0000 7f02cc6ec640 1 -- 172.21.15.27:0/55223937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c404e420 con 0x7f02b0003970 2023-08-11T15:52:09.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.058+0000 7f02baffd640 1 -- 172.21.15.27:0/55223937 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f02c0040ae0 con 0x7f02b0003970 2023-08-11T15:52:09.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.059+0000 7f02baffd640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f02a806a050 0x7f02a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.059+0000 7f02baffd640 1 -- 172.21.15.27:0/55223937 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f02c00ab260 con 0x7f02b0003970 2023-08-11T15:52:09.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.060+0000 7f02c9c87640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f02a806a050 0x7f02a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.061+0000 7f02c9c87640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f02a806a050 0x7f02a806c510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f02c4151990 tx=0x7f02b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:09.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.064+0000 7f02baffd640 1 -- 172.21.15.27:0/55223937 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f02c0076580 con 0x7f02b0003970 2023-08-11T15:52:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.260+0000 7f02cc6ec640 1 -- 172.21.15.27:0/55223937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02c406d380 con 0x7f02b0003970 2023-08-11T15:52:09.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.261+0000 7f02baffd640 1 -- 172.21.15.27:0/55223937 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02c0079ba0 con 0x7f02b0003970 2023-08-11T15:52:09.263 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:09.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.263+0000 7f02cc6ec640 1 -- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f02a806a050 msgr2=0x7f02a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.263+0000 7f02cc6ec640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f02a806a050 0x7f02a806c510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f02c4151990 tx=0x7f02b4009040 comp rx=0 tx=0).stop 2023-08-11T15:52:09.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.263+0000 7f02cc6ec640 1 -- 172.21.15.27:0/55223937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 msgr2=0x7f02c41413e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.263+0000 7f02cc6ec640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02c41413e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02c002c570 tx=0x7f02c0009750 comp rx=0 tx=0).stop 2023-08-11T15:52:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.264+0000 7f02cc6ec640 1 -- 172.21.15.27:0/55223937 shutdown_connections 2023-08-11T15:52:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.264+0000 7f02cc6ec640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f02a806a050 0x7f02a806c510 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.264+0000 7f02cc6ec640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02c4159b50 0x7f02c4142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.264+0000 7f02cc6ec640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02c41448d0 0x7f02c4144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.264+0000 7f02cc6ec640 1 --2- 172.21.15.27:0/55223937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b0003970 0x7f02c41413e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.264+0000 7f02cc6ec640 1 -- 172.21.15.27:0/55223937 >> 172.21.15.27:0/55223937 conn(0x7f02c40afbe0 msgr2=0x7f02c40c4920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.264+0000 7f02cc6ec640 1 -- 172.21.15.27:0/55223937 shutdown_connections 2023-08-11T15:52:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.264+0000 7f02cc6ec640 1 -- 172.21.15.27:0/55223937 wait complete. 2023-08-11T15:52:09.286 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:09.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.529+0000 7f69e6998640 1 Processor -- start 2023-08-11T15:52:09.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.529+0000 7f69e6998640 1 -- start start 2023-08-11T15:52:09.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.530+0000 7f69e6998640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b38f0 0x7f69e00d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.530+0000 7f69e6998640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69e00b3090 0x7f69e0160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.530+0000 7f69e6998640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69e00b3cd0 0x7f69e0161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.530+0000 7f69e6998640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e00b0190 con 0x7f69e00b3cd0 2023-08-11T15:52:09.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.530+0000 7f69e6998640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e00af250 con 0x7f69e00b3090 2023-08-11T15:52:09.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.530+0000 7f69e6998640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e00af510 con 0x7f69e00b38f0 2023-08-11T15:52:09.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69e4f35640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69e00b3cd0 0x7f69e0161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69e4f35640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69e00b3cd0 0x7f69e0161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40842/0 (socket says 172.21.15.27:40842) 2023-08-11T15:52:09.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69e4f35640 1 -- 172.21.15.27:0/3705424561 learned_addr learned my addr 172.21.15.27:0/3705424561 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69dffff640 1 --2- 172.21.15.27:0/3705424561 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b38f0 0x7f69e00d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69df7fe640 1 --2- 172.21.15.27:0/3705424561 >> v2:172.21.15.119:3300/0 conn(0x7f69e00b3090 0x7f69e0160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69e4f35640 1 -- 172.21.15.27:0/3705424561 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b38f0 msgr2=0x7f69e00d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69e4f35640 1 --2- 172.21.15.27:0/3705424561 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b38f0 0x7f69e00d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69e4f35640 1 -- 172.21.15.27:0/3705424561 >> v2:172.21.15.119:3300/0 conn(0x7f69e00b3090 msgr2=0x7f69e0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69e4f35640 1 --2- 172.21.15.27:0/3705424561 >> v2:172.21.15.119:3300/0 conn(0x7f69e00b3090 0x7f69e0160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69e4f35640 1 -- 172.21.15.27:0/3705424561 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e00af7f0 con 0x7f69e00b3cd0 2023-08-11T15:52:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.531+0000 7f69df7fe640 1 --2- 172.21.15.27:0/3705424561 >> v2:172.21.15.119:3300/0 conn(0x7f69e00b3090 0x7f69e0160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.532+0000 7f69e4f35640 1 --2- 172.21.15.27:0/3705424561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b3cd0 0x7f69e0161040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f69e008b140 tx=0x7f69d802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51343532b05191fd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.533+0000 7f69deffd640 1 -- 172.21.15.27:0/3705424561 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8031860 con 0x7f69e00b3cd0 2023-08-11T15:52:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.533+0000 7f69deffd640 1 -- 172.21.15.27:0/3705424561 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d80319c0 con 0x7f69e00b3cd0 2023-08-11T15:52:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.533+0000 7f69deffd640 1 -- 172.21.15.27:0/3705424561 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8031d50 con 0x7f69e00b3cd0 2023-08-11T15:52:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.533+0000 7f69e6998640 1 -- 172.21.15.27:0/3705424561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b3cd0 msgr2=0x7f69e0161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.533+0000 7f69e6998640 1 --2- 172.21.15.27:0/3705424561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b3cd0 0x7f69e0161040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f69e008b140 tx=0x7f69d802b900 comp rx=0 tx=0).stop 2023-08-11T15:52:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.534+0000 7f69e6998640 1 -- 172.21.15.27:0/3705424561 shutdown_connections 2023-08-11T15:52:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.534+0000 7f69e6998640 1 --2- 172.21.15.27:0/3705424561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b3cd0 0x7f69e0161040 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.534+0000 7f69e6998640 1 --2- 172.21.15.27:0/3705424561 >> v2:172.21.15.119:3300/0 conn(0x7f69e00b3090 0x7f69e0160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.534+0000 7f69e6998640 1 --2- 172.21.15.27:0/3705424561 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b38f0 0x7f69e00d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.534+0000 7f69e6998640 1 -- 172.21.15.27:0/3705424561 >> 172.21.15.27:0/3705424561 conn(0x7f69e009b960 msgr2=0x7f69e00d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.534+0000 7f69e6998640 1 -- 172.21.15.27:0/3705424561 shutdown_connections 2023-08-11T15:52:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.534+0000 7f69e6998640 1 -- 172.21.15.27:0/3705424561 wait complete. 2023-08-11T15:52:09.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.535+0000 7f69e6998640 1 Processor -- start 2023-08-11T15:52:09.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.536+0000 7f69e6998640 1 -- start start 2023-08-11T15:52:09.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69e6998640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e00b3090 0x7f69e01f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69e6998640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b38f0 0x7f69e01f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69e6998640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b3cd0 0x7f69e01f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69e6998640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e014ba20 con 0x7f69e00b38f0 2023-08-11T15:52:09.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69e6998640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e01587b0 con 0x7f69e00b3090 2023-08-11T15:52:09.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69e6998640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e0158a90 con 0x7f69e00b3cd0 2023-08-11T15:52:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69df7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b38f0 0x7f69e01f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69df7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b38f0 0x7f69e01f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40856/0 (socket says 172.21.15.27:40856) 2023-08-11T15:52:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69df7fe640 1 -- 172.21.15.27:0/439951110 learned_addr learned my addr 172.21.15.27:0/439951110 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69dffff640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e00b3090 0x7f69e01f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69e4f35640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b3cd0 0x7f69e01f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69df7fe640 1 -- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b3cd0 msgr2=0x7f69e01f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69df7fe640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b3cd0 0x7f69e01f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69df7fe640 1 -- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e00b3090 msgr2=0x7f69e01f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69df7fe640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e00b3090 0x7f69e01f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69df7fe640 1 -- 172.21.15.27:0/439951110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80094d0 con 0x7f69e00b38f0 2023-08-11T15:52:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69e4f35640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b3cd0 0x7f69e01f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.537+0000 7f69dffff640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e00b3090 0x7f69e01f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.538+0000 7f69df7fe640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b38f0 0x7f69e01f5a80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f69c800e7e0 tx=0x7f69c800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.538+0000 7f69dcff9640 1 -- 172.21.15.27:0/439951110 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8004030 con 0x7f69e00b38f0 2023-08-11T15:52:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.538+0000 7f69dcff9640 1 -- 172.21.15.27:0/439951110 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c8004190 con 0x7f69e00b38f0 2023-08-11T15:52:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.538+0000 7f69dcff9640 1 -- 172.21.15.27:0/439951110 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8004500 con 0x7f69e00b38f0 2023-08-11T15:52:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.538+0000 7f69e6998640 1 -- 172.21.15.27:0/439951110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e00cacc0 con 0x7f69e00b38f0 2023-08-11T15:52:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.538+0000 7f69e6998640 1 -- 172.21.15.27:0/439951110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e00cb280 con 0x7f69e00b38f0 2023-08-11T15:52:09.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.540+0000 7f69e6998640 1 -- 172.21.15.27:0/439951110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e00cab20 con 0x7f69e00b38f0 2023-08-11T15:52:09.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.540+0000 7f69dcff9640 1 -- 172.21.15.27:0/439951110 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f69c800c6d0 con 0x7f69e00b38f0 2023-08-11T15:52:09.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.541+0000 7f69dcff9640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f69c006a050 0x7f69c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:09.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.542+0000 7f69dcff9640 1 -- 172.21.15.27:0/439951110 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f69c8088470 con 0x7f69e00b38f0 2023-08-11T15:52:09.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.542+0000 7f69dffff640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f69c006a050 0x7f69c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:09.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.543+0000 7f69dffff640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f69c006a050 0x7f69c006c510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009fd0 tx=0x7f69d000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:09.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.546+0000 7f69dcff9640 1 -- 172.21.15.27:0/439951110 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f69e00cab20 con 0x7f69e00b38f0 2023-08-11T15:52:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.739+0000 7f69e6998640 1 -- 172.21.15.27:0/439951110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f69e00cab20 con 0x7f69e00b38f0 2023-08-11T15:52:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.740+0000 7f69dcff9640 1 -- 172.21.15.27:0/439951110 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f69e00cab20 con 0x7f69e00b38f0 2023-08-11T15:52:09.742 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:09.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.742+0000 7f69e6998640 1 -- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f69c006a050 msgr2=0x7f69c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.742+0000 7f69e6998640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f69c006a050 0x7f69c006c510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009fd0 tx=0x7f69d000b040 comp rx=0 tx=0).stop 2023-08-11T15:52:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.742+0000 7f69e6998640 1 -- 172.21.15.27:0/439951110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b38f0 msgr2=0x7f69e01f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.742+0000 7f69e6998640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b38f0 0x7f69e01f5a80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f69c800e7e0 tx=0x7f69c800eca0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.742+0000 7f69e6998640 1 -- 172.21.15.27:0/439951110 shutdown_connections 2023-08-11T15:52:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.742+0000 7f69e6998640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f69c006a050 0x7f69c006c510 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.743+0000 7f69e6998640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69e00b3cd0 0x7f69e01f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.743+0000 7f69e6998640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00b38f0 0x7f69e01f5a80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.743+0000 7f69e6998640 1 --2- 172.21.15.27:0/439951110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e00b3090 0x7f69e01f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.743+0000 7f69e6998640 1 -- 172.21.15.27:0/439951110 >> 172.21.15.27:0/439951110 conn(0x7f69e009b960 msgr2=0x7f69e015f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.743+0000 7f69e6998640 1 -- 172.21.15.27:0/439951110 shutdown_connections 2023-08-11T15:52:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:09.743+0000 7f69e6998640 1 -- 172.21.15.27:0/439951110 wait complete. 2023-08-11T15:52:09.790 INFO:tasks.cephfs_test_runner:Starting test: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-08-11T15:52:09.791 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-08-11T15:52:10.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.034+0000 7ff100f86640 1 Processor -- start 2023-08-11T15:52:10.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.034+0000 7ff100f86640 1 -- start start 2023-08-11T15:52:10.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.035+0000 7ff100f86640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc15a3b0 0x7ff0fc071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.035+0000 7ff100f86640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0fc0cb520 con 0x7ff0fc06f6e0 2023-08-11T15:52:10.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.035+0000 7ff100f86640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0fc15a790 con 0x7ff0fc15df00 2023-08-11T15:52:10.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.035+0000 7ff100f86640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc15eea0 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.036+0000 7ff0fad9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff0fc06f6e0 0x7ff0fc0c8f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41978/0 (socket says 172.21.15.27:41978) 2023-08-11T15:52:10.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.036+0000 7ff0fad9d640 1 -- 172.21.15.27:0/318864912 learned_addr learned my addr 172.21.15.27:0/318864912 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:10.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.036+0000 7ff0fa59c640 1 --2- 172.21.15.27:0/318864912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc15a3b0 0x7ff0fc071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:10.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.036+0000 7ff0f9d9b640 1 -- 172.21.15.27:0/318864912 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227920476 0 0) 0x7ff0fc0cb520 con 0x7ff0fc06f6e0 2023-08-11T15:52:10.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.037+0000 7ff0f9d9b640 1 -- 172.21.15.27:0/318864912 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e8003520 con 0x7ff0fc06f6e0 2023-08-11T15:52:10.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.037+0000 7ff0f9d9b640 1 -- 172.21.15.27:0/318864912 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832176420 0 0) 0x7ff0fc15a790 con 0x7ff0fc15df00 2023-08-11T15:52:10.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.037+0000 7ff0fa59c640 1 -- 172.21.15.27:0/318864912 >> v1:172.21.15.119:6789/0 conn(0x7ff0fc15df00 legacy=0x7ff0fc0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:10.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.037+0000 7ff0fa59c640 1 -- 172.21.15.27:0/318864912 >> v1:172.21.15.27:6789/0 conn(0x7ff0fc06f6e0 legacy=0x7ff0fc0c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:10.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.037+0000 7ff0fa59c640 1 -- 172.21.15.27:0/318864912 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc15e360 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.038+0000 7ff0fa59c640 1 --2- 172.21.15.27:0/318864912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc15a3b0 0x7ff0fc071690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0097d0 tx=0x7ff0ec02b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79881bf1c7a6aedc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:10.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.039+0000 7ff0f9d9b640 1 -- 172.21.15.27:0/318864912 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40019f0 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.039+0000 7ff0f9d9b640 1 -- 172.21.15.27:0/318864912 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec02e050 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.039+0000 7ff0f9d9b640 1 -- 172.21.15.27:0/318864912 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec030730 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.039+0000 7ff100f86640 1 -- 172.21.15.27:0/318864912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc15a3b0 msgr2=0x7ff0fc071690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:10.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.040+0000 7ff100f86640 1 --2- 172.21.15.27:0/318864912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc15a3b0 0x7ff0fc071690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0097d0 tx=0x7ff0ec02b730 comp rx=0 tx=0).stop 2023-08-11T15:52:10.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.040+0000 7ff100f86640 1 -- 172.21.15.27:0/318864912 shutdown_connections 2023-08-11T15:52:10.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.040+0000 7ff100f86640 1 --2- 172.21.15.27:0/318864912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc15a3b0 0x7ff0fc071690 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.040+0000 7ff100f86640 1 -- 172.21.15.27:0/318864912 >> 172.21.15.27:0/318864912 conn(0x7ff0fc069e90 msgr2=0x7ff0fc0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:10.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.040+0000 7ff100f86640 1 -- 172.21.15.27:0/318864912 shutdown_connections 2023-08-11T15:52:10.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.040+0000 7ff100f86640 1 -- 172.21.15.27:0/318864912 wait complete. 2023-08-11T15:52:10.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.042+0000 7ff100f86640 1 Processor -- start 2023-08-11T15:52:10.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.042+0000 7ff100f86640 1 -- start start 2023-08-11T15:52:10.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.043+0000 7ff100f86640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc06f6e0 0x7ff0fc15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.043+0000 7ff100f86640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc15a3b0 0x7ff0fc1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff100f86640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0fc15df00 0x7ff0fc15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff100f86640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc15b440 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff100f86640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc15b720 con 0x7ff0fc15df00 2023-08-11T15:52:10.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff100f86640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc15ba00 con 0x7ff0fc06f6e0 2023-08-11T15:52:10.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff0f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc15a3b0 0x7ff0fc1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:10.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff0f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc15a3b0 0x7ff0fc1597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40870/0 (socket says 172.21.15.27:40870) 2023-08-11T15:52:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff0f3fff640 1 -- 172.21.15.27:0/2313507809 learned_addr learned my addr 172.21.15.27:0/2313507809 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff0fa59c640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc06f6e0 0x7ff0fc15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff0fad9d640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0fc15df00 0x7ff0fc15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:10.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff0f3fff640 1 -- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc06f6e0 msgr2=0x7ff0fc15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:10.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff0f3fff640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc06f6e0 0x7ff0fc15c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff0f3fff640 1 -- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0fc15df00 msgr2=0x7ff0fc15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff0f3fff640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0fc15df00 0x7ff0fc15a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.044+0000 7ff0f3fff640 1 -- 172.21.15.27:0/2313507809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec009400 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.045+0000 7ff0f3fff640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc15a3b0 0x7ff0fc1597d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400c410 tx=0x7ff0e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.045+0000 7ff0f2ffd640 1 -- 172.21.15.27:0/2313507809 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e40035f0 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.045+0000 7ff0f2ffd640 1 -- 172.21.15.27:0/2313507809 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e4003770 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.045+0000 7ff0f2ffd640 1 -- 172.21.15.27:0/2313507809 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4003b20 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.045+0000 7ff100f86640 1 -- 172.21.15.27:0/2313507809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc15bce0 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.045+0000 7ff100f86640 1 -- 172.21.15.27:0/2313507809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc0c35c0 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.046+0000 7ff100f86640 1 -- 172.21.15.27:0/2313507809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc070cb0 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.047+0000 7ff0f2ffd640 1 -- 172.21.15.27:0/2313507809 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff0e4003c80 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.048+0000 7ff0f2ffd640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff0dc06a080 0x7ff0dc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.049+0000 7ff0f2ffd640 1 -- 172.21.15.27:0/2313507809 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff0e4087fd0 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.049+0000 7ff0fa59c640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff0dc06a080 0x7ff0dc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.050+0000 7ff0fa59c640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff0dc06a080 0x7ff0dc06c540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec034000 tx=0x7ff0ec008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.053+0000 7ff0f2ffd640 1 -- 172.21.15.27:0/2313507809 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ff0e4053300 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.316+0000 7ff100f86640 1 -- 172.21.15.27:0/2313507809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7ff0fc073990 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.708+0000 7ff0f2ffd640 1 -- 172.21.15.27:0/2313507809 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v88) v1 ==== 163+0+0 (secure 0 0 0) 0x7ff0e4017020 con 0x7ff0fc15a3b0 2023-08-11T15:52:10.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.711+0000 7ff100f86640 1 -- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff0dc06a080 msgr2=0x7ff0dc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:10.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.711+0000 7ff100f86640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff0dc06a080 0x7ff0dc06c540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec034000 tx=0x7ff0ec008030 comp rx=0 tx=0).stop 2023-08-11T15:52:10.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.711+0000 7ff100f86640 1 -- 172.21.15.27:0/2313507809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc15a3b0 msgr2=0x7ff0fc1597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:10.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.711+0000 7ff100f86640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc15a3b0 0x7ff0fc1597d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400c410 tx=0x7ff0e400c8d0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.711+0000 7ff100f86640 1 -- 172.21.15.27:0/2313507809 shutdown_connections 2023-08-11T15:52:10.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.711+0000 7ff100f86640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff0dc06a080 0x7ff0dc06c540 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.711+0000 7ff100f86640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0fc15df00 0x7ff0fc15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.711+0000 7ff100f86640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc15a3b0 0x7ff0fc1597d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.711+0000 7ff100f86640 1 --2- 172.21.15.27:0/2313507809 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0fc06f6e0 0x7ff0fc15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.711+0000 7ff100f86640 1 -- 172.21.15.27:0/2313507809 >> 172.21.15.27:0/2313507809 conn(0x7ff0fc069e90 msgr2=0x7ff0fc0acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:10.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.712+0000 7ff100f86640 1 -- 172.21.15.27:0/2313507809 shutdown_connections 2023-08-11T15:52:10.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.712+0000 7ff100f86640 1 -- 172.21.15.27:0/2313507809 wait complete. 2023-08-11T15:52:10.736 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:10.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.967+0000 7f368d5b5640 1 Processor -- start 2023-08-11T15:52:10.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.967+0000 7f368d5b5640 1 -- start start 2023-08-11T15:52:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.969+0000 7f368d5b5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f368815a140 0x7f368806b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.969+0000 7f368d5b5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f36880cb430 0x7f36880b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.969+0000 7f368d5b5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36880c8a30 con 0x7f368806f5d0 2023-08-11T15:52:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.969+0000 7f368d5b5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368806c0c0 con 0x7f36880cb430 2023-08-11T15:52:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.969+0000 7f368d5b5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368806c3a0 con 0x7f368815a140 2023-08-11T15:52:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.969+0000 7f367ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f36880cb430 0x7f36880b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.969+0000 7f367ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f36880cb430 0x7f36880b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40882/0 (socket says 172.21.15.27:40882) 2023-08-11T15:52:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.969+0000 7f367ffff640 1 -- 172.21.15.27:0/2269016365 learned_addr learned my addr 172.21.15.27:0/2269016365 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.969+0000 7f3686ffd640 1 --2- 172.21.15.27:0/2269016365 >> v2:172.21.15.119:3300/0 conn(0x7f368815a140 0x7f368806b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.970+0000 7f367ffff640 1 -- 172.21.15.27:0/2269016365 >> v2:172.21.15.119:3300/0 conn(0x7f368815a140 msgr2=0x7f368806b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.970+0000 7f367ffff640 1 --2- 172.21.15.27:0/2269016365 >> v2:172.21.15.119:3300/0 conn(0x7f368815a140 0x7f368806b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.970+0000 7f367ffff640 1 -- 172.21.15.27:0/2269016365 >> v1:172.21.15.119:6789/0 conn(0x7f368806f5d0 legacy=0x7f36880c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.970+0000 7f367ffff640 1 -- 172.21.15.27:0/2269016365 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368806c680 con 0x7f36880cb430 2023-08-11T15:52:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.971+0000 7f367ffff640 1 --2- 172.21.15.27:0/2269016365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36880cb430 0x7f36880b0e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f368806dae0 tx=0x7f367002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d35f2c7acd5fc6f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.971+0000 7f36867fc640 1 -- 172.21.15.27:0/2269016365 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670031870 con 0x7f36880cb430 2023-08-11T15:52:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.971+0000 7f36867fc640 1 -- 172.21.15.27:0/2269016365 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36700319d0 con 0x7f36880cb430 2023-08-11T15:52:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.971+0000 7f36867fc640 1 -- 172.21.15.27:0/2269016365 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670031d60 con 0x7f36880cb430 2023-08-11T15:52:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.972+0000 7f368d5b5640 1 -- 172.21.15.27:0/2269016365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36880cb430 msgr2=0x7f36880b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.972+0000 7f368d5b5640 1 --2- 172.21.15.27:0/2269016365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36880cb430 0x7f36880b0e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f368806dae0 tx=0x7f367002b910 comp rx=0 tx=0).stop 2023-08-11T15:52:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.972+0000 7f368d5b5640 1 -- 172.21.15.27:0/2269016365 shutdown_connections 2023-08-11T15:52:10.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.972+0000 7f368d5b5640 1 --2- 172.21.15.27:0/2269016365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36880cb430 0x7f36880b0e70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.972+0000 7f368d5b5640 1 --2- 172.21.15.27:0/2269016365 >> v2:172.21.15.119:3300/0 conn(0x7f368815a140 0x7f368806b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.972+0000 7f368d5b5640 1 -- 172.21.15.27:0/2269016365 >> 172.21.15.27:0/2269016365 conn(0x7f36880afb70 msgr2=0x7f36880aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:10.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.972+0000 7f368d5b5640 1 -- 172.21.15.27:0/2269016365 shutdown_connections 2023-08-11T15:52:10.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.972+0000 7f368d5b5640 1 -- 172.21.15.27:0/2269016365 wait complete. 2023-08-11T15:52:10.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.974+0000 7f368d5b5640 1 Processor -- start 2023-08-11T15:52:10.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.974+0000 7f368d5b5640 1 -- start start 2023-08-11T15:52:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.975+0000 7f368d5b5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368806f5d0 0x7f3688158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.975+0000 7f368d5b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36880cb430 0x7f36880c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.975+0000 7f368d5b5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f368815a140 0x7f368815c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.975+0000 7f368d5b5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881f52c0 con 0x7f368806f5d0 2023-08-11T15:52:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.975+0000 7f368d5b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881f55a0 con 0x7f36880cb430 2023-08-11T15:52:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.975+0000 7f368d5b5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36881f5880 con 0x7f368815a140 2023-08-11T15:52:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.975+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368806f5d0 0x7f3688158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:10.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.975+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368806f5d0 0x7f3688158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40894/0 (socket says 172.21.15.27:40894) 2023-08-11T15:52:10.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.975+0000 7f3686ffd640 1 -- 172.21.15.27:0/1066838822 learned_addr learned my addr 172.21.15.27:0/1066838822 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.976+0000 7f3686ffd640 1 -- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f368815a140 msgr2=0x7f368815c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.976+0000 7f3686ffd640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f368815a140 0x7f368815c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.976+0000 7f3686ffd640 1 -- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36880cb430 msgr2=0x7f36880c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.976+0000 7f3686ffd640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36880cb430 0x7f36880c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.976+0000 7f3686ffd640 1 -- 172.21.15.27:0/1066838822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36700094d0 con 0x7f368806f5d0 2023-08-11T15:52:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.976+0000 7f3686ffd640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368806f5d0 0x7f3688158450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f367800d7d0 tx=0x7f367800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.976+0000 7f367f7fe640 1 -- 172.21.15.27:0/1066838822 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36780096c0 con 0x7f368806f5d0 2023-08-11T15:52:10.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.977+0000 7f367f7fe640 1 -- 172.21.15.27:0/1066838822 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3678009820 con 0x7f368806f5d0 2023-08-11T15:52:10.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.977+0000 7f368d5b5640 1 -- 172.21.15.27:0/1066838822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36881f5b60 con 0x7f368806f5d0 2023-08-11T15:52:10.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.977+0000 7f367f7fe640 1 -- 172.21.15.27:0/1066838822 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678009bd0 con 0x7f368806f5d0 2023-08-11T15:52:10.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.977+0000 7f368d5b5640 1 -- 172.21.15.27:0/1066838822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36881f29f0 con 0x7f368806f5d0 2023-08-11T15:52:10.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.978+0000 7f368d5b5640 1 -- 172.21.15.27:0/1066838822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368806bd90 con 0x7f368806f5d0 2023-08-11T15:52:10.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.979+0000 7f367f7fe640 1 -- 172.21.15.27:0/1066838822 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3678009d30 con 0x7f368806f5d0 2023-08-11T15:52:10.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.980+0000 7f367f7fe640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f366806a050 0x7f366806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:10.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.980+0000 7f367f7fe640 1 -- 172.21.15.27:0/1066838822 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3678089190 con 0x7f368806f5d0 2023-08-11T15:52:10.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.981+0000 7f367ffff640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f366806a050 0x7f366806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:10.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.982+0000 7f367ffff640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f366806a050 0x7f366806c510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3670036000 tx=0x7f3670030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:10.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:10.985+0000 7f367f7fe640 1 -- 172.21.15.27:0/1066838822 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f368806bd90 con 0x7f368806f5d0 2023-08-11T15:52:11.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.181+0000 7f368d5b5640 1 -- 172.21.15.27:0/1066838822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f368806bd90 con 0x7f368806f5d0 2023-08-11T15:52:11.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.183+0000 7f367f7fe640 1 -- 172.21.15.27:0/1066838822 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f368806bd90 con 0x7f368806f5d0 2023-08-11T15:52:11.185 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:11.185 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:11.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.185+0000 7f368d5b5640 1 -- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f366806a050 msgr2=0x7f366806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.185+0000 7f368d5b5640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f366806a050 0x7f366806c510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3670036000 tx=0x7f3670030000 comp rx=0 tx=0).stop 2023-08-11T15:52:11.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.185+0000 7f368d5b5640 1 -- 172.21.15.27:0/1066838822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368806f5d0 msgr2=0x7f3688158450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.185+0000 7f368d5b5640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368806f5d0 0x7f3688158450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f367800d7d0 tx=0x7f367800dc90 comp rx=0 tx=0).stop 2023-08-11T15:52:11.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.186+0000 7f368d5b5640 1 -- 172.21.15.27:0/1066838822 shutdown_connections 2023-08-11T15:52:11.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.186+0000 7f368d5b5640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f366806a050 0x7f366806c510 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.186+0000 7f368d5b5640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f368815a140 0x7f368815c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.186+0000 7f368d5b5640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36880cb430 0x7f36880c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.186+0000 7f368d5b5640 1 --2- 172.21.15.27:0/1066838822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368806f5d0 0x7f3688158450 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.186+0000 7f368d5b5640 1 -- 172.21.15.27:0/1066838822 >> 172.21.15.27:0/1066838822 conn(0x7f36880afb70 msgr2=0x7f36881591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:11.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.186+0000 7f368d5b5640 1 -- 172.21.15.27:0/1066838822 shutdown_connections 2023-08-11T15:52:11.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.186+0000 7f368d5b5640 1 -- 172.21.15.27:0/1066838822 wait complete. 2023-08-11T15:52:11.208 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:11.209 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.438+0000 7fd869302640 1 Processor -- start 2023-08-11T15:52:11.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.438+0000 7fd869302640 1 -- start start 2023-08-11T15:52:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.440+0000 7fd869302640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640d1b90 0x7fd8640b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.440+0000 7fd869302640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd8640cd0e0 0x7fd8640cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.440+0000 7fd869302640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8640cda90 con 0x7fd86415c590 2023-08-11T15:52:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.440+0000 7fd869302640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8640d0ed0 con 0x7fd8640cd0e0 2023-08-11T15:52:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.440+0000 7fd869302640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86415c970 con 0x7fd8640d1b90 2023-08-11T15:52:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.440+0000 7fd868b01640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd8640cd0e0 0x7fd8640cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.440+0000 7fd868b01640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd8640cd0e0 0x7fd8640cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40898/0 (socket says 172.21.15.27:40898) 2023-08-11T15:52:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.440+0000 7fd868b01640 1 -- 172.21.15.27:0/1194889663 learned_addr learned my addr 172.21.15.27:0/1194889663 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.440+0000 7fd8637fe640 1 --2- 172.21.15.27:0/1194889663 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640d1b90 0x7fd8640b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.441+0000 7fd868b01640 1 -- 172.21.15.27:0/1194889663 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640d1b90 msgr2=0x7fd8640b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.441+0000 7fd868b01640 1 --2- 172.21.15.27:0/1194889663 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640d1b90 0x7fd8640b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.441+0000 7fd868b01640 1 -- 172.21.15.27:0/1194889663 >> v1:172.21.15.119:6789/0 conn(0x7fd86415c590 legacy=0x7fd8640d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:11.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.441+0000 7fd868b01640 1 -- 172.21.15.27:0/1194889663 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8640d0250 con 0x7fd8640cd0e0 2023-08-11T15:52:11.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.441+0000 7fd8637fe640 1 --2- 172.21.15.27:0/1194889663 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640d1b90 0x7fd8640b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:11.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.442+0000 7fd868b01640 1 --2- 172.21.15.27:0/1194889663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640cd0e0 0x7fd8640cd550 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009870 tx=0x7fd85c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101db282daeef50d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:11.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.442+0000 7fd862ffd640 1 -- 172.21.15.27:0/1194889663 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c031740 con 0x7fd8640cd0e0 2023-08-11T15:52:11.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.442+0000 7fd862ffd640 1 -- 172.21.15.27:0/1194889663 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c0318a0 con 0x7fd8640cd0e0 2023-08-11T15:52:11.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.442+0000 7fd862ffd640 1 -- 172.21.15.27:0/1194889663 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c031c30 con 0x7fd8640cd0e0 2023-08-11T15:52:11.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.443+0000 7fd869302640 1 -- 172.21.15.27:0/1194889663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640cd0e0 msgr2=0x7fd8640cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.443+0000 7fd869302640 1 --2- 172.21.15.27:0/1194889663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640cd0e0 0x7fd8640cd550 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009870 tx=0x7fd85c02b7e0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.443+0000 7fd869302640 1 -- 172.21.15.27:0/1194889663 shutdown_connections 2023-08-11T15:52:11.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.443+0000 7fd869302640 1 --2- 172.21.15.27:0/1194889663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640cd0e0 0x7fd8640cd550 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.443+0000 7fd869302640 1 --2- 172.21.15.27:0/1194889663 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640d1b90 0x7fd8640b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.443+0000 7fd869302640 1 -- 172.21.15.27:0/1194889663 >> 172.21.15.27:0/1194889663 conn(0x7fd86406a250 msgr2=0x7fd86406a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:11.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.443+0000 7fd869302640 1 -- 172.21.15.27:0/1194889663 shutdown_connections 2023-08-11T15:52:11.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.443+0000 7fd869302640 1 -- 172.21.15.27:0/1194889663 wait complete. 2023-08-11T15:52:11.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.445+0000 7fd869302640 1 Processor -- start 2023-08-11T15:52:11.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.445+0000 7fd869302640 1 -- start start 2023-08-11T15:52:11.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.446+0000 7fd869302640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640cd0e0 0x7fd8640b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.446+0000 7fd869302640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640d1b90 0x7fd8640c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.446+0000 7fd869302640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd86415c590 0x7fd8640c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.446+0000 7fd869302640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8641f2950 con 0x7fd8640d1b90 2023-08-11T15:52:11.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.446+0000 7fd869302640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640b49e0 con 0x7fd86415c590 2023-08-11T15:52:11.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.446+0000 7fd869302640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640b3dd0 con 0x7fd8640cd0e0 2023-08-11T15:52:11.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd8637fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640d1b90 0x7fd8640c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd8637fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640d1b90 0x7fd8640c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40900/0 (socket says 172.21.15.27:40900) 2023-08-11T15:52:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd8637fe640 1 -- 172.21.15.27:0/3289544889 learned_addr learned my addr 172.21.15.27:0/3289544889 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd863fff640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640cd0e0 0x7fd8640b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd868b01640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd86415c590 0x7fd8640c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd8637fe640 1 -- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640cd0e0 msgr2=0x7fd8640b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd8637fe640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640cd0e0 0x7fd8640b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd8637fe640 1 -- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd86415c590 msgr2=0x7fd8640c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd8637fe640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd86415c590 0x7fd8640c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd8637fe640 1 -- 172.21.15.27:0/3289544889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0094d0 con 0x7fd8640d1b90 2023-08-11T15:52:11.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd863fff640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640cd0e0 0x7fd8640b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:11.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.447+0000 7fd8637fe640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640d1b90 0x7fd8640c6350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c7f0 tx=0x7fd84c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.448+0000 7fd860ff9640 1 -- 172.21.15.27:0/3289544889 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c007a30 con 0x7fd8640d1b90 2023-08-11T15:52:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.448+0000 7fd860ff9640 1 -- 172.21.15.27:0/3289544889 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c007b90 con 0x7fd8640d1b90 2023-08-11T15:52:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.448+0000 7fd869302640 1 -- 172.21.15.27:0/3289544889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8640b40b0 con 0x7fd8640d1b90 2023-08-11T15:52:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.448+0000 7fd860ff9640 1 -- 172.21.15.27:0/3289544889 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00ce40 con 0x7fd8640d1b90 2023-08-11T15:52:11.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.448+0000 7fd869302640 1 -- 172.21.15.27:0/3289544889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8640b4670 con 0x7fd8640d1b90 2023-08-11T15:52:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.449+0000 7fd869302640 1 -- 172.21.15.27:0/3289544889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8640670b0 con 0x7fd8640d1b90 2023-08-11T15:52:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.450+0000 7fd860ff9640 1 -- 172.21.15.27:0/3289544889 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd84c0113f0 con 0x7fd8640d1b90 2023-08-11T15:52:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.451+0000 7fd860ff9640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd84406a080 0x7fd84406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.451+0000 7fd860ff9640 1 -- 172.21.15.27:0/3289544889 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd84c088a20 con 0x7fd8640d1b90 2023-08-11T15:52:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.452+0000 7fd863fff640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd84406a080 0x7fd84406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.453+0000 7fd863fff640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd84406a080 0x7fd84406c540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd8640cb140 tx=0x7fd8540073c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.455+0000 7fd860ff9640 1 -- 172.21.15.27:0/3289544889 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fd84c053de0 con 0x7fd8640d1b90 2023-08-11T15:52:11.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.650+0000 7fd869302640 1 -- 172.21.15.27:0/3289544889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd864066e80 con 0x7fd8640d1b90 2023-08-11T15:52:11.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.651+0000 7fd860ff9640 1 -- 172.21.15.27:0/3289544889 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd84c057400 con 0x7fd8640d1b90 2023-08-11T15:52:11.652 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:11.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.653+0000 7fd869302640 1 -- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd84406a080 msgr2=0x7fd84406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.653+0000 7fd869302640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd84406a080 0x7fd84406c540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd8640cb140 tx=0x7fd8540073c0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.653+0000 7fd869302640 1 -- 172.21.15.27:0/3289544889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640d1b90 msgr2=0x7fd8640c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.653+0000 7fd869302640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640d1b90 0x7fd8640c6350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c7f0 tx=0x7fd84c00cbf0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.653+0000 7fd869302640 1 -- 172.21.15.27:0/3289544889 shutdown_connections 2023-08-11T15:52:11.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.653+0000 7fd869302640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd84406a080 0x7fd84406c540 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.653+0000 7fd869302640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd86415c590 0x7fd8640c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.653+0000 7fd869302640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8640d1b90 0x7fd8640c6350 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.653+0000 7fd869302640 1 --2- 172.21.15.27:0/3289544889 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8640cd0e0 0x7fd8640b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.653+0000 7fd869302640 1 -- 172.21.15.27:0/3289544889 >> 172.21.15.27:0/3289544889 conn(0x7fd86406a250 msgr2=0x7fd864159840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:11.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.654+0000 7fd869302640 1 -- 172.21.15.27:0/3289544889 shutdown_connections 2023-08-11T15:52:11.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.654+0000 7fd869302640 1 -- 172.21.15.27:0/3289544889 wait complete. 2023-08-11T15:52:11.673 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:11.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.916+0000 7f9fa4426640 1 Processor -- start 2023-08-11T15:52:11.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.916+0000 7f9fa4426640 1 -- start start 2023-08-11T15:52:11.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.918+0000 7f9fa4426640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9f9c15dab0 0x7f9f9c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.918+0000 7f9fa4426640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 0x7f9f9c0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.918+0000 7f9fa4426640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c0c3e10 con 0x7f9f9c0ccaf0 2023-08-11T15:52:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.918+0000 7f9fa4426640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c0cb840 con 0x7f9f9c15dab0 2023-08-11T15:52:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.918+0000 7f9fa4426640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0cbb20 con 0x7f9f9c0ce330 2023-08-11T15:52:11.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.919+0000 7f9fa21c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9f9c15dab0 0x7f9f9c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.919+0000 7f9fa21c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9f9c15dab0 0x7f9f9c0cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47082/0 (socket says 172.21.15.27:47082) 2023-08-11T15:52:11.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.919+0000 7f9fa21c2640 1 -- 172.21.15.27:0/1001939507 learned_addr learned my addr 172.21.15.27:0/1001939507 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:11.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.919+0000 7f9fa19c1640 1 --2- 172.21.15.27:0/1001939507 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 0x7f9f9c0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.919+0000 7f9fa19c1640 1 -- 172.21.15.27:0/1001939507 >> v2:172.21.15.119:3300/0 conn(0x7f9f9c15dab0 msgr2=0x7f9f9c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:11.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.919+0000 7f9fa19c1640 1 --2- 172.21.15.27:0/1001939507 >> v2:172.21.15.119:3300/0 conn(0x7f9f9c15dab0 0x7f9f9c0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.919+0000 7f9fa19c1640 1 -- 172.21.15.27:0/1001939507 >> v1:172.21.15.119:6789/0 conn(0x7f9f9c0ccaf0 legacy=0x7f9f9c0ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.919+0000 7f9fa19c1640 1 -- 172.21.15.27:0/1001939507 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c0cbe00 con 0x7f9f9c0ce330 2023-08-11T15:52:11.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.921+0000 7f9fa19c1640 1 --2- 172.21.15.27:0/1001939507 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 0x7f9f9c0ce790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0097d0 tx=0x7f9f8c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed0df4502a11546b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:11.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.921+0000 7f9fa11c0640 1 -- 172.21.15.27:0/1001939507 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c02d040 con 0x7f9f9c0ce330 2023-08-11T15:52:11.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.921+0000 7f9fa11c0640 1 -- 172.21.15.27:0/1001939507 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c031810 con 0x7f9f9c0ce330 2023-08-11T15:52:11.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.921+0000 7f9fa11c0640 1 -- 172.21.15.27:0/1001939507 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c031ba0 con 0x7f9f9c0ce330 2023-08-11T15:52:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.922+0000 7f9fa4426640 1 -- 172.21.15.27:0/1001939507 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 msgr2=0x7f9f9c0ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.922+0000 7f9fa4426640 1 --2- 172.21.15.27:0/1001939507 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 0x7f9f9c0ce790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0097d0 tx=0x7f9f8c02b8b0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.922+0000 7f9fa4426640 1 -- 172.21.15.27:0/1001939507 shutdown_connections 2023-08-11T15:52:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.922+0000 7f9fa4426640 1 --2- 172.21.15.27:0/1001939507 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 0x7f9f9c0ce790 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.922+0000 7f9fa4426640 1 --2- 172.21.15.27:0/1001939507 >> v2:172.21.15.119:3300/0 conn(0x7f9f9c15dab0 0x7f9f9c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.922+0000 7f9fa4426640 1 -- 172.21.15.27:0/1001939507 >> 172.21.15.27:0/1001939507 conn(0x7f9f9c0d0220 msgr2=0x7f9f9c0d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.922+0000 7f9fa4426640 1 -- 172.21.15.27:0/1001939507 shutdown_connections 2023-08-11T15:52:11.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.922+0000 7f9fa4426640 1 -- 172.21.15.27:0/1001939507 wait complete. 2023-08-11T15:52:11.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.925+0000 7f9fa4426640 1 Processor -- start 2023-08-11T15:52:11.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.925+0000 7f9fa4426640 1 -- start start 2023-08-11T15:52:11.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.926+0000 7f9fa4426640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f9c0ccaf0 0x7f9f9c14a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.926+0000 7f9fa4426640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 0x7f9f9c0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.926+0000 7f9fa4426640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f9c15dab0 0x7f9f9c0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.926+0000 7f9fa4426640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0c9700 con 0x7f9f9c15dab0 2023-08-11T15:52:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.926+0000 7f9fa4426640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c14aba0 con 0x7f9f9c0ccaf0 2023-08-11T15:52:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.926+0000 7f9fa4426640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0c8af0 con 0x7f9f9c0ce330 2023-08-11T15:52:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa29c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f9c15dab0 0x7f9f9c0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa29c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f9c15dab0 0x7f9f9c0c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40910/0 (socket says 172.21.15.27:40910) 2023-08-11T15:52:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa29c3640 1 -- 172.21.15.27:0/2073108625 learned_addr learned my addr 172.21.15.27:0/2073108625 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa21c2640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f9c0ccaf0 0x7f9f9c14a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa19c1640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 0x7f9f9c0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa29c3640 1 -- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 msgr2=0x7f9f9c0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa29c3640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 0x7f9f9c0b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa29c3640 1 -- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f9c0ccaf0 msgr2=0x7f9f9c14a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa29c3640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f9c0ccaf0 0x7f9f9c14a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa29c3640 1 -- 172.21.15.27:0/2073108625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c009400 con 0x7f9f9c15dab0 2023-08-11T15:52:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.927+0000 7f9fa21c2640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f9c0ccaf0 0x7f9f9c14a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.928+0000 7f9fa29c3640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f9c15dab0 0x7f9f9c0c9b90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c3c0 tx=0x7f9f9400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.928+0000 7f9f92ffd640 1 -- 172.21.15.27:0/2073108625 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f940052d0 con 0x7f9f9c15dab0 2023-08-11T15:52:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.928+0000 7f9f92ffd640 1 -- 172.21.15.27:0/2073108625 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f94005430 con 0x7f9f9c15dab0 2023-08-11T15:52:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.928+0000 7f9f92ffd640 1 -- 172.21.15.27:0/2073108625 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94003810 con 0x7f9f9c15dab0 2023-08-11T15:52:11.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.928+0000 7f9fa4426640 1 -- 172.21.15.27:0/2073108625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c0c8dd0 con 0x7f9f9c15dab0 2023-08-11T15:52:11.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.928+0000 7f9fa4426640 1 -- 172.21.15.27:0/2073108625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c0c9390 con 0x7f9f9c15dab0 2023-08-11T15:52:11.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.930+0000 7f9fa4426640 1 -- 172.21.15.27:0/2073108625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c0c98b0 con 0x7f9f9c15dab0 2023-08-11T15:52:11.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.930+0000 7f9f92ffd640 1 -- 172.21.15.27:0/2073108625 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9f94003970 con 0x7f9f9c15dab0 2023-08-11T15:52:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.931+0000 7f9f92ffd640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9f8006a050 0x7f9f8006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.932+0000 7f9f92ffd640 1 -- 172.21.15.27:0/2073108625 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9f94087f20 con 0x7f9f9c15dab0 2023-08-11T15:52:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.932+0000 7f9fa21c2640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9f8006a050 0x7f9f8006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.933+0000 7f9fa21c2640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9f8006a050 0x7f9f8006c510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009dc0 tx=0x7f9f9800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:11.936+0000 7f9f92ffd640 1 -- 172.21.15.27:0/2073108625 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9f9c0c98b0 con 0x7f9f9c15dab0 2023-08-11T15:52:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.131+0000 7f9fa4426640 1 -- 172.21.15.27:0/2073108625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9f9c0c98b0 con 0x7f9f9c15dab0 2023-08-11T15:52:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.132+0000 7f9f92ffd640 1 -- 172.21.15.27:0/2073108625 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9f9c0c98b0 con 0x7f9f9c15dab0 2023-08-11T15:52:12.133 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:12.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.134+0000 7f9fa4426640 1 -- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9f8006a050 msgr2=0x7f9f8006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.134+0000 7f9fa4426640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9f8006a050 0x7f9f8006c510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009dc0 tx=0x7f9f9800b040 comp rx=0 tx=0).stop 2023-08-11T15:52:12.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.134+0000 7f9fa4426640 1 -- 172.21.15.27:0/2073108625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f9c15dab0 msgr2=0x7f9f9c0c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.134+0000 7f9fa4426640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f9c15dab0 0x7f9f9c0c9b90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c3c0 tx=0x7f9f9400c880 comp rx=0 tx=0).stop 2023-08-11T15:52:12.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.134+0000 7f9fa4426640 1 -- 172.21.15.27:0/2073108625 shutdown_connections 2023-08-11T15:52:12.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.134+0000 7f9fa4426640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9f8006a050 0x7f9f8006c510 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.134+0000 7f9fa4426640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f9c0ce330 0x7f9f9c0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.134+0000 7f9fa4426640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f9c15dab0 0x7f9f9c0c9b90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.134+0000 7f9fa4426640 1 --2- 172.21.15.27:0/2073108625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f9c0ccaf0 0x7f9f9c14a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.134+0000 7f9fa4426640 1 -- 172.21.15.27:0/2073108625 >> 172.21.15.27:0/2073108625 conn(0x7f9f9c0d0220 msgr2=0x7f9f9c06d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:12.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.135+0000 7f9fa4426640 1 -- 172.21.15.27:0/2073108625 shutdown_connections 2023-08-11T15:52:12.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.135+0000 7f9fa4426640 1 -- 172.21.15.27:0/2073108625 wait complete. 2023-08-11T15:52:12.156 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:12.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.383+0000 7f6bb8f31640 1 Processor -- start 2023-08-11T15:52:12.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.383+0000 7f6bb8f31640 1 -- start start 2023-08-11T15:52:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.384+0000 7f6bb8f31640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6bb40b3090 0x7f6bb40d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.385+0000 7f6bb8f31640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6bb4161210 0x7f6bb40b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.385+0000 7f6bb8f31640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb40af250 con 0x7f6bb40b38f0 2023-08-11T15:52:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.385+0000 7f6bb8f31640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb40b0190 con 0x7f6bb4161210 2023-08-11T15:52:12.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.385+0000 7f6bb8f31640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb40af510 con 0x7f6bb40b3090 2023-08-11T15:52:12.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.385+0000 7f6bb2d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6bb4161210 0x7f6bb40b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.385+0000 7f6bb2d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6bb4161210 0x7f6bb40b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40920/0 (socket says 172.21.15.27:40920) 2023-08-11T15:52:12.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.385+0000 7f6bb2d9d640 1 -- 172.21.15.27:0/1534108747 learned_addr learned my addr 172.21.15.27:0/1534108747 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:12.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.385+0000 7f6bb1d9b640 1 --2- 172.21.15.27:0/1534108747 >> v2:172.21.15.119:3300/0 conn(0x7f6bb40b3090 0x7f6bb40d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.386+0000 7f6bb2d9d640 1 -- 172.21.15.27:0/1534108747 >> v2:172.21.15.119:3300/0 conn(0x7f6bb40b3090 msgr2=0x7f6bb40d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:12.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.386+0000 7f6bb2d9d640 1 --2- 172.21.15.27:0/1534108747 >> v2:172.21.15.119:3300/0 conn(0x7f6bb40b3090 0x7f6bb40d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.386+0000 7f6bb2d9d640 1 -- 172.21.15.27:0/1534108747 >> v1:172.21.15.119:6789/0 conn(0x7f6bb40b38f0 legacy=0x7f6bb4160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.386+0000 7f6bb2d9d640 1 -- 172.21.15.27:0/1534108747 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb40af7f0 con 0x7f6bb4161210 2023-08-11T15:52:12.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.387+0000 7f6bb2d9d640 1 --2- 172.21.15.27:0/1534108747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb4161210 0x7f6bb40b05b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40c6230 tx=0x7f6bac02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea4c6806d35e633 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:12.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.387+0000 7f6bb159a640 1 -- 172.21.15.27:0/1534108747 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac02cb20 con 0x7f6bb4161210 2023-08-11T15:52:12.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.387+0000 7f6bb159a640 1 -- 172.21.15.27:0/1534108747 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac02cc80 con 0x7f6bb4161210 2023-08-11T15:52:12.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.387+0000 7f6bb159a640 1 -- 172.21.15.27:0/1534108747 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0343f0 con 0x7f6bb4161210 2023-08-11T15:52:12.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.388+0000 7f6bb8f31640 1 -- 172.21.15.27:0/1534108747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb4161210 msgr2=0x7f6bb40b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.388+0000 7f6bb8f31640 1 --2- 172.21.15.27:0/1534108747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb4161210 0x7f6bb40b05b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40c6230 tx=0x7f6bac02bf80 comp rx=0 tx=0).stop 2023-08-11T15:52:12.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.388+0000 7f6bb8f31640 1 -- 172.21.15.27:0/1534108747 shutdown_connections 2023-08-11T15:52:12.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.388+0000 7f6bb8f31640 1 --2- 172.21.15.27:0/1534108747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb4161210 0x7f6bb40b05b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.388+0000 7f6bb8f31640 1 --2- 172.21.15.27:0/1534108747 >> v2:172.21.15.119:3300/0 conn(0x7f6bb40b3090 0x7f6bb40d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.388+0000 7f6bb8f31640 1 -- 172.21.15.27:0/1534108747 >> 172.21.15.27:0/1534108747 conn(0x7f6bb409b960 msgr2=0x7f6bb40d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:12.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.388+0000 7f6bb8f31640 1 -- 172.21.15.27:0/1534108747 shutdown_connections 2023-08-11T15:52:12.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.388+0000 7f6bb8f31640 1 -- 172.21.15.27:0/1534108747 wait complete. 2023-08-11T15:52:12.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.390+0000 7f6bb8f31640 1 Processor -- start 2023-08-11T15:52:12.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.390+0000 7f6bb8f31640 1 -- start start 2023-08-11T15:52:12.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.391+0000 7f6bb8f31640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb40b3090 0x7f6bb415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.391+0000 7f6bb8f31640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6bb40b38f0 0x7f6bb4157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.391+0000 7f6bb8f31640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb4161210 0x7f6bb40cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.391+0000 7f6bb8f31640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40ca980 con 0x7f6bb40b3090 2023-08-11T15:52:12.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.391+0000 7f6bb8f31640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb415b740 con 0x7f6bb4161210 2023-08-11T15:52:12.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.391+0000 7f6bb8f31640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40c9d70 con 0x7f6bb40b38f0 2023-08-11T15:52:12.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.391+0000 7f6bb259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb40b3090 0x7f6bb415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb40b3090 0x7f6bb415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40928/0 (socket says 172.21.15.27:40928) 2023-08-11T15:52:12.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb259c640 1 -- 172.21.15.27:0/3324182443 learned_addr learned my addr 172.21.15.27:0/3324182443 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb2d9d640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb4161210 0x7f6bb40cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb1d9b640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6bb40b38f0 0x7f6bb4157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb259c640 1 -- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6bb40b38f0 msgr2=0x7f6bb4157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb259c640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6bb40b38f0 0x7f6bb4157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb259c640 1 -- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb4161210 msgr2=0x7f6bb40cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb259c640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb4161210 0x7f6bb40cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb259c640 1 -- 172.21.15.27:0/3324182443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac009b30 con 0x7f6bb40b3090 2023-08-11T15:52:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb1d9b640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6bb40b38f0 0x7f6bb4157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.392+0000 7f6bb259c640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb40b3090 0x7f6bb415b200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c450 tx=0x7f6ba400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.393+0000 7f6ba2ffd640 1 -- 172.21.15.27:0/3324182443 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40052d0 con 0x7f6bb40b3090 2023-08-11T15:52:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.393+0000 7f6ba2ffd640 1 -- 172.21.15.27:0/3324182443 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba4005450 con 0x7f6bb40b3090 2023-08-11T15:52:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.393+0000 7f6ba2ffd640 1 -- 172.21.15.27:0/3324182443 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4003880 con 0x7f6bb40b3090 2023-08-11T15:52:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.393+0000 7f6bb8f31640 1 -- 172.21.15.27:0/3324182443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb40ca050 con 0x7f6bb40b3090 2023-08-11T15:52:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.393+0000 7f6bb8f31640 1 -- 172.21.15.27:0/3324182443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb40ca610 con 0x7f6bb40b3090 2023-08-11T15:52:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.394+0000 7f6bb8f31640 1 -- 172.21.15.27:0/3324182443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb40ca7a0 con 0x7f6bb40b3090 2023-08-11T15:52:12.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.395+0000 7f6ba2ffd640 1 -- 172.21.15.27:0/3324182443 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6ba40039e0 con 0x7f6bb40b3090 2023-08-11T15:52:12.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.396+0000 7f6ba2ffd640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6b9406a050 0x7f6b9406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.396+0000 7f6ba2ffd640 1 -- 172.21.15.27:0/3324182443 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6ba4088460 con 0x7f6bb40b3090 2023-08-11T15:52:12.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.397+0000 7f6bb1d9b640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6b9406a050 0x7f6b9406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.398+0000 7f6bb1d9b640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6b9406a050 0x7f6b9406c510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6bb41f5370 tx=0x7f6ba8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:12.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.400+0000 7f6ba2ffd640 1 -- 172.21.15.27:0/3324182443 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6ba4053820 con 0x7f6bb40b3090 2023-08-11T15:52:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.594+0000 7f6bb8f31640 1 -- 172.21.15.27:0/3324182443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6bb404e420 con 0x7f6bb40b3090 2023-08-11T15:52:12.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.595+0000 7f6ba2ffd640 1 -- 172.21.15.27:0/3324182443 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6ba4056e40 con 0x7f6bb40b3090 2023-08-11T15:52:12.597 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:12.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 -- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6b9406a050 msgr2=0x7f6b9406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6b9406a050 0x7f6b9406c510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6bb41f5370 tx=0x7f6ba8008040 comp rx=0 tx=0).stop 2023-08-11T15:52:12.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 -- 172.21.15.27:0/3324182443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb40b3090 msgr2=0x7f6bb415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb40b3090 0x7f6bb415b200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c450 tx=0x7f6ba400c910 comp rx=0 tx=0).stop 2023-08-11T15:52:12.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 -- 172.21.15.27:0/3324182443 shutdown_connections 2023-08-11T15:52:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6b9406a050 0x7f6b9406c510 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6bb40b38f0 0x7f6bb4157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb4161210 0x7f6bb40cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 --2- 172.21.15.27:0/3324182443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb40b3090 0x7f6bb415b200 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 -- 172.21.15.27:0/3324182443 >> 172.21.15.27:0/3324182443 conn(0x7f6bb409b960 msgr2=0x7f6bb406b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.598+0000 7f6bb8f31640 1 -- 172.21.15.27:0/3324182443 shutdown_connections 2023-08-11T15:52:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.599+0000 7f6bb8f31640 1 -- 172.21.15.27:0/3324182443 wait complete. 2023-08-11T15:52:12.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.857+0000 7f02ab59e640 1 Processor -- start 2023-08-11T15:52:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.857+0000 7f02ab59e640 1 -- start start 2023-08-11T15:52:12.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.858+0000 7f02ab59e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac15a4f0 0x7f02ac0cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.858+0000 7f02ab59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f02ac0cd770 0x7f02ac0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.858+0000 7f02ab59e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ac0ce300 con 0x7f02ac15a110 2023-08-11T15:52:12.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.858+0000 7f02ab59e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac0ce540 con 0x7f02ac0cd770 2023-08-11T15:52:12.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.858+0000 7f02ab59e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0ce820 con 0x7f02ac15a4f0 2023-08-11T15:52:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.859+0000 7f02a9d9b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac15a4f0 0x7f02ac0cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.859+0000 7f02a9d9b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac15a4f0 0x7f02ac0cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:46082/0 (socket says 172.21.15.27:46082) 2023-08-11T15:52:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.859+0000 7f02a9d9b640 1 -- 172.21.15.27:0/2262897445 learned_addr learned my addr 172.21.15.27:0/2262897445 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.859+0000 7f02aad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f02ac0cd770 0x7f02ac0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.860+0000 7f02aad9d640 1 -- 172.21.15.27:0/2262897445 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac15a4f0 msgr2=0x7f02ac0cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:12.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.860+0000 7f02aad9d640 1 --2- 172.21.15.27:0/2262897445 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac15a4f0 0x7f02ac0cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.860+0000 7f02aad9d640 1 -- 172.21.15.27:0/2262897445 >> v1:172.21.15.119:6789/0 conn(0x7f02ac15a110 legacy=0x7f02ac0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.860+0000 7f02aad9d640 1 -- 172.21.15.27:0/2262897445 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac0ceb00 con 0x7f02ac0cd770 2023-08-11T15:52:12.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.860+0000 7f02a9d9b640 1 --2- 172.21.15.27:0/2262897445 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac15a4f0 0x7f02ac0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:12.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.861+0000 7f02aad9d640 1 --2- 172.21.15.27:0/2262897445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac0cd770 0x7f02ac0ccec0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f02a40097d0 tx=0x7f02a402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec956191541f6d5f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:12.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.862+0000 7f02a959a640 1 -- 172.21.15.27:0/2262897445 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a402d040 con 0x7f02ac0cd770 2023-08-11T15:52:12.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.862+0000 7f02a959a640 1 -- 172.21.15.27:0/2262897445 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a4031810 con 0x7f02ac0cd770 2023-08-11T15:52:12.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.862+0000 7f02a959a640 1 -- 172.21.15.27:0/2262897445 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4031ba0 con 0x7f02ac0cd770 2023-08-11T15:52:12.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.863+0000 7f02ab59e640 1 -- 172.21.15.27:0/2262897445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac0cd770 msgr2=0x7f02ac0ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.863+0000 7f02ab59e640 1 --2- 172.21.15.27:0/2262897445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac0cd770 0x7f02ac0ccec0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f02a40097d0 tx=0x7f02a402b8b0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.863+0000 7f02ab59e640 1 -- 172.21.15.27:0/2262897445 shutdown_connections 2023-08-11T15:52:12.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.863+0000 7f02ab59e640 1 --2- 172.21.15.27:0/2262897445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac0cd770 0x7f02ac0ccec0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.863+0000 7f02ab59e640 1 --2- 172.21.15.27:0/2262897445 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac15a4f0 0x7f02ac0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.863+0000 7f02ab59e640 1 -- 172.21.15.27:0/2262897445 >> 172.21.15.27:0/2262897445 conn(0x7f02ac099390 msgr2=0x7f02ac0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:12.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.863+0000 7f02ab59e640 1 -- 172.21.15.27:0/2262897445 shutdown_connections 2023-08-11T15:52:12.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.863+0000 7f02ab59e640 1 -- 172.21.15.27:0/2262897445 wait complete. 2023-08-11T15:52:12.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.865+0000 7f02ab59e640 1 Processor -- start 2023-08-11T15:52:12.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.865+0000 7f02ab59e640 1 -- start start 2023-08-11T15:52:12.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.866+0000 7f02ab59e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac0cd770 0x7f02ac15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.866+0000 7f02ab59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02ac15a110 0x7f02ac14eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.866+0000 7f02ab59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac15a4f0 0x7f02ac1f1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.866+0000 7f02ab59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac148cf0 con 0x7f02ac15a110 2023-08-11T15:52:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.866+0000 7f02ab59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac1f1760 con 0x7f02ac15a4f0 2023-08-11T15:52:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.866+0000 7f02ab59e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac15afa0 con 0x7f02ac0cd770 2023-08-11T15:52:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.866+0000 7f02a9d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02ac15a110 0x7f02ac14eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.866+0000 7f02a9d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02ac15a110 0x7f02ac14eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40932/0 (socket says 172.21.15.27:40932) 2023-08-11T15:52:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.866+0000 7f02a9d9b640 1 -- 172.21.15.27:0/3772541471 learned_addr learned my addr 172.21.15.27:0/3772541471 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:12.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02aad9d640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac15a4f0 0x7f02ac1f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02a9d9b640 1 -- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac0cd770 msgr2=0x7f02ac15a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02aa59c640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac0cd770 0x7f02ac15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02a9d9b640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac0cd770 0x7f02ac15a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02a9d9b640 1 -- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac15a4f0 msgr2=0x7f02ac1f1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:12.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02a9d9b640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac15a4f0 0x7f02ac1f1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:12.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02a9d9b640 1 -- 172.21.15.27:0/3772541471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294009b30 con 0x7f02ac15a110 2023-08-11T15:52:12.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02aad9d640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac15a4f0 0x7f02ac1f1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:12.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02a9d9b640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02ac15a110 0x7f02ac14eb20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f029400efc0 tx=0x7f029400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:12.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02a2ffd640 1 -- 172.21.15.27:0/3772541471 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400ce60 con 0x7f02ac15a110 2023-08-11T15:52:12.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02a2ffd640 1 -- 172.21.15.27:0/3772541471 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029400f040 con 0x7f02ac15a110 2023-08-11T15:52:12.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02ab59e640 1 -- 172.21.15.27:0/3772541471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a4009400 con 0x7f02ac15a110 2023-08-11T15:52:12.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.867+0000 7f02a2ffd640 1 -- 172.21.15.27:0/3772541471 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02940146c0 con 0x7f02ac15a110 2023-08-11T15:52:12.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.868+0000 7f02ab59e640 1 -- 172.21.15.27:0/3772541471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac15cc60 con 0x7f02ac15a110 2023-08-11T15:52:12.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.869+0000 7f02ab59e640 1 -- 172.21.15.27:0/3772541471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac04e420 con 0x7f02ac15a110 2023-08-11T15:52:12.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.870+0000 7f02a2ffd640 1 -- 172.21.15.27:0/3772541471 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0294014820 con 0x7f02ac15a110 2023-08-11T15:52:12.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.871+0000 7f02a2ffd640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f028c06a050 0x7f028c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:12.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.871+0000 7f02a2ffd640 1 -- 172.21.15.27:0/3772541471 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0294089160 con 0x7f02ac15a110 2023-08-11T15:52:12.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.871+0000 7f02aa59c640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f028c06a050 0x7f028c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:12.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.873+0000 7f02aa59c640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f028c06a050 0x7f028c06c510 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f029c000fb0 tx=0x7f029c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:12.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:12.875+0000 7f02a2ffd640 1 -- 172.21.15.27:0/3772541471 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0294054510 con 0x7f02ac15a110 2023-08-11T15:52:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.068+0000 7f02ab59e640 1 -- 172.21.15.27:0/3772541471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02ac0ccf20 con 0x7f02ac15a110 2023-08-11T15:52:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.069+0000 7f02a2ffd640 1 -- 172.21.15.27:0/3772541471 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0294057b30 con 0x7f02ac15a110 2023-08-11T15:52:13.071 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:13.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.071+0000 7f02ab59e640 1 -- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f028c06a050 msgr2=0x7f028c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.071+0000 7f02ab59e640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f028c06a050 0x7f028c06c510 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f029c000fb0 tx=0x7f029c009040 comp rx=0 tx=0).stop 2023-08-11T15:52:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.072+0000 7f02ab59e640 1 -- 172.21.15.27:0/3772541471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02ac15a110 msgr2=0x7f02ac14eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.072+0000 7f02ab59e640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02ac15a110 0x7f02ac14eb20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f029400efc0 tx=0x7f029400c490 comp rx=0 tx=0).stop 2023-08-11T15:52:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.072+0000 7f02ab59e640 1 -- 172.21.15.27:0/3772541471 shutdown_connections 2023-08-11T15:52:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.072+0000 7f02ab59e640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f028c06a050 0x7f028c06c510 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.072+0000 7f02ab59e640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac15a4f0 0x7f02ac1f1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.072+0000 7f02ab59e640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02ac15a110 0x7f02ac14eb20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.072+0000 7f02ab59e640 1 --2- 172.21.15.27:0/3772541471 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac0cd770 0x7f02ac15a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.072+0000 7f02ab59e640 1 -- 172.21.15.27:0/3772541471 >> 172.21.15.27:0/3772541471 conn(0x7f02ac099390 msgr2=0x7f02ac0ad7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.072+0000 7f02ab59e640 1 -- 172.21.15.27:0/3772541471 shutdown_connections 2023-08-11T15:52:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.072+0000 7f02ab59e640 1 -- 172.21.15.27:0/3772541471 wait complete. 2023-08-11T15:52:13.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:13.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.317+0000 7f766eb42640 1 Processor -- start 2023-08-11T15:52:13.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.318+0000 7f766eb42640 1 -- start start 2023-08-11T15:52:13.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.319+0000 7f766eb42640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f76681592e0 0x7f766815d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.319+0000 7f766eb42640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f76680ccbc0 0x7f76680afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.319+0000 7f766eb42640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76680b0420 con 0x7f766806c960 2023-08-11T15:52:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.319+0000 7f766eb42640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76681596c0 con 0x7f76681592e0 2023-08-11T15:52:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.319+0000 7f766eb42640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76680b09e0 con 0x7f76680ccbc0 2023-08-11T15:52:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.319+0000 7f766db40640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f76681592e0 0x7f766815d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.319+0000 7f766db40640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f76681592e0 0x7f766815d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40936/0 (socket says 172.21.15.27:40936) 2023-08-11T15:52:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.319+0000 7f766db40640 1 -- 172.21.15.27:0/2273990325 learned_addr learned my addr 172.21.15.27:0/2273990325 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.320+0000 7f766d33f640 1 --2- 172.21.15.27:0/2273990325 >> v2:172.21.15.119:3300/0 conn(0x7f76680ccbc0 0x7f76680afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.320+0000 7f766db40640 1 -- 172.21.15.27:0/2273990325 >> v2:172.21.15.119:3300/0 conn(0x7f76680ccbc0 msgr2=0x7f76680afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.320+0000 7f766db40640 1 --2- 172.21.15.27:0/2273990325 >> v2:172.21.15.119:3300/0 conn(0x7f76680ccbc0 0x7f76680afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.320+0000 7f766db40640 1 -- 172.21.15.27:0/2273990325 >> v1:172.21.15.119:6789/0 conn(0x7f766806c960 legacy=0x7f76680703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.320+0000 7f766db40640 1 -- 172.21.15.27:0/2273990325 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76680b0700 con 0x7f76681592e0 2023-08-11T15:52:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.321+0000 7f766db40640 1 --2- 172.21.15.27:0/2273990325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 0x7f766815d730 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f76680c9700 tx=0x7f766402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61188f4ea2704408 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766cb3e640 1 -- 172.21.15.27:0/2273990325 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766402c9f0 con 0x7f76681592e0 2023-08-11T15:52:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766cb3e640 1 -- 172.21.15.27:0/2273990325 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766402cb50 con 0x7f76681592e0 2023-08-11T15:52:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766cb3e640 1 -- 172.21.15.27:0/2273990325 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640343f0 con 0x7f76681592e0 2023-08-11T15:52:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766eb42640 1 -- 172.21.15.27:0/2273990325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 msgr2=0x7f766815d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766eb42640 1 --2- 172.21.15.27:0/2273990325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 0x7f766815d730 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f76680c9700 tx=0x7f766402be50 comp rx=0 tx=0).stop 2023-08-11T15:52:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766eb42640 1 -- 172.21.15.27:0/2273990325 shutdown_connections 2023-08-11T15:52:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766eb42640 1 --2- 172.21.15.27:0/2273990325 >> v2:172.21.15.119:3300/0 conn(0x7f76680ccbc0 0x7f76680afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766eb42640 1 --2- 172.21.15.27:0/2273990325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 0x7f766815d730 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766eb42640 1 -- 172.21.15.27:0/2273990325 >> 172.21.15.27:0/2273990325 conn(0x7f76680caab0 msgr2=0x7f76680cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766eb42640 1 -- 172.21.15.27:0/2273990325 shutdown_connections 2023-08-11T15:52:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.322+0000 7f766eb42640 1 -- 172.21.15.27:0/2273990325 wait complete. 2023-08-11T15:52:13.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.324+0000 7f766eb42640 1 Processor -- start 2023-08-11T15:52:13.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.324+0000 7f766eb42640 1 -- start start 2023-08-11T15:52:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.325+0000 7f766eb42640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f766806c960 0x7f76680ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.325+0000 7f766eb42640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76680ccbc0 0x7f766815c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.325+0000 7f766eb42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 0x7f766814ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.325+0000 7f766eb42640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76680acd60 con 0x7f76681592e0 2023-08-11T15:52:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.325+0000 7f766eb42640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76680ac1e0 con 0x7f766806c960 2023-08-11T15:52:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.325+0000 7f766eb42640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76680ac4c0 con 0x7f76680ccbc0 2023-08-11T15:52:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.326+0000 7f766e341640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 0x7f766814ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.326+0000 7f766e341640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 0x7f766814ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40948/0 (socket says 172.21.15.27:40948) 2023-08-11T15:52:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.326+0000 7f766e341640 1 -- 172.21.15.27:0/1361418961 learned_addr learned my addr 172.21.15.27:0/1361418961 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.326+0000 7f766e341640 1 -- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76680ccbc0 msgr2=0x7f766815c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.326+0000 7f766e341640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76680ccbc0 0x7f766815c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.326+0000 7f766db40640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f766806c960 0x7f76680ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.326+0000 7f766e341640 1 -- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f766806c960 msgr2=0x7f76680ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.326+0000 7f766e341640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f766806c960 0x7f76680ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.326+0000 7f766e341640 1 -- 172.21.15.27:0/1361418961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7664009b30 con 0x7f76681592e0 2023-08-11T15:52:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.326+0000 7f766e341640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 0x7f766814ff30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f766000b410 tx=0x7f766000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.327+0000 7f765e7fc640 1 -- 172.21.15.27:0/1361418961 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660004020 con 0x7f76681592e0 2023-08-11T15:52:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.327+0000 7f765e7fc640 1 -- 172.21.15.27:0/1361418961 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660004180 con 0x7f76681592e0 2023-08-11T15:52:13.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.327+0000 7f765e7fc640 1 -- 172.21.15.27:0/1361418961 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660011630 con 0x7f76681592e0 2023-08-11T15:52:13.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.327+0000 7f766eb42640 1 -- 172.21.15.27:0/1361418961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76680ac7a0 con 0x7f76681592e0 2023-08-11T15:52:13.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.327+0000 7f766eb42640 1 -- 172.21.15.27:0/1361418961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76680abe20 con 0x7f76681592e0 2023-08-11T15:52:13.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.328+0000 7f766eb42640 1 -- 172.21.15.27:0/1361418961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766804e420 con 0x7f76681592e0 2023-08-11T15:52:13.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.329+0000 7f765e7fc640 1 -- 172.21.15.27:0/1361418961 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7660011790 con 0x7f76681592e0 2023-08-11T15:52:13.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.331+0000 7f765e7fc640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f764c06a050 0x7f764c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.331+0000 7f765e7fc640 1 -- 172.21.15.27:0/1361418961 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7660088580 con 0x7f76681592e0 2023-08-11T15:52:13.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.331+0000 7f766db40640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f764c06a050 0x7f764c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.332+0000 7f766db40640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f764c06a050 0x7f764c06c510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7664035000 tx=0x7f7664030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:13.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.336+0000 7f765e7fc640 1 -- 172.21.15.27:0/1361418961 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f76600539d0 con 0x7f76681592e0 2023-08-11T15:52:13.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.529+0000 7f766eb42640 1 -- 172.21.15.27:0/1361418961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f76680cc910 con 0x7f76681592e0 2023-08-11T15:52:13.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.531+0000 7f765e7fc640 1 -- 172.21.15.27:0/1361418961 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7660056ff0 con 0x7f76681592e0 2023-08-11T15:52:13.532 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.533+0000 7f766eb42640 1 -- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f764c06a050 msgr2=0x7f764c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:13.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.533+0000 7f766eb42640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f764c06a050 0x7f764c06c510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7664035000 tx=0x7f7664030000 comp rx=0 tx=0).stop 2023-08-11T15:52:13.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.533+0000 7f766eb42640 1 -- 172.21.15.27:0/1361418961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 msgr2=0x7f766814ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:13.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.533+0000 7f766eb42640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 0x7f766814ff30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f766000b410 tx=0x7f766000b8d0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.534+0000 7f766eb42640 1 -- 172.21.15.27:0/1361418961 shutdown_connections 2023-08-11T15:52:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.534+0000 7f766eb42640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f764c06a050 0x7f764c06c510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.534+0000 7f766eb42640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76680ccbc0 0x7f766815c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.534+0000 7f766eb42640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76681592e0 0x7f766814ff30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.534+0000 7f766eb42640 1 --2- 172.21.15.27:0/1361418961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f766806c960 0x7f76680ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.534+0000 7f766eb42640 1 -- 172.21.15.27:0/1361418961 >> 172.21.15.27:0/1361418961 conn(0x7f76680caab0 msgr2=0x7f76680b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.534+0000 7f766eb42640 1 -- 172.21.15.27:0/1361418961 shutdown_connections 2023-08-11T15:52:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.534+0000 7f766eb42640 1 -- 172.21.15.27:0/1361418961 wait complete. 2023-08-11T15:52:13.555 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-11T15:52:13.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.795+0000 7fd4c13bd640 1 Processor -- start 2023-08-11T15:52:13.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.796+0000 7fd4c13bd640 1 -- start start 2023-08-11T15:52:13.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.798+0000 7fd4c13bd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd4bc13ce80 0x7fd4bc13f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.798+0000 7fd4c13bd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc13d260 0x7fd4bc13fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.798+0000 7fd4c13bd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4bc0cc720 0x7fd4bc0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.798+0000 7fd4c13bd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4bc140670 con 0x7fd4bc13ce80 2023-08-11T15:52:13.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.798+0000 7fd4c13bd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4bc0cd470 con 0x7fd4bc0cc720 2023-08-11T15:52:13.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.798+0000 7fd4c13bd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc0cd750 con 0x7fd4bc13d260 2023-08-11T15:52:13.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.798+0000 7fd4bbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd4bc13ce80 0x7fd4bc13f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.798+0000 7fd4bbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd4bc13ce80 0x7fd4bc13f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40958/0 (socket says 172.21.15.27:40958) 2023-08-11T15:52:13.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.798+0000 7fd4bbfff640 1 -- 172.21.15.27:0/3304799112 learned_addr learned my addr 172.21.15.27:0/3304799112 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:13.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.798+0000 7fd4c0bbc640 1 --2- 172.21.15.27:0/3304799112 >> v2:172.21.15.119:3300/0 conn(0x7fd4bc0cc720 0x7fd4bc0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.799+0000 7fd4bb7fe640 1 --2- 172.21.15.27:0/3304799112 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc13d260 0x7fd4bc13fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.799+0000 7fd4bbfff640 1 -- 172.21.15.27:0/3304799112 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc13d260 msgr2=0x7fd4bc13fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:13.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.799+0000 7fd4bbfff640 1 --2- 172.21.15.27:0/3304799112 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc13d260 0x7fd4bc13fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.799+0000 7fd4bbfff640 1 -- 172.21.15.27:0/3304799112 >> v2:172.21.15.119:3300/0 conn(0x7fd4bc0cc720 msgr2=0x7fd4bc0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.799+0000 7fd4bbfff640 1 --2- 172.21.15.27:0/3304799112 >> v2:172.21.15.119:3300/0 conn(0x7fd4bc0cc720 0x7fd4bc0ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.799+0000 7fd4bbfff640 1 -- 172.21.15.27:0/3304799112 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc0ad7d0 con 0x7fd4bc13ce80 2023-08-11T15:52:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.799+0000 7fd4bb7fe640 1 --2- 172.21.15.27:0/3304799112 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc13d260 0x7fd4bc13fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:13.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.800+0000 7fd4bbfff640 1 --2- 172.21.15.27:0/3304799112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 0x7fd4bc13f810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0cbfb0 tx=0x7fd4ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd0dd4df5a1b16a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.801+0000 7fd4baffd640 1 -- 172.21.15.27:0/3304799112 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac031870 con 0x7fd4bc13ce80 2023-08-11T15:52:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.801+0000 7fd4baffd640 1 -- 172.21.15.27:0/3304799112 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ac0319d0 con 0x7fd4bc13ce80 2023-08-11T15:52:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.801+0000 7fd4baffd640 1 -- 172.21.15.27:0/3304799112 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac031d60 con 0x7fd4bc13ce80 2023-08-11T15:52:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.801+0000 7fd4c13bd640 1 -- 172.21.15.27:0/3304799112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 msgr2=0x7fd4bc13f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:13.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.801+0000 7fd4c13bd640 1 --2- 172.21.15.27:0/3304799112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 0x7fd4bc13f810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0cbfb0 tx=0x7fd4ac02b910 comp rx=0 tx=0).stop 2023-08-11T15:52:13.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.802+0000 7fd4c13bd640 1 -- 172.21.15.27:0/3304799112 shutdown_connections 2023-08-11T15:52:13.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.802+0000 7fd4c13bd640 1 --2- 172.21.15.27:0/3304799112 >> v2:172.21.15.119:3300/0 conn(0x7fd4bc0cc720 0x7fd4bc0ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.802+0000 7fd4c13bd640 1 --2- 172.21.15.27:0/3304799112 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc13d260 0x7fd4bc13fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.802+0000 7fd4c13bd640 1 --2- 172.21.15.27:0/3304799112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 0x7fd4bc13f810 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.802+0000 7fd4c13bd640 1 -- 172.21.15.27:0/3304799112 >> 172.21.15.27:0/3304799112 conn(0x7fd4bc069800 msgr2=0x7fd4bc069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:13.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.802+0000 7fd4c13bd640 1 -- 172.21.15.27:0/3304799112 shutdown_connections 2023-08-11T15:52:13.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.802+0000 7fd4c13bd640 1 -- 172.21.15.27:0/3304799112 wait complete. 2023-08-11T15:52:13.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.804+0000 7fd4c13bd640 1 Processor -- start 2023-08-11T15:52:13.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.804+0000 7fd4c13bd640 1 -- start start 2023-08-11T15:52:13.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4c13bd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc0cc720 0x7fd4bc0b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4c13bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 0x7fd4bc0ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4c13bd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4bc13d260 0x7fd4bc153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4c13bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc139240 con 0x7fd4bc13ce80 2023-08-11T15:52:13.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4c13bd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc0b3240 con 0x7fd4bc13d260 2023-08-11T15:52:13.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4c13bd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc139450 con 0x7fd4bc0cc720 2023-08-11T15:52:13.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4bb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 0x7fd4bc0ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4bb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 0x7fd4bc0ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40972/0 (socket says 172.21.15.27:40972) 2023-08-11T15:52:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4bb7fe640 1 -- 172.21.15.27:0/647271032 learned_addr learned my addr 172.21.15.27:0/647271032 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4c0bbc640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4bc13d260 0x7fd4bc153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.805+0000 7fd4bbfff640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc0cc720 0x7fd4bc0b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.806+0000 7fd4bb7fe640 1 -- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc0cc720 msgr2=0x7fd4bc0b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:13.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.806+0000 7fd4bb7fe640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc0cc720 0x7fd4bc0b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.806+0000 7fd4bb7fe640 1 -- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4bc13d260 msgr2=0x7fd4bc153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:13.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.806+0000 7fd4bb7fe640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4bc13d260 0x7fd4bc153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:13.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.806+0000 7fd4bb7fe640 1 -- 172.21.15.27:0/647271032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac0094d0 con 0x7fd4bc13ce80 2023-08-11T15:52:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.806+0000 7fd4bb7fe640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 0x7fd4bc0ae140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000c7f0 tx=0x7fd4b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.806+0000 7fd4b8ff9640 1 -- 172.21.15.27:0/647271032 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0007820 con 0x7fd4bc13ce80 2023-08-11T15:52:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.806+0000 7fd4b8ff9640 1 -- 172.21.15.27:0/647271032 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b0007980 con 0x7fd4bc13ce80 2023-08-11T15:52:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.806+0000 7fd4b8ff9640 1 -- 172.21.15.27:0/647271032 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0007cf0 con 0x7fd4bc13ce80 2023-08-11T15:52:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.806+0000 7fd4c13bd640 1 -- 172.21.15.27:0/647271032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4bc139730 con 0x7fd4bc13ce80 2023-08-11T15:52:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.807+0000 7fd4c13bd640 1 -- 172.21.15.27:0/647271032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4bc139cf0 con 0x7fd4bc13ce80 2023-08-11T15:52:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.808+0000 7fd4c13bd640 1 -- 172.21.15.27:0/647271032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4bc066e80 con 0x7fd4bc13ce80 2023-08-11T15:52:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.809+0000 7fd4b8ff9640 1 -- 172.21.15.27:0/647271032 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd4b00113f0 con 0x7fd4bc13ce80 2023-08-11T15:52:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.810+0000 7fd4b8ff9640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd4a806a050 0x7fd4a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.810+0000 7fd4b8ff9640 1 -- 172.21.15.27:0/647271032 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd4b0088c60 con 0x7fd4bc13ce80 2023-08-11T15:52:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.810+0000 7fd4bbfff640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd4a806a050 0x7fd4a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.811+0000 7fd4bbfff640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd4a806a050 0x7fd4a806c510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac036000 tx=0x7fd4ac03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:13.814+0000 7fd4b8ff9640 1 -- 172.21.15.27:0/647271032 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fd4b0053f80 con 0x7fd4bc13ce80 2023-08-11T15:52:14.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.023+0000 7fd4c13bd640 1 -- 172.21.15.27:0/647271032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd4bc06a6e0 con 0x7fd4bc13ce80 2023-08-11T15:52:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.032+0000 7fd4b8ff9640 1 -- 172.21.15.27:0/647271032 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd4b00575a0 con 0x7fd4bc13ce80 2023-08-11T15:52:14.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.034+0000 7fd4c13bd640 1 -- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd4a806a050 msgr2=0x7fd4a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.034+0000 7fd4c13bd640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd4a806a050 0x7fd4a806c510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac036000 tx=0x7fd4ac03c040 comp rx=0 tx=0).stop 2023-08-11T15:52:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.034+0000 7fd4c13bd640 1 -- 172.21.15.27:0/647271032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 msgr2=0x7fd4bc0ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.034+0000 7fd4c13bd640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 0x7fd4bc0ae140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000c7f0 tx=0x7fd4b000ccb0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.034+0000 7fd4c13bd640 1 -- 172.21.15.27:0/647271032 shutdown_connections 2023-08-11T15:52:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.035+0000 7fd4c13bd640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd4a806a050 0x7fd4a806c510 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.035+0000 7fd4c13bd640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4bc13d260 0x7fd4bc153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.035+0000 7fd4c13bd640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc13ce80 0x7fd4bc0ae140 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.035+0000 7fd4c13bd640 1 --2- 172.21.15.27:0/647271032 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4bc0cc720 0x7fd4bc0b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.035+0000 7fd4c13bd640 1 -- 172.21.15.27:0/647271032 >> 172.21.15.27:0/647271032 conn(0x7fd4bc069800 msgr2=0x7fd4bc15e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.035+0000 7fd4c13bd640 1 -- 172.21.15.27:0/647271032 shutdown_connections 2023-08-11T15:52:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.035+0000 7fd4c13bd640 1 -- 172.21.15.27:0/647271032 wait complete. 2023-08-11T15:52:14.055 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-11T15:52:14.055 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-11T15:52:14.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-11T15:52:14.306 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-11T15:52:14.307 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:52:14.307 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:14.371 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:52:14.371 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:14.429 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:14.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.660+0000 7fdd655bd640 1 Processor -- start 2023-08-11T15:52:14.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.660+0000 7fdd655bd640 1 -- start start 2023-08-11T15:52:14.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.662+0000 7fdd655bd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd6015a020 0x7fdd6015e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:14.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.662+0000 7fdd655bd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd600c9600 0x7fdd60071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:14.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.662+0000 7fdd655bd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdd6006f5d0 0x7fdd6006fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:14.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.662+0000 7fdd655bd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6015eae0 con 0x7fdd6006f5d0 2023-08-11T15:52:14.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.662+0000 7fdd655bd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6006ff80 con 0x7fdd6015a020 2023-08-11T15:52:14.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.662+0000 7fdd655bd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600c85f0 con 0x7fdd600c9600 2023-08-11T15:52:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.662+0000 7fdd64dbc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdd6006f5d0 0x7fdd6006fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.662+0000 7fdd64dbc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdd6006f5d0 0x7fdd6006fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40974/0 (socket says 172.21.15.27:40974) 2023-08-11T15:52:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.662+0000 7fdd64dbc640 1 -- 172.21.15.27:0/2411279716 learned_addr learned my addr 172.21.15.27:0/2411279716 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.662+0000 7fdd5f7fe640 1 --2- 172.21.15.27:0/2411279716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd600c9600 0x7fdd60071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.663+0000 7fdd5ffff640 1 --2- 172.21.15.27:0/2411279716 >> v2:172.21.15.119:3300/0 conn(0x7fdd6015a020 0x7fdd6015e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:14.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.663+0000 7fdd64dbc640 1 -- 172.21.15.27:0/2411279716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd600c9600 msgr2=0x7fdd60071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:14.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.663+0000 7fdd64dbc640 1 --2- 172.21.15.27:0/2411279716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd600c9600 0x7fdd60071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.663+0000 7fdd64dbc640 1 -- 172.21.15.27:0/2411279716 >> v2:172.21.15.119:3300/0 conn(0x7fdd6015a020 msgr2=0x7fdd6015e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:14.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.663+0000 7fdd64dbc640 1 --2- 172.21.15.27:0/2411279716 >> v2:172.21.15.119:3300/0 conn(0x7fdd6015a020 0x7fdd6015e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.663+0000 7fdd64dbc640 1 -- 172.21.15.27:0/2411279716 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd60071980 con 0x7fdd6006f5d0 2023-08-11T15:52:14.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.664+0000 7fdd64dbc640 1 --2- 172.21.15.27:0/2411279716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6006f5d0 0x7fdd6006fa40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdd600cdf00 tx=0x7fdd5802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82994d64b2ed1e0e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:14.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd5effd640 1 -- 172.21.15.27:0/2411279716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5802c9e0 con 0x7fdd6006f5d0 2023-08-11T15:52:14.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd5effd640 1 -- 172.21.15.27:0/2411279716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5802cb40 con 0x7fdd6006f5d0 2023-08-11T15:52:14.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd5effd640 1 -- 172.21.15.27:0/2411279716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580343f0 con 0x7fdd6006f5d0 2023-08-11T15:52:14.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd655bd640 1 -- 172.21.15.27:0/2411279716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6006f5d0 msgr2=0x7fdd6006fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:14.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd655bd640 1 --2- 172.21.15.27:0/2411279716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6006f5d0 0x7fdd6006fa40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdd600cdf00 tx=0x7fdd5802be40 comp rx=0 tx=0).stop 2023-08-11T15:52:14.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd655bd640 1 -- 172.21.15.27:0/2411279716 shutdown_connections 2023-08-11T15:52:14.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd655bd640 1 --2- 172.21.15.27:0/2411279716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd6006f5d0 0x7fdd6006fa40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd655bd640 1 --2- 172.21.15.27:0/2411279716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd600c9600 0x7fdd60071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd655bd640 1 --2- 172.21.15.27:0/2411279716 >> v2:172.21.15.119:3300/0 conn(0x7fdd6015a020 0x7fdd6015e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd655bd640 1 -- 172.21.15.27:0/2411279716 >> 172.21.15.27:0/2411279716 conn(0x7fdd600afb70 msgr2=0x7fdd600aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:14.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.665+0000 7fdd655bd640 1 -- 172.21.15.27:0/2411279716 shutdown_connections 2023-08-11T15:52:14.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.666+0000 7fdd655bd640 1 -- 172.21.15.27:0/2411279716 wait complete. 2023-08-11T15:52:14.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.667+0000 7fdd655bd640 1 Processor -- start 2023-08-11T15:52:14.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.667+0000 7fdd655bd640 1 -- start start 2023-08-11T15:52:14.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.668+0000 7fdd655bd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6006f5d0 0x7fdd601509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:14.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.668+0000 7fdd655bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd600c9600 0x7fdd6015cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:14.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.668+0000 7fdd655bd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd6015a020 0x7fdd600c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:14.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.668+0000 7fdd655bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60152980 con 0x7fdd600c9600 2023-08-11T15:52:14.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.668+0000 7fdd655bd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601520e0 con 0x7fdd6006f5d0 2023-08-11T15:52:14.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.668+0000 7fdd655bd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601523c0 con 0x7fdd6015a020 2023-08-11T15:52:14.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.669+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd600c9600 0x7fdd6015cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:14.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.669+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd600c9600 0x7fdd6015cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40988/0 (socket says 172.21.15.27:40988) 2023-08-11T15:52:14.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.669+0000 7fdd5f7fe640 1 -- 172.21.15.27:0/3004049748 learned_addr learned my addr 172.21.15.27:0/3004049748 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:14.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.669+0000 7fdd5ffff640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6006f5d0 0x7fdd601509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:14.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.669+0000 7fdd64dbc640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd6015a020 0x7fdd600c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:14.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.669+0000 7fdd5f7fe640 1 -- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd6015a020 msgr2=0x7fdd600c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:14.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.669+0000 7fdd5f7fe640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd6015a020 0x7fdd600c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.669+0000 7fdd5f7fe640 1 -- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6006f5d0 msgr2=0x7fdd601509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:14.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.669+0000 7fdd5f7fe640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6006f5d0 0x7fdd601509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.669+0000 7fdd5f7fe640 1 -- 172.21.15.27:0/3004049748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58009b30 con 0x7fdd600c9600 2023-08-11T15:52:14.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.670+0000 7fdd5f7fe640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd600c9600 0x7fdd6015cc00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400d800 tx=0x7fdd5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:14.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.670+0000 7fdd5cff9640 1 -- 172.21.15.27:0/3004049748 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400b6c0 con 0x7fdd600c9600 2023-08-11T15:52:14.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.670+0000 7fdd5cff9640 1 -- 172.21.15.27:0/3004049748 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5400b820 con 0x7fdd600c9600 2023-08-11T15:52:14.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.670+0000 7fdd5cff9640 1 -- 172.21.15.27:0/3004049748 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400bbb0 con 0x7fdd600c9600 2023-08-11T15:52:14.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.670+0000 7fdd655bd640 1 -- 172.21.15.27:0/3004049748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd601526a0 con 0x7fdd600c9600 2023-08-11T15:52:14.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.670+0000 7fdd655bd640 1 -- 172.21.15.27:0/3004049748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd60150ef0 con 0x7fdd600c9600 2023-08-11T15:52:14.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.672+0000 7fdd655bd640 1 -- 172.21.15.27:0/3004049748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd60151080 con 0x7fdd600c9600 2023-08-11T15:52:14.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.672+0000 7fdd5cff9640 1 -- 172.21.15.27:0/3004049748 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdd5400bd10 con 0x7fdd600c9600 2023-08-11T15:52:14.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.673+0000 7fdd5cff9640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdd4c06a050 0x7fdd4c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:14.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.674+0000 7fdd5cff9640 1 -- 172.21.15.27:0/3004049748 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdd54088dc0 con 0x7fdd600c9600 2023-08-11T15:52:14.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.674+0000 7fdd5ffff640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdd4c06a050 0x7fdd4c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:14.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.675+0000 7fdd5ffff640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdd4c06a050 0x7fdd4c06c510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd500099b0 tx=0x7fdd50008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:14.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.678+0000 7fdd5cff9640 1 -- 172.21.15.27:0/3004049748 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fdd60151080 con 0x7fdd600c9600 2023-08-11T15:52:14.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.873+0000 7fdd655bd640 1 -- 172.21.15.27:0/3004049748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd60151080 con 0x7fdd600c9600 2023-08-11T15:52:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.877+0000 7fdd5cff9640 1 -- 172.21.15.27:0/3004049748 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7fdd60151080 con 0x7fdd600c9600 2023-08-11T15:52:14.879 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:14.879 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 -- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdd4c06a050 msgr2=0x7fdd4c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdd4c06a050 0x7fdd4c06c510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd500099b0 tx=0x7fdd50008040 comp rx=0 tx=0).stop 2023-08-11T15:52:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 -- 172.21.15.27:0/3004049748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd600c9600 msgr2=0x7fdd6015cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd600c9600 0x7fdd6015cc00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400d800 tx=0x7fdd5400dcc0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 -- 172.21.15.27:0/3004049748 shutdown_connections 2023-08-11T15:52:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdd4c06a050 0x7fdd4c06c510 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdd6015a020 0x7fdd600c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd600c9600 0x7fdd6015cc00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 --2- 172.21.15.27:0/3004049748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6006f5d0 0x7fdd601509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 -- 172.21.15.27:0/3004049748 >> 172.21.15.27:0/3004049748 conn(0x7fdd600afb70 msgr2=0x7fdd600aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 -- 172.21.15.27:0/3004049748 shutdown_connections 2023-08-11T15:52:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:14.879+0000 7fdd655bd640 1 -- 172.21.15.27:0/3004049748 wait complete. 2023-08-11T15:52:14.899 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:14.900 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:15.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.130+0000 7f5c8efa4640 1 Processor -- start 2023-08-11T15:52:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.130+0000 7f5c8efa4640 1 -- start start 2023-08-11T15:52:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.132+0000 7f5c8efa4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c881403a0 0x7f5c880b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.132+0000 7f5c8efa4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c88140ba0 con 0x7f5c88154ac0 2023-08-11T15:52:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.132+0000 7f5c8efa4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8815b220 con 0x7f5c880cb700 2023-08-11T15:52:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.132+0000 7f5c8efa4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c88071b10 con 0x7f5c881403a0 2023-08-11T15:52:15.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.133+0000 7f5c8d541640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c881403a0 0x7f5c880b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.133+0000 7f5c8d541640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c881403a0 0x7f5c880b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40990/0 (socket says 172.21.15.27:40990) 2023-08-11T15:52:15.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.133+0000 7f5c8d541640 1 -- 172.21.15.27:0/1596790845 learned_addr learned my addr 172.21.15.27:0/1596790845 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:15.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.133+0000 7f5c8d541640 1 -- 172.21.15.27:0/1596790845 >> v1:172.21.15.119:6789/0 conn(0x7f5c880cb700 legacy=0x7f5c8813fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.133+0000 7f5c8d541640 1 -- 172.21.15.27:0/1596790845 >> v1:172.21.15.27:6789/0 conn(0x7f5c88154ac0 legacy=0x7f5c880caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.133+0000 7f5c8d541640 1 -- 172.21.15.27:0/1596790845 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c881408c0 con 0x7f5c881403a0 2023-08-11T15:52:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.134+0000 7f5c8d541640 1 --2- 172.21.15.27:0/1596790845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c881403a0 0x7f5c880b2fc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009e50 tx=0x7f5c8002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b9cf5e94d496d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.135+0000 7f5c877fe640 1 -- 172.21.15.27:0/1596790845 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8002ca80 con 0x7f5c881403a0 2023-08-11T15:52:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.135+0000 7f5c877fe640 1 -- 172.21.15.27:0/1596790845 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8002cbe0 con 0x7f5c881403a0 2023-08-11T15:52:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.135+0000 7f5c877fe640 1 -- 172.21.15.27:0/1596790845 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800343f0 con 0x7f5c881403a0 2023-08-11T15:52:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.135+0000 7f5c8efa4640 1 -- 172.21.15.27:0/1596790845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c881403a0 msgr2=0x7f5c880b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.135+0000 7f5c8efa4640 1 --2- 172.21.15.27:0/1596790845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c881403a0 0x7f5c880b2fc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009e50 tx=0x7f5c8002bee0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.135+0000 7f5c8efa4640 1 -- 172.21.15.27:0/1596790845 shutdown_connections 2023-08-11T15:52:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.135+0000 7f5c8efa4640 1 --2- 172.21.15.27:0/1596790845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c881403a0 0x7f5c880b2fc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.135+0000 7f5c8efa4640 1 -- 172.21.15.27:0/1596790845 >> 172.21.15.27:0/1596790845 conn(0x7f5c88069bf0 msgr2=0x7f5c8806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.136+0000 7f5c8efa4640 1 -- 172.21.15.27:0/1596790845 shutdown_connections 2023-08-11T15:52:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.136+0000 7f5c8efa4640 1 -- 172.21.15.27:0/1596790845 wait complete. 2023-08-11T15:52:15.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.137+0000 7f5c8efa4640 1 Processor -- start 2023-08-11T15:52:15.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.138+0000 7f5c8efa4640 1 -- start start 2023-08-11T15:52:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.138+0000 7f5c8efa4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c880cb700 0x7f5c88153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.138+0000 7f5c8efa4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c881403a0 0x7f5c880ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8efa4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c88154ac0 0x7f5c880ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8efa4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c88153da0 con 0x7f5c88154ac0 2023-08-11T15:52:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8efa4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c880c29c0 con 0x7f5c881403a0 2023-08-11T15:52:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8efa4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c880c1db0 con 0x7f5c880cb700 2023-08-11T15:52:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8d541640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c88154ac0 0x7f5c880ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8d541640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c88154ac0 0x7f5c880ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41004/0 (socket says 172.21.15.27:41004) 2023-08-11T15:52:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8d541640 1 -- 172.21.15.27:0/2488287242 learned_addr learned my addr 172.21.15.27:0/2488287242 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8cd40640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c880cb700 0x7f5c88153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c87fff640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c881403a0 0x7f5c880ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8d541640 1 -- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c880cb700 msgr2=0x7f5c88153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8d541640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c880cb700 0x7f5c88153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8d541640 1 -- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c881403a0 msgr2=0x7f5c880ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8d541640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c881403a0 0x7f5c880ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8d541640 1 -- 172.21.15.27:0/2488287242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c80009b30 con 0x7f5c88154ac0 2023-08-11T15:52:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c8cd40640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c880cb700 0x7f5c88153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:15.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.139+0000 7f5c87fff640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c881403a0 0x7f5c880ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.140+0000 7f5c8d541640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c88154ac0 0x7f5c880ace70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5c80036040 tx=0x7f5c80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.140+0000 7f5c857fa640 1 -- 172.21.15.27:0/2488287242 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8003e450 con 0x7f5c88154ac0 2023-08-11T15:52:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.140+0000 7f5c857fa640 1 -- 172.21.15.27:0/2488287242 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8003e5d0 con 0x7f5c88154ac0 2023-08-11T15:52:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.140+0000 7f5c857fa640 1 -- 172.21.15.27:0/2488287242 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8003e980 con 0x7f5c88154ac0 2023-08-11T15:52:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.140+0000 7f5c8efa4640 1 -- 172.21.15.27:0/2488287242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c880c2090 con 0x7f5c88154ac0 2023-08-11T15:52:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.140+0000 7f5c8efa4640 1 -- 172.21.15.27:0/2488287242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c880c2650 con 0x7f5c88154ac0 2023-08-11T15:52:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.142+0000 7f5c8efa4640 1 -- 172.21.15.27:0/2488287242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c880c2b70 con 0x7f5c88154ac0 2023-08-11T15:52:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.142+0000 7f5c857fa640 1 -- 172.21.15.27:0/2488287242 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5c8002cd50 con 0x7f5c88154ac0 2023-08-11T15:52:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.144+0000 7f5c857fa640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5c6c06a050 0x7f5c6c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.144+0000 7f5c857fa640 1 -- 172.21.15.27:0/2488287242 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5c800ab3d0 con 0x7f5c88154ac0 2023-08-11T15:52:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.144+0000 7f5c8cd40640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5c6c06a050 0x7f5c6c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.145+0000 7f5c8cd40640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5c6c06a050 0x7f5c6c06c510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c000f60 tx=0x7f5c7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.148+0000 7f5c857fa640 1 -- 172.21.15.27:0/2488287242 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f5c880c2b70 con 0x7f5c88154ac0 2023-08-11T15:52:15.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.344+0000 7f5c8efa4640 1 -- 172.21.15.27:0/2488287242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c880b3cf0 con 0x7f5c88154ac0 2023-08-11T15:52:15.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.345+0000 7f5c857fa640 1 -- 172.21.15.27:0/2488287242 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c80043020 con 0x7f5c88154ac0 2023-08-11T15:52:15.347 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:15.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 -- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5c6c06a050 msgr2=0x7f5c6c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5c6c06a050 0x7f5c6c06c510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c000f60 tx=0x7f5c7c009040 comp rx=0 tx=0).stop 2023-08-11T15:52:15.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 -- 172.21.15.27:0/2488287242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c88154ac0 msgr2=0x7f5c880ace70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c88154ac0 0x7f5c880ace70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5c80036040 tx=0x7f5c80004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:15.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 -- 172.21.15.27:0/2488287242 shutdown_connections 2023-08-11T15:52:15.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f5c6c06a050 0x7f5c6c06c510 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c88154ac0 0x7f5c880ace70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c881403a0 0x7f5c880ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 --2- 172.21.15.27:0/2488287242 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c880cb700 0x7f5c88153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 -- 172.21.15.27:0/2488287242 >> 172.21.15.27:0/2488287242 conn(0x7f5c88069bf0 msgr2=0x7f5c880c8710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:15.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 -- 172.21.15.27:0/2488287242 shutdown_connections 2023-08-11T15:52:15.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.348+0000 7f5c8efa4640 1 -- 172.21.15.27:0/2488287242 wait complete. 2023-08-11T15:52:15.369 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:15.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.614+0000 7f9349be3640 1 Processor -- start 2023-08-11T15:52:15.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.614+0000 7f9349be3640 1 -- start start 2023-08-11T15:52:15.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.616+0000 7f9349be3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93441403a0 0x7f93440b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.616+0000 7f9349be3640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9344140ba0 con 0x7f93440cb700 2023-08-11T15:52:15.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.616+0000 7f9349be3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934415b220 con 0x7f9344154ac0 2023-08-11T15:52:15.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.616+0000 7f9349be3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9344071b10 con 0x7f93441403a0 2023-08-11T15:52:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.616+0000 7f9342ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f93440cb700 0x7f934413fc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42000/0 (socket says 172.21.15.27:42000) 2023-08-11T15:52:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.617+0000 7f9342ffd640 1 -- 172.21.15.27:0/384384746 learned_addr learned my addr 172.21.15.27:0/384384746 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.617+0000 7f9343fff640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93441403a0 0x7f93440b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.617+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152115075 0 0) 0x7f9344140ba0 con 0x7f93440cb700 2023-08-11T15:52:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.617+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f932c003520 con 0x7f93440cb700 2023-08-11T15:52:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.617+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4028268685 0 0) 0x7f932c003520 con 0x7f93440cb700 2023-08-11T15:52:15.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.617+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93441403a0 msgr2=0x7f93440b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:15.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.617+0000 7f93427fc640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93441403a0 0x7f93440b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.617+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 >> v1:172.21.15.119:6789/0 conn(0x7f9344154ac0 legacy=0x7f93440caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93441408c0 con 0x7f93440cb700 2023-08-11T15:52:15.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f9338001670 con 0x7f93440cb700 2023-08-11T15:52:15.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 >> v1:172.21.15.27:6789/0 conn(0x7f93440cb700 legacy=0x7f934413fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f93427fc640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f932c0039a0 0x7f932c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f93427fc640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93441403a0 0x7f932c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f93427fc640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f932c004d00 0x7f932c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9344071b10 con 0x7f932c004d00 2023-08-11T15:52:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c00bae0 con 0x7f932c0039a0 2023-08-11T15:52:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c00bca0 con 0x7f93441403a0 2023-08-11T15:52:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f9343fff640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f932c004d00 0x7f932c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f93437fe640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f932c0039a0 0x7f932c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.618+0000 7f9342ffd640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93441403a0 0x7f932c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.619+0000 7f9343fff640 1 -- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93441403a0 msgr2=0x7f932c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:15.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.619+0000 7f9343fff640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93441403a0 0x7f932c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.619+0000 7f9343fff640 1 -- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f932c0039a0 msgr2=0x7f932c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:15.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.619+0000 7f9343fff640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f932c0039a0 0x7f932c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.619+0000 7f9343fff640 1 -- 172.21.15.27:0/384384746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c003520 con 0x7f932c004d00 2023-08-11T15:52:15.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.620+0000 7f9343fff640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f932c004d00 0x7f932c005180 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f933c009f90 tx=0x7f932c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad5864a4572b747 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:15.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.620+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c02cbd0 con 0x7f932c004d00 2023-08-11T15:52:15.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.620+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c00f040 con 0x7f932c004d00 2023-08-11T15:52:15.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.620+0000 7f93427fc640 1 -- 172.21.15.27:0/384384746 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c034440 con 0x7f932c004d00 2023-08-11T15:52:15.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.621+0000 7f9349be3640 1 -- 172.21.15.27:0/384384746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f932c004d00 msgr2=0x7f932c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.621+0000 7f9349be3640 1 --2- 172.21.15.27:0/384384746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f932c004d00 0x7f932c005180 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f933c009f90 tx=0x7f932c0057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.621+0000 7f9343fff640 1 -- 172.21.15.27:0/384384746 reap_dead start 2023-08-11T15:52:15.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.621+0000 7f9349be3640 1 -- 172.21.15.27:0/384384746 shutdown_connections 2023-08-11T15:52:15.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.621+0000 7f9349be3640 1 -- 172.21.15.27:0/384384746 >> 172.21.15.27:0/384384746 conn(0x7f9344069bf0 msgr2=0x7f934406a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:15.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.621+0000 7f9349be3640 1 -- 172.21.15.27:0/384384746 shutdown_connections 2023-08-11T15:52:15.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.621+0000 7f9349be3640 1 -- 172.21.15.27:0/384384746 wait complete. 2023-08-11T15:52:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.623+0000 7f9349be3640 1 Processor -- start 2023-08-11T15:52:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.623+0000 7f9349be3640 1 -- start start 2023-08-11T15:52:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.624+0000 7f9349be3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9344150b50 0x7f93440acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.624+0000 7f9349be3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934415a5a0 0x7f93440ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.624+0000 7f9349be3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93440b38a0 0x7f93440c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.624+0000 7f9349be3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440acb20 con 0x7f934415a5a0 2023-08-11T15:52:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.624+0000 7f9349be3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440c8cd0 con 0x7f9344150b50 2023-08-11T15:52:15.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.624+0000 7f9349be3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93440b0960 con 0x7f93440b38a0 2023-08-11T15:52:15.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.624+0000 7f9342ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934415a5a0 0x7f93440ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.625+0000 7f9342ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934415a5a0 0x7f93440ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41024/0 (socket says 172.21.15.27:41024) 2023-08-11T15:52:15.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.625+0000 7f9342ffd640 1 -- 172.21.15.27:0/2298149814 learned_addr learned my addr 172.21.15.27:0/2298149814 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:15.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.625+0000 7f93437fe640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9344150b50 0x7f93440acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.625+0000 7f9343fff640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93440b38a0 0x7f93440c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.625+0000 7f9342ffd640 1 -- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93440b38a0 msgr2=0x7f93440c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.625+0000 7f9342ffd640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93440b38a0 0x7f93440c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.625+0000 7f9342ffd640 1 -- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9344150b50 msgr2=0x7f93440acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.625+0000 7f9342ffd640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9344150b50 0x7f93440acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.625+0000 7f9342ffd640 1 -- 172.21.15.27:0/2298149814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440c8710 con 0x7f934415a5a0 2023-08-11T15:52:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.625+0000 7f9342ffd640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934415a5a0 0x7f93440ad480 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f93380011e0 tx=0x7f9338007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.626+0000 7f932bfff640 1 -- 172.21.15.27:0/2298149814 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800f9c0 con 0x7f934415a5a0 2023-08-11T15:52:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.626+0000 7f932bfff640 1 -- 172.21.15.27:0/2298149814 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933800fb20 con 0x7f934415a5a0 2023-08-11T15:52:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.626+0000 7f932bfff640 1 -- 172.21.15.27:0/2298149814 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800fed0 con 0x7f934415a5a0 2023-08-11T15:52:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.626+0000 7f9349be3640 1 -- 172.21.15.27:0/2298149814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93440c89f0 con 0x7f934415a5a0 2023-08-11T15:52:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.626+0000 7f9349be3640 1 -- 172.21.15.27:0/2298149814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93440b05f0 con 0x7f934415a5a0 2023-08-11T15:52:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.627+0000 7f9349be3640 1 -- 172.21.15.27:0/2298149814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93440acc60 con 0x7f934415a5a0 2023-08-11T15:52:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.628+0000 7f932bfff640 1 -- 172.21.15.27:0/2298149814 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f933801d460 con 0x7f934415a5a0 2023-08-11T15:52:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.629+0000 7f932bfff640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f933006a050 0x7f933006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.629+0000 7f932bfff640 1 -- 172.21.15.27:0/2298149814 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9338088b10 con 0x7f934415a5a0 2023-08-11T15:52:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.630+0000 7f93437fe640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f933006a050 0x7f933006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.631+0000 7f93437fe640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f933006a050 0x7f933006c510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f93340035f0 tx=0x7f9334009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:15.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.633+0000 7f932bfff640 1 -- 172.21.15.27:0/2298149814 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f93440acc60 con 0x7f934415a5a0 2023-08-11T15:52:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.829+0000 7f9349be3640 1 -- 172.21.15.27:0/2298149814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f93440b0ad0 con 0x7f934415a5a0 2023-08-11T15:52:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.830+0000 7f932bfff640 1 -- 172.21.15.27:0/2298149814 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9338053ec0 con 0x7f934415a5a0 2023-08-11T15:52:15.832 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 -- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f933006a050 msgr2=0x7f933006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f933006a050 0x7f933006c510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f93340035f0 tx=0x7f9334009040 comp rx=0 tx=0).stop 2023-08-11T15:52:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 -- 172.21.15.27:0/2298149814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934415a5a0 msgr2=0x7f93440ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934415a5a0 0x7f93440ad480 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f93380011e0 tx=0x7f9338007720 comp rx=0 tx=0).stop 2023-08-11T15:52:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 -- 172.21.15.27:0/2298149814 shutdown_connections 2023-08-11T15:52:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f933006a050 0x7f933006c510 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93440b38a0 0x7f93440c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934415a5a0 0x7f93440ad480 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 --2- 172.21.15.27:0/2298149814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9344150b50 0x7f93440acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 -- 172.21.15.27:0/2298149814 >> 172.21.15.27:0/2298149814 conn(0x7f9344069bf0 msgr2=0x7f934415c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 -- 172.21.15.27:0/2298149814 shutdown_connections 2023-08-11T15:52:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:15.833+0000 7f9349be3640 1 -- 172.21.15.27:0/2298149814 wait complete. 2023-08-11T15:52:15.856 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:16.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.099+0000 7f0d8ffff640 1 Processor -- start 2023-08-11T15:52:16.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.099+0000 7f0d8ffff640 1 -- start start 2023-08-11T15:52:16.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.100+0000 7f0d8ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d9015c5b0 0x7f0d900d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.101+0000 7f0d8ffff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d900cf310 0x7f0d900b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.101+0000 7f0d8ffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d900d0dc0 con 0x7f0d900d0790 2023-08-11T15:52:16.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.101+0000 7f0d8ffff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d900d10a0 con 0x7f0d9015c5b0 2023-08-11T15:52:16.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.101+0000 7f0d8ffff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900d1380 con 0x7f0d900cf310 2023-08-11T15:52:16.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.101+0000 7f0d8e7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0d900d0790 0x7f0d900cec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42016/0 (socket says 172.21.15.27:42016) 2023-08-11T15:52:16.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.101+0000 7f0d8e7fc640 1 -- 172.21.15.27:0/311028103 learned_addr learned my addr 172.21.15.27:0/311028103 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:16.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.101+0000 7f0d8f7fe640 1 --2- 172.21.15.27:0/311028103 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d900cf310 0x7f0d900b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.101+0000 7f0d8effd640 1 --2- 172.21.15.27:0/311028103 >> v2:172.21.15.119:3300/0 conn(0x7f0d9015c5b0 0x7f0d900d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.102+0000 7f0d8dffb640 1 -- 172.21.15.27:0/311028103 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365310423 0 0) 0x7f0d900d0dc0 con 0x7f0d900d0790 2023-08-11T15:52:16.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.102+0000 7f0d8dffb640 1 -- 172.21.15.27:0/311028103 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d7c003520 con 0x7f0d900d0790 2023-08-11T15:52:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.102+0000 7f0d8f7fe640 1 -- 172.21.15.27:0/311028103 >> v2:172.21.15.119:3300/0 conn(0x7f0d9015c5b0 msgr2=0x7f0d900d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.102+0000 7f0d8f7fe640 1 --2- 172.21.15.27:0/311028103 >> v2:172.21.15.119:3300/0 conn(0x7f0d9015c5b0 0x7f0d900d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.102+0000 7f0d8f7fe640 1 -- 172.21.15.27:0/311028103 >> v1:172.21.15.27:6789/0 conn(0x7f0d900d0790 legacy=0x7f0d900cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.102+0000 7f0d8dffb640 1 -- 172.21.15.27:0/311028103 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1241198989 0 0) 0x7f0d7c003520 con 0x7f0d900d0790 2023-08-11T15:52:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.102+0000 7f0d8f7fe640 1 -- 172.21.15.27:0/311028103 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900d1660 con 0x7f0d900cf310 2023-08-11T15:52:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.102+0000 7f0d8effd640 1 --2- 172.21.15.27:0/311028103 >> v2:172.21.15.119:3300/0 conn(0x7f0d9015c5b0 0x7f0d900d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.104+0000 7f0d8f7fe640 1 --2- 172.21.15.27:0/311028103 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d900cf310 0x7f0d900b51f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0d880097a0 tx=0x7f0d8802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9daca64ded0469d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.104+0000 7f0d8dffb640 1 -- 172.21.15.27:0/311028103 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8802f050 con 0x7f0d900cf310 2023-08-11T15:52:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.104+0000 7f0d8dffb640 1 -- 172.21.15.27:0/311028103 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d88031500 con 0x7f0d900cf310 2023-08-11T15:52:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.104+0000 7f0d8dffb640 1 -- 172.21.15.27:0/311028103 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780019f0 con 0x7f0d900cf310 2023-08-11T15:52:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.105+0000 7f0d8ffff640 1 -- 172.21.15.27:0/311028103 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d900cf310 msgr2=0x7f0d900b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.105+0000 7f0d8ffff640 1 --2- 172.21.15.27:0/311028103 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d900cf310 0x7f0d900b51f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0d880097a0 tx=0x7f0d8802b820 comp rx=0 tx=0).stop 2023-08-11T15:52:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.105+0000 7f0d8ffff640 1 -- 172.21.15.27:0/311028103 shutdown_connections 2023-08-11T15:52:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.105+0000 7f0d8ffff640 1 --2- 172.21.15.27:0/311028103 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d900cf310 0x7f0d900b51f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.105+0000 7f0d8ffff640 1 --2- 172.21.15.27:0/311028103 >> v2:172.21.15.119:3300/0 conn(0x7f0d9015c5b0 0x7f0d900d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.105+0000 7f0d8ffff640 1 -- 172.21.15.27:0/311028103 >> 172.21.15.27:0/311028103 conn(0x7f0d9006a270 msgr2=0x7f0d9006a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.105+0000 7f0d8ffff640 1 -- 172.21.15.27:0/311028103 shutdown_connections 2023-08-11T15:52:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.105+0000 7f0d8ffff640 1 -- 172.21.15.27:0/311028103 wait complete. 2023-08-11T15:52:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.107+0000 7f0d8ffff640 1 Processor -- start 2023-08-11T15:52:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.107+0000 7f0d8ffff640 1 -- start start 2023-08-11T15:52:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.108+0000 7f0d8ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d900cf310 0x7f0d901515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.108+0000 7f0d8ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d900d0790 0x7f0d90144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.108+0000 7f0d8ffff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d9015c5b0 0x7f0d901f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.108+0000 7f0d8ffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d90144860 con 0x7f0d900d0790 2023-08-11T15:52:16.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.108+0000 7f0d8ffff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900caf20 con 0x7f0d900cf310 2023-08-11T15:52:16.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.108+0000 7f0d8ffff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900ca350 con 0x7f0d9015c5b0 2023-08-11T15:52:16.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.108+0000 7f0d8e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d900d0790 0x7f0d90144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.108+0000 7f0d8e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d900d0790 0x7f0d90144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41036/0 (socket says 172.21.15.27:41036) 2023-08-11T15:52:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.108+0000 7f0d8e7fc640 1 -- 172.21.15.27:0/2938642314 learned_addr learned my addr 172.21.15.27:0/2938642314 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d8f7fe640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d9015c5b0 0x7f0d901f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d8effd640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d900cf310 0x7f0d901515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d8e7fc640 1 -- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d9015c5b0 msgr2=0x7f0d901f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d8e7fc640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d9015c5b0 0x7f0d901f15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d8e7fc640 1 -- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d900cf310 msgr2=0x7f0d901515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d8e7fc640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d900cf310 0x7f0d901515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d8e7fc640 1 -- 172.21.15.27:0/2938642314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80009400 con 0x7f0d900d0790 2023-08-11T15:52:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d8effd640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d900cf310 0x7f0d901515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d8e7fc640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d900d0790 0x7f0d90144320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0d78001ff0 tx=0x7f0d7800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d877fe640 1 -- 172.21.15.27:0/2938642314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78012400 con 0x7f0d900d0790 2023-08-11T15:52:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d877fe640 1 -- 172.21.15.27:0/2938642314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d78012560 con 0x7f0d900d0790 2023-08-11T15:52:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d877fe640 1 -- 172.21.15.27:0/2938642314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780128f0 con 0x7f0d900d0790 2023-08-11T15:52:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.109+0000 7f0d8ffff640 1 -- 172.21.15.27:0/2938642314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d88009400 con 0x7f0d900d0790 2023-08-11T15:52:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.110+0000 7f0d8ffff640 1 -- 172.21.15.27:0/2938642314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d900ca8d0 con 0x7f0d900d0790 2023-08-11T15:52:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.111+0000 7f0d8ffff640 1 -- 172.21.15.27:0/2938642314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d900cb0d0 con 0x7f0d900d0790 2023-08-11T15:52:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.112+0000 7f0d877fe640 1 -- 172.21.15.27:0/2938642314 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0d78012a50 con 0x7f0d900d0790 2023-08-11T15:52:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.113+0000 7f0d877fe640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0d7006a050 0x7f0d7006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.113+0000 7f0d877fe640 1 -- 172.21.15.27:0/2938642314 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0d780890b0 con 0x7f0d900d0790 2023-08-11T15:52:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.114+0000 7f0d8effd640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0d7006a050 0x7f0d7006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.115+0000 7f0d8effd640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0d7006a050 0x7f0d7006c510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009c50 tx=0x7f0d80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.118+0000 7f0d877fe640 1 -- 172.21.15.27:0/2938642314 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0d900cb0d0 con 0x7f0d900d0790 2023-08-11T15:52:16.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.312+0000 7f0d8ffff640 1 -- 172.21.15.27:0/2938642314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d900cb090 con 0x7f0d900d0790 2023-08-11T15:52:16.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.313+0000 7f0d877fe640 1 -- 172.21.15.27:0/2938642314 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d78054460 con 0x7f0d900d0790 2023-08-11T15:52:16.315 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:16.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.315+0000 7f0d8ffff640 1 -- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0d7006a050 msgr2=0x7f0d7006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.315+0000 7f0d8ffff640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0d7006a050 0x7f0d7006c510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009c50 tx=0x7f0d80008040 comp rx=0 tx=0).stop 2023-08-11T15:52:16.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.315+0000 7f0d8ffff640 1 -- 172.21.15.27:0/2938642314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d900d0790 msgr2=0x7f0d90144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.315+0000 7f0d8ffff640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d900d0790 0x7f0d90144320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0d78001ff0 tx=0x7f0d7800a430 comp rx=0 tx=0).stop 2023-08-11T15:52:16.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.315+0000 7f0d8ffff640 1 -- 172.21.15.27:0/2938642314 shutdown_connections 2023-08-11T15:52:16.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.315+0000 7f0d8ffff640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0d7006a050 0x7f0d7006c510 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.315+0000 7f0d8ffff640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d9015c5b0 0x7f0d901f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.315+0000 7f0d8ffff640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d900d0790 0x7f0d90144320 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.315+0000 7f0d8ffff640 1 --2- 172.21.15.27:0/2938642314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d900cf310 0x7f0d901515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.316+0000 7f0d8ffff640 1 -- 172.21.15.27:0/2938642314 >> 172.21.15.27:0/2938642314 conn(0x7f0d9006a270 msgr2=0x7f0d901599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:16.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.316+0000 7f0d8ffff640 1 -- 172.21.15.27:0/2938642314 shutdown_connections 2023-08-11T15:52:16.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.316+0000 7f0d8ffff640 1 -- 172.21.15.27:0/2938642314 wait complete. 2023-08-11T15:52:16.337 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:16.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.567+0000 7fb77ffff640 1 Processor -- start 2023-08-11T15:52:16.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.568+0000 7fb77ffff640 1 -- start start 2023-08-11T15:52:16.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.569+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7800c86c0 0x7fb7800afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.569+0000 7fb77ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb7800afff0 0x7fb78006ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.569+0000 7fb77ffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7801600f0 con 0x7fb78015c3f0 2023-08-11T15:52:16.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.569+0000 7fb77ffff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800ca810 con 0x7fb7800afff0 2023-08-11T15:52:16.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.569+0000 7fb77ffff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800b0460 con 0x7fb7800c86c0 2023-08-11T15:52:16.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.569+0000 7fb77effd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb78015c3f0 0x7fb7800cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42020/0 (socket says 172.21.15.27:42020) 2023-08-11T15:52:16.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.569+0000 7fb77effd640 1 -- 172.21.15.27:0/649705855 learned_addr learned my addr 172.21.15.27:0/649705855 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77f7fe640 1 --2- 172.21.15.27:0/649705855 >> v2:172.21.15.119:3300/0 conn(0x7fb7800afff0 0x7fb78006ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7800c86c0 0x7fb7800afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118298478 0 0) 0x7fb7801600f0 con 0x7fb78015c3f0 2023-08-11T15:52:16.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb76c003520 con 0x7fb78015c3f0 2023-08-11T15:52:16.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1568173899 0 0) 0x7fb76c003520 con 0x7fb78015c3f0 2023-08-11T15:52:16.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7800c86c0 msgr2=0x7fb7800afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:16.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77dffb640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7800c86c0 0x7fb7800afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 >> v2:172.21.15.119:3300/0 conn(0x7fb7800afff0 msgr2=0x7fb78006ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77dffb640 1 --2- 172.21.15.27:0/649705855 >> v2:172.21.15.119:3300/0 conn(0x7fb7800afff0 0x7fb78006ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7801603d0 con 0x7fb78015c3f0 2023-08-11T15:52:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fb770001a10 con 0x7fb78015c3f0 2023-08-11T15:52:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.570+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 >> v1:172.21.15.27:6789/0 conn(0x7fb78015c3f0 legacy=0x7fb7800cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.571+0000 7fb77dffb640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb76c0039a0 0x7fb76c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.571+0000 7fb77dffb640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb76c0047c0 0x7fb76c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.571+0000 7fb77dffb640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7800c86c0 0x7fb76c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.571+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800ca810 con 0x7fb76c0039a0 2023-08-11T15:52:16.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.571+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800b0460 con 0x7fb76c0047c0 2023-08-11T15:52:16.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.571+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c00b9b0 con 0x7fb7800c86c0 2023-08-11T15:52:16.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.571+0000 7fb77effd640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb76c0039a0 0x7fb76c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.571+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb76c0047c0 0x7fb76c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.571+0000 7fb77f7fe640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7800c86c0 0x7fb76c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.572+0000 7fb77effd640 1 -- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7800c86c0 msgr2=0x7fb76c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.572+0000 7fb77effd640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7800c86c0 0x7fb76c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.572+0000 7fb77effd640 1 -- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb76c0047c0 msgr2=0x7fb76c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.572+0000 7fb77effd640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb76c0047c0 0x7fb76c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.572+0000 7fb77effd640 1 -- 172.21.15.27:0/649705855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c003520 con 0x7fb76c0039a0 2023-08-11T15:52:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.572+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb76c0047c0 0x7fb76c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.573+0000 7fb77effd640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb76c0039a0 0x7fb76c003dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb77000a810 tx=0x7fb76c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2964536e4de9fcf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.574+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770034630 con 0x7fb76c0039a0 2023-08-11T15:52:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.574+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77002f070 con 0x7fb76c0039a0 2023-08-11T15:52:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.574+0000 7fb77dffb640 1 -- 172.21.15.27:0/649705855 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7700349d0 con 0x7fb76c0039a0 2023-08-11T15:52:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.574+0000 7fb77ffff640 1 -- 172.21.15.27:0/649705855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb76c0039a0 msgr2=0x7fb76c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.574+0000 7fb77ffff640 1 --2- 172.21.15.27:0/649705855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb76c0039a0 0x7fb76c003dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb77000a810 tx=0x7fb76c0057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.574+0000 7fb77f7fe640 1 -- 172.21.15.27:0/649705855 reap_dead start 2023-08-11T15:52:16.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.574+0000 7fb77ffff640 1 -- 172.21.15.27:0/649705855 shutdown_connections 2023-08-11T15:52:16.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.575+0000 7fb77ffff640 1 -- 172.21.15.27:0/649705855 >> 172.21.15.27:0/649705855 conn(0x7fb7800caab0 msgr2=0x7fb7800cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:16.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.575+0000 7fb77ffff640 1 -- 172.21.15.27:0/649705855 shutdown_connections 2023-08-11T15:52:16.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.575+0000 7fb77ffff640 1 -- 172.21.15.27:0/649705855 wait complete. 2023-08-11T15:52:16.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.576+0000 7fb77ffff640 1 Processor -- start 2023-08-11T15:52:16.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.577+0000 7fb77ffff640 1 -- start start 2023-08-11T15:52:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780159510 0x7fb78015f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7801598f0 0x7fb7801f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7801f25c0 0x7fb7801f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77ffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78015c880 con 0x7fb7801598f0 2023-08-11T15:52:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77ffff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78015cb60 con 0x7fb780159510 2023-08-11T15:52:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77ffff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7801f2a70 con 0x7fb7801f25c0 2023-08-11T15:52:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7801598f0 0x7fb7801f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7801598f0 0x7fb7801f2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41048/0 (socket says 172.21.15.27:41048) 2023-08-11T15:52:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77e7fc640 1 -- 172.21.15.27:0/86157198 learned_addr learned my addr 172.21.15.27:0/86157198 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77e7fc640 1 -- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7801f25c0 msgr2=0x7fb7801f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7801f25c0 0x7fb7801f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77effd640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780159510 0x7fb78015f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77e7fc640 1 -- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780159510 msgr2=0x7fb78015f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.578+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780159510 0x7fb78015f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.579+0000 7fb77e7fc640 1 -- 172.21.15.27:0/86157198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7800c78f0 con 0x7fb7801598f0 2023-08-11T15:52:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.579+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7801598f0 0x7fb7801f2080 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb7680114f0 tx=0x7fb7680119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.579+0000 7fb777fff640 1 -- 172.21.15.27:0/86157198 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768015870 con 0x7fb7801598f0 2023-08-11T15:52:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.579+0000 7fb777fff640 1 -- 172.21.15.27:0/86157198 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7680159d0 con 0x7fb7801598f0 2023-08-11T15:52:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.579+0000 7fb777fff640 1 -- 172.21.15.27:0/86157198 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768015d40 con 0x7fb7801598f0 2023-08-11T15:52:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.579+0000 7fb77ffff640 1 -- 172.21.15.27:0/86157198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7800c7bd0 con 0x7fb7801598f0 2023-08-11T15:52:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.579+0000 7fb77ffff640 1 -- 172.21.15.27:0/86157198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78015f730 con 0x7fb7801598f0 2023-08-11T15:52:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.580+0000 7fb77ffff640 1 -- 172.21.15.27:0/86157198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78004e420 con 0x7fb7801598f0 2023-08-11T15:52:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.581+0000 7fb777fff640 1 -- 172.21.15.27:0/86157198 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb76801d400 con 0x7fb7801598f0 2023-08-11T15:52:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.582+0000 7fb777fff640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb76006a080 0x7fb76006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.583+0000 7fb777fff640 1 -- 172.21.15.27:0/86157198 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb768089100 con 0x7fb7801598f0 2023-08-11T15:52:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.583+0000 7fb77effd640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb76006a080 0x7fb76006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.584+0000 7fb77effd640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb76006a080 0x7fb76006c540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb770034000 tx=0x7fb77002f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.586+0000 7fb777fff640 1 -- 172.21.15.27:0/86157198 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fb768054420 con 0x7fb7801598f0 2023-08-11T15:52:16.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.784+0000 7fb77ffff640 1 -- 172.21.15.27:0/86157198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb78015cca0 con 0x7fb7801598f0 2023-08-11T15:52:16.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.785+0000 7fb777fff640 1 -- 172.21.15.27:0/86157198 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb768057a40 con 0x7fb7801598f0 2023-08-11T15:52:16.787 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:16.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 -- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb76006a080 msgr2=0x7fb76006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb76006a080 0x7fb76006c540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb770034000 tx=0x7fb77002f000 comp rx=0 tx=0).stop 2023-08-11T15:52:16.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 -- 172.21.15.27:0/86157198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7801598f0 msgr2=0x7fb7801f2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:16.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7801598f0 0x7fb7801f2080 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb7680114f0 tx=0x7fb7680119b0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 -- 172.21.15.27:0/86157198 shutdown_connections 2023-08-11T15:52:16.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb76006a080 0x7fb76006c540 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7801f25c0 0x7fb7801f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7801598f0 0x7fb7801f2080 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 --2- 172.21.15.27:0/86157198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780159510 0x7fb78015f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 -- 172.21.15.27:0/86157198 >> 172.21.15.27:0/86157198 conn(0x7fb7800caab0 msgr2=0x7fb7800ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 -- 172.21.15.27:0/86157198 shutdown_connections 2023-08-11T15:52:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:16.788+0000 7fb77ffff640 1 -- 172.21.15.27:0/86157198 wait complete. 2023-08-11T15:52:16.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.053+0000 7fd9e2dcf640 1 Processor -- start 2023-08-11T15:52:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.053+0000 7fd9e2dcf640 1 -- start start 2023-08-11T15:52:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.055+0000 7fd9e2dcf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd9dc0b3090 0x7fd9dc0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.055+0000 7fd9e2dcf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc0af250 con 0x7fd9dc0b38f0 2023-08-11T15:52:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.055+0000 7fd9e2dcf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc0af510 con 0x7fd9dc161210 2023-08-11T15:52:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.055+0000 7fd9e2dcf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9dc0b0190 con 0x7fd9dc0b3090 2023-08-11T15:52:17.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.055+0000 7fd9e0b6b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd9dc0b38f0 0x7fd9dc160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42024/0 (socket says 172.21.15.27:42024) 2023-08-11T15:52:17.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.055+0000 7fd9e0b6b640 1 -- 172.21.15.27:0/3601771913 learned_addr learned my addr 172.21.15.27:0/3601771913 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.055+0000 7fd9d3fff640 1 --2- 172.21.15.27:0/3601771913 >> v2:172.21.15.119:3300/0 conn(0x7fd9dc0b3090 0x7fd9dc0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376141778 0 0) 0x7fd9dc0af250 con 0x7fd9dc0b38f0 2023-08-11T15:52:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c4003520 con 0x7fd9dc0b38f0 2023-08-11T15:52:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 746650806 0 0) 0x7fd9c4003520 con 0x7fd9dc0b38f0 2023-08-11T15:52:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 >> v2:172.21.15.119:3300/0 conn(0x7fd9dc0b3090 msgr2=0x7fd9dc0d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 --2- 172.21.15.27:0/3601771913 >> v2:172.21.15.119:3300/0 conn(0x7fd9dc0b3090 0x7fd9dc0d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 >> v1:172.21.15.119:6789/0 conn(0x7fd9dc161210 legacy=0x7fd9dc0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc0af7f0 con 0x7fd9dc0b38f0 2023-08-11T15:52:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fd9d80017f0 con 0x7fd9dc0b38f0 2023-08-11T15:52:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 >> v1:172.21.15.27:6789/0 conn(0x7fd9dc0b38f0 legacy=0x7fd9dc160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9c40039c0 0x7fd9c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c40047c0 0x7fd9c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9c4005180 0x7fd9c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0b0190 con 0x7fd9c4005180 2023-08-11T15:52:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c400c050 con 0x7fd9c40047c0 2023-08-11T15:52:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c400c210 con 0x7fd9c40039c0 2023-08-11T15:52:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.056+0000 7fd9e136c640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9c4005180 0x7fd9c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.057+0000 7fd9e0b6b640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9c40039c0 0x7fd9c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.057+0000 7fd9e136c640 1 -- 172.21.15.27:0/3601771913 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9c40039c0 msgr2=0x7fd9c4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.057+0000 7fd9e136c640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9c40039c0 0x7fd9c4003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.057+0000 7fd9e136c640 1 -- 172.21.15.27:0/3601771913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c40047c0 msgr2=0x7fd9c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.057+0000 7fd9d3fff640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c40047c0 0x7fd9c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.057+0000 7fd9e136c640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c40047c0 0x7fd9c4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.057+0000 7fd9e136c640 1 -- 172.21.15.27:0/3601771913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4003520 con 0x7fd9c4005180 2023-08-11T15:52:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.058+0000 7fd9e136c640 1 -- 172.21.15.27:0/3601771913 reap_dead start 2023-08-11T15:52:17.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.058+0000 7fd9e136c640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9c4005180 0x7fd9c400b980 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4004480 tx=0x7fd9c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c475e5e11edbc9b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.059+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4030400 con 0x7fd9c4005180 2023-08-11T15:52:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.059+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4030560 con 0x7fd9c4005180 2023-08-11T15:52:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.059+0000 7fd9d37fe640 1 -- 172.21.15.27:0/3601771913 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d40308f0 con 0x7fd9c4005180 2023-08-11T15:52:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.059+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/3601771913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9c4005180 msgr2=0x7fd9c400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.059+0000 7fd9e2dcf640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9c4005180 0x7fd9c400b980 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4004480 tx=0x7fd9c40057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.059+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/3601771913 shutdown_connections 2023-08-11T15:52:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.059+0000 7fd9e2dcf640 1 --2- 172.21.15.27:0/3601771913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9c4005180 0x7fd9c400b980 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.060+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/3601771913 >> 172.21.15.27:0/3601771913 conn(0x7fd9dc09b960 msgr2=0x7fd9dc0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.060+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/3601771913 shutdown_connections 2023-08-11T15:52:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.060+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/3601771913 wait complete. 2023-08-11T15:52:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.061+0000 7fd9e2dcf640 1 Processor -- start 2023-08-11T15:52:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.062+0000 7fd9e2dcf640 1 -- start start 2023-08-11T15:52:17.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.062+0000 7fd9e2dcf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9c4005180 0x7fd9dc0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.062+0000 7fd9e2dcf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9dc157c30 0x7fd9dc158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.062+0000 7fd9e2dcf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc0cf0a0 0x7fd9dc0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9e2dcf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0d0090 con 0x7fd9dc157c30 2023-08-11T15:52:17.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9e2dcf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc15ba80 con 0x7fd9dc0cf0a0 2023-08-11T15:52:17.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9e2dcf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0cfdb0 con 0x7fd9c4005180 2023-08-11T15:52:17.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9d3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9dc157c30 0x7fd9dc158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9d3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9dc157c30 0x7fd9dc158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41058/0 (socket says 172.21.15.27:41058) 2023-08-11T15:52:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9d3fff640 1 -- 172.21.15.27:0/1659929490 learned_addr learned my addr 172.21.15.27:0/1659929490 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9e136c640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc0cf0a0 0x7fd9dc0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9e0b6b640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9c4005180 0x7fd9dc0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9d3fff640 1 -- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9c4005180 msgr2=0x7fd9dc0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9d3fff640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9c4005180 0x7fd9dc0c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9d3fff640 1 -- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc0cf0a0 msgr2=0x7fd9dc0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9d3fff640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc0cf0a0 0x7fd9dc0cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9d3fff640 1 -- 172.21.15.27:0/1659929490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc158a10 con 0x7fd9dc157c30 2023-08-11T15:52:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.063+0000 7fd9e136c640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc0cf0a0 0x7fd9dc0cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.064+0000 7fd9d3fff640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9dc157c30 0x7fd9dc158090 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc010860 tx=0x7fd9cc010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.064+0000 7fd9d17fa640 1 -- 172.21.15.27:0/1659929490 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc009610 con 0x7fd9dc157c30 2023-08-11T15:52:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.064+0000 7fd9d17fa640 1 -- 172.21.15.27:0/1659929490 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc00eb60 con 0x7fd9dc157c30 2023-08-11T15:52:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.064+0000 7fd9d17fa640 1 -- 172.21.15.27:0/1659929490 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc01c400 con 0x7fd9dc157c30 2023-08-11T15:52:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.064+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/1659929490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc158cf0 con 0x7fd9dc157c30 2023-08-11T15:52:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.064+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/1659929490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc1585d0 con 0x7fd9dc157c30 2023-08-11T15:52:17.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.065+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/1659929490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc158870 con 0x7fd9dc157c30 2023-08-11T15:52:17.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.066+0000 7fd9d17fa640 1 -- 172.21.15.27:0/1659929490 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd9cc01c560 con 0x7fd9dc157c30 2023-08-11T15:52:17.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.067+0000 7fd9d17fa640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd9c806a050 0x7fd9c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.067+0000 7fd9d17fa640 1 -- 172.21.15.27:0/1659929490 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd9cc088ef0 con 0x7fd9dc157c30 2023-08-11T15:52:17.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.068+0000 7fd9e0b6b640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd9c806a050 0x7fd9c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.069+0000 7fd9e0b6b640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd9c806a050 0x7fd9c806c510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8000c00 tx=0x7fd9d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:17.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.071+0000 7fd9d17fa640 1 -- 172.21.15.27:0/1659929490 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fd9dc158870 con 0x7fd9dc157c30 2023-08-11T15:52:17.141 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:52:17.150 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:52:17.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.267+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/1659929490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd9dc158870 con 0x7fd9dc157c30 2023-08-11T15:52:17.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.268+0000 7fd9d17fa640 1 -- 172.21.15.27:0/1659929490 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd9dc158870 con 0x7fd9dc157c30 2023-08-11T15:52:17.270 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:17.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.270+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd9c806a050 msgr2=0x7fd9c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.270+0000 7fd9e2dcf640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd9c806a050 0x7fd9c806c510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8000c00 tx=0x7fd9d8009040 comp rx=0 tx=0).stop 2023-08-11T15:52:17.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.270+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/1659929490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9dc157c30 msgr2=0x7fd9dc158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.270+0000 7fd9e2dcf640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9dc157c30 0x7fd9dc158090 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc010860 tx=0x7fd9cc010d20 comp rx=0 tx=0).stop 2023-08-11T15:52:17.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.271+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/1659929490 shutdown_connections 2023-08-11T15:52:17.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.271+0000 7fd9e2dcf640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd9c806a050 0x7fd9c806c510 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.271+0000 7fd9e2dcf640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9dc0cf0a0 0x7fd9dc0cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.271+0000 7fd9e2dcf640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9dc157c30 0x7fd9dc158090 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.271+0000 7fd9e2dcf640 1 --2- 172.21.15.27:0/1659929490 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9c4005180 0x7fd9dc0c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.271+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/1659929490 >> 172.21.15.27:0/1659929490 conn(0x7fd9dc09b960 msgr2=0x7fd9dc15e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:17.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.271+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/1659929490 shutdown_connections 2023-08-11T15:52:17.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.271+0000 7fd9e2dcf640 1 -- 172.21.15.27:0/1659929490 wait complete. 2023-08-11T15:52:17.290 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:52:17.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.576+0000 7f2b64bcf640 1 Processor -- start 2023-08-11T15:52:17.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.577+0000 7f2b64bcf640 1 -- start start 2023-08-11T15:52:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.578+0000 7f2b64bcf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b6015c570 0x7f2b600d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.578+0000 7f2b64bcf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2b600cd920 0x7f2b600d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.578+0000 7f2b64bcf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6006aa30 con 0x7f2b6015c950 2023-08-11T15:52:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.578+0000 7f2b64bcf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6006b2d0 con 0x7f2b6015c570 2023-08-11T15:52:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.578+0000 7f2b64bcf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b600ce090 con 0x7f2b600cd920 2023-08-11T15:52:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.579+0000 7f2b5e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b6015c570 0x7f2b600d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.579+0000 7f2b5dd9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2b6015c950 0x7f2b6006b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42026/0 (socket says 172.21.15.27:42026) 2023-08-11T15:52:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.579+0000 7f2b5dd9b640 1 -- 172.21.15.27:0/3417445793 learned_addr learned my addr 172.21.15.27:0/3417445793 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.579+0000 7f2b5ed9d640 1 --2- 172.21.15.27:0/3417445793 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2b600cd920 0x7f2b600d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.579+0000 7f2b5e59c640 1 -- 172.21.15.27:0/3417445793 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2b600cd920 msgr2=0x7f2b600d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:17.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.579+0000 7f2b5e59c640 1 --2- 172.21.15.27:0/3417445793 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2b600cd920 0x7f2b600d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.579+0000 7f2b5e59c640 1 -- 172.21.15.27:0/3417445793 >> v1:172.21.15.27:6789/0 conn(0x7f2b6015c950 legacy=0x7f2b6006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.579+0000 7f2b5e59c640 1 -- 172.21.15.27:0/3417445793 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6006ac80 con 0x7f2b6015c570 2023-08-11T15:52:17.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.580+0000 7f2b5e59c640 1 --2- 172.21.15.27:0/3417445793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 0x7f2b600d0e30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009790 tx=0x7f2b5002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e307cd8e9e771ef server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:17.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.581+0000 7f2b5d59a640 1 -- 172.21.15.27:0/3417445793 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50031780 con 0x7f2b6015c570 2023-08-11T15:52:17.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.581+0000 7f2b5d59a640 1 -- 172.21.15.27:0/3417445793 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b500318e0 con 0x7f2b6015c570 2023-08-11T15:52:17.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.581+0000 7f2b5d59a640 1 -- 172.21.15.27:0/3417445793 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50031c70 con 0x7f2b6015c570 2023-08-11T15:52:17.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.581+0000 7f2b64bcf640 1 -- 172.21.15.27:0/3417445793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 msgr2=0x7f2b600d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.581+0000 7f2b64bcf640 1 --2- 172.21.15.27:0/3417445793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 0x7f2b600d0e30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009790 tx=0x7f2b5002b820 comp rx=0 tx=0).stop 2023-08-11T15:52:17.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.582+0000 7f2b64bcf640 1 -- 172.21.15.27:0/3417445793 shutdown_connections 2023-08-11T15:52:17.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.582+0000 7f2b64bcf640 1 --2- 172.21.15.27:0/3417445793 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2b600cd920 0x7f2b600d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.582+0000 7f2b64bcf640 1 --2- 172.21.15.27:0/3417445793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 0x7f2b600d0e30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.582+0000 7f2b64bcf640 1 -- 172.21.15.27:0/3417445793 >> 172.21.15.27:0/3417445793 conn(0x7f2b60089ff0 msgr2=0x7f2b6009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:17.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.582+0000 7f2b64bcf640 1 -- 172.21.15.27:0/3417445793 shutdown_connections 2023-08-11T15:52:17.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.582+0000 7f2b64bcf640 1 -- 172.21.15.27:0/3417445793 wait complete. 2023-08-11T15:52:17.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.583+0000 7f2b64bcf640 1 Processor -- start 2023-08-11T15:52:17.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.584+0000 7f2b64bcf640 1 -- start start 2023-08-11T15:52:17.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b64bcf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b600cd920 0x7f2b600c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b64bcf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 0x7f2b601f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b64bcf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2b6015c950 0x7f2b600cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b64bcf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6015ed80 con 0x7f2b6015c570 2023-08-11T15:52:17.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b64bcf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b600c7410 con 0x7f2b600cd920 2023-08-11T15:52:17.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b64bcf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6015d2a0 con 0x7f2b6015c950 2023-08-11T15:52:17.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b5dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 0x7f2b601f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b5dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 0x7f2b601f1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41076/0 (socket says 172.21.15.27:41076) 2023-08-11T15:52:17.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b5dd9b640 1 -- 172.21.15.27:0/412676113 learned_addr learned my addr 172.21.15.27:0/412676113 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:17.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b5e59c640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b600cd920 0x7f2b600c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b5dd9b640 1 -- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2b6015c950 msgr2=0x7f2b600cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b5dd9b640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2b6015c950 0x7f2b600cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b5dd9b640 1 -- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b600cd920 msgr2=0x7f2b600c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b5dd9b640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b600cd920 0x7f2b600c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.585+0000 7f2b5dd9b640 1 -- 172.21.15.27:0/412676113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b500093a0 con 0x7f2b6015c570 2023-08-11T15:52:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.586+0000 7f2b5dd9b640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 0x7f2b601f1fc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800cba0 tx=0x7f2b4800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.586+0000 7f2b56ffd640 1 -- 172.21.15.27:0/412676113 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800ad40 con 0x7f2b6015c570 2023-08-11T15:52:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.586+0000 7f2b56ffd640 1 -- 172.21.15.27:0/412676113 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b48012400 con 0x7f2b6015c570 2023-08-11T15:52:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.586+0000 7f2b56ffd640 1 -- 172.21.15.27:0/412676113 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800aea0 con 0x7f2b6015c570 2023-08-11T15:52:17.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.586+0000 7f2b64bcf640 1 -- 172.21.15.27:0/412676113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6015e170 con 0x7f2b6015c570 2023-08-11T15:52:17.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.586+0000 7f2b64bcf640 1 -- 172.21.15.27:0/412676113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6015e730 con 0x7f2b6015c570 2023-08-11T15:52:17.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.588+0000 7f2b64bcf640 1 -- 172.21.15.27:0/412676113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6015ef30 con 0x7f2b6015c570 2023-08-11T15:52:17.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.589+0000 7f2b56ffd640 1 -- 172.21.15.27:0/412676113 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2b48007070 con 0x7f2b6015c570 2023-08-11T15:52:17.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.590+0000 7f2b56ffd640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2b4006a050 0x7f2b4006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:17.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.590+0000 7f2b56ffd640 1 -- 172.21.15.27:0/412676113 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2b48088780 con 0x7f2b6015c570 2023-08-11T15:52:17.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.591+0000 7f2b5e59c640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2b4006a050 0x7f2b4006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:17.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.592+0000 7f2b5e59c640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2b4006a050 0x7f2b4006c510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b50036000 tx=0x7f2b5003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:17.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.595+0000 7f2b56ffd640 1 -- 172.21.15.27:0/412676113 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2b6015ef30 con 0x7f2b6015c570 2023-08-11T15:52:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.803+0000 7f2b64bcf640 1 -- 172.21.15.27:0/412676113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b60073990 con 0x7f2b6015c570 2023-08-11T15:52:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.803+0000 7f2b56ffd640 1 -- 172.21.15.27:0/412676113 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2b48057680 con 0x7f2b6015c570 2023-08-11T15:52:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:52:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.806+0000 7f2b64bcf640 1 -- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2b4006a050 msgr2=0x7f2b4006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.806+0000 7f2b64bcf640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2b4006a050 0x7f2b4006c510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b50036000 tx=0x7f2b5003d040 comp rx=0 tx=0).stop 2023-08-11T15:52:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.806+0000 7f2b64bcf640 1 -- 172.21.15.27:0/412676113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 msgr2=0x7f2b601f1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.806+0000 7f2b64bcf640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 0x7f2b601f1fc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800cba0 tx=0x7f2b4800a430 comp rx=0 tx=0).stop 2023-08-11T15:52:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.806+0000 7f2b64bcf640 1 -- 172.21.15.27:0/412676113 shutdown_connections 2023-08-11T15:52:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.806+0000 7f2b64bcf640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2b4006a050 0x7f2b4006c510 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.806+0000 7f2b64bcf640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2b6015c950 0x7f2b600cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.806+0000 7f2b64bcf640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6015c570 0x7f2b601f1fc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.806+0000 7f2b64bcf640 1 --2- 172.21.15.27:0/412676113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b600cd920 0x7f2b600c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.806+0000 7f2b64bcf640 1 -- 172.21.15.27:0/412676113 >> 172.21.15.27:0/412676113 conn(0x7f2b60089ff0 msgr2=0x7f2b600ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.807+0000 7f2b64bcf640 1 -- 172.21.15.27:0/412676113 shutdown_connections 2023-08-11T15:52:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:17.807+0000 7f2b64bcf640 1 -- 172.21.15.27:0/412676113 wait complete. 2023-08-11T15:52:17.827 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:52:17.827 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:17.888 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.119+0000 7fe89d20d640 1 Processor -- start 2023-08-11T15:52:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.119+0000 7fe89d20d640 1 -- start start 2023-08-11T15:52:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.120+0000 7fe89d20d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe89806f0b0 0x7fe8980cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.120+0000 7fe89d20d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8980b3070 0x7fe8980b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.120+0000 7fe89d20d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89815e390 con 0x7fe89815a4a0 2023-08-11T15:52:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.120+0000 7fe89d20d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89806ff30 con 0x7fe8980b3070 2023-08-11T15:52:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.120+0000 7fe89d20d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89806f690 con 0x7fe89806f0b0 2023-08-11T15:52:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.121+0000 7fe896d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe89815a4a0 0x7fe89815e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42042/0 (socket says 172.21.15.27:42042) 2023-08-11T15:52:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.121+0000 7fe896d9d640 1 -- 172.21.15.27:0/3724244494 learned_addr learned my addr 172.21.15.27:0/3724244494 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.121+0000 7fe89759e640 1 --2- 172.21.15.27:0/3724244494 >> v2:172.21.15.27:3300/0 conn(0x7fe8980b3070 0x7fe8980b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.121+0000 7fe895d9b640 1 -- 172.21.15.27:0/3724244494 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699871863 0 0) 0x7fe89815e390 con 0x7fe89815a4a0 2023-08-11T15:52:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.121+0000 7fe89659c640 1 --2- 172.21.15.27:0/3724244494 >> v2:172.21.15.119:3300/0 conn(0x7fe89806f0b0 0x7fe8980cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.121+0000 7fe895d9b640 1 -- 172.21.15.27:0/3724244494 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe884003520 con 0x7fe89815a4a0 2023-08-11T15:52:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.122+0000 7fe89759e640 1 -- 172.21.15.27:0/3724244494 >> v2:172.21.15.119:3300/0 conn(0x7fe89806f0b0 msgr2=0x7fe8980cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.122+0000 7fe89759e640 1 --2- 172.21.15.27:0/3724244494 >> v2:172.21.15.119:3300/0 conn(0x7fe89806f0b0 0x7fe8980cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.122+0000 7fe89759e640 1 -- 172.21.15.27:0/3724244494 >> v1:172.21.15.27:6789/0 conn(0x7fe89815a4a0 legacy=0x7fe89815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.122+0000 7fe89759e640 1 -- 172.21.15.27:0/3724244494 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89806f970 con 0x7fe8980b3070 2023-08-11T15:52:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.123+0000 7fe89759e640 1 --2- 172.21.15.27:0/3724244494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8980b3070 0x7fe8980b34a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe8980cf5a0 tx=0x7fe89002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cd44e0eb966cb3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.124+0000 7fe895d9b640 1 -- 172.21.15.27:0/3724244494 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe890031830 con 0x7fe8980b3070 2023-08-11T15:52:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.124+0000 7fe895d9b640 1 -- 172.21.15.27:0/3724244494 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe890031990 con 0x7fe8980b3070 2023-08-11T15:52:18.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.124+0000 7fe895d9b640 1 -- 172.21.15.27:0/3724244494 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe890031d20 con 0x7fe8980b3070 2023-08-11T15:52:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.124+0000 7fe89d20d640 1 -- 172.21.15.27:0/3724244494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8980b3070 msgr2=0x7fe8980b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.124+0000 7fe89d20d640 1 --2- 172.21.15.27:0/3724244494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8980b3070 0x7fe8980b34a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe8980cf5a0 tx=0x7fe89002b8d0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.124+0000 7fe89d20d640 1 -- 172.21.15.27:0/3724244494 shutdown_connections 2023-08-11T15:52:18.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.124+0000 7fe89d20d640 1 --2- 172.21.15.27:0/3724244494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8980b3070 0x7fe8980b34a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.124+0000 7fe89d20d640 1 --2- 172.21.15.27:0/3724244494 >> v2:172.21.15.119:3300/0 conn(0x7fe89806f0b0 0x7fe8980cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.124+0000 7fe89d20d640 1 -- 172.21.15.27:0/3724244494 >> 172.21.15.27:0/3724244494 conn(0x7fe8980d0220 msgr2=0x7fe8980d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:18.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.124+0000 7fe89d20d640 1 -- 172.21.15.27:0/3724244494 shutdown_connections 2023-08-11T15:52:18.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.125+0000 7fe89d20d640 1 -- 172.21.15.27:0/3724244494 wait complete. 2023-08-11T15:52:18.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.126+0000 7fe89d20d640 1 Processor -- start 2023-08-11T15:52:18.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.126+0000 7fe89d20d640 1 -- start start 2023-08-11T15:52:18.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.127+0000 7fe89d20d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe89806f0b0 0x7fe8980c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.127+0000 7fe89d20d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8980b3070 0x7fe89814b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.127+0000 7fe89d20d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89815a4a0 0x7fe8980c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.127+0000 7fe89d20d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89814ba80 con 0x7fe89815a4a0 2023-08-11T15:52:18.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.127+0000 7fe89d20d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980c7d70 con 0x7fe8980b3070 2023-08-11T15:52:18.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.127+0000 7fe89d20d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980c8050 con 0x7fe89806f0b0 2023-08-11T15:52:18.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.128+0000 7fe89759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89815a4a0 0x7fe8980c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.128+0000 7fe89759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89815a4a0 0x7fe8980c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41092/0 (socket says 172.21.15.27:41092) 2023-08-11T15:52:18.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.128+0000 7fe89759e640 1 -- 172.21.15.27:0/3720007195 learned_addr learned my addr 172.21.15.27:0/3720007195 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:18.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.128+0000 7fe896d9d640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe89806f0b0 0x7fe8980c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.128+0000 7fe89659c640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8980b3070 0x7fe89814b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.128+0000 7fe89759e640 1 -- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe89806f0b0 msgr2=0x7fe8980c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.128+0000 7fe89759e640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe89806f0b0 0x7fe8980c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.128+0000 7fe89759e640 1 -- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8980b3070 msgr2=0x7fe89814b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.129+0000 7fe89759e640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8980b3070 0x7fe89814b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.129+0000 7fe89759e640 1 -- 172.21.15.27:0/3720007195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe890009400 con 0x7fe89815a4a0 2023-08-11T15:52:18.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.129+0000 7fe896d9d640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe89806f0b0 0x7fe8980c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:18.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.129+0000 7fe89759e640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89815a4a0 0x7fe8980c72f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe890009890 tx=0x7fe8900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:18.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.129+0000 7fe88f7fe640 1 -- 172.21.15.27:0/3720007195 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89003a630 con 0x7fe89815a4a0 2023-08-11T15:52:18.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.129+0000 7fe88f7fe640 1 -- 172.21.15.27:0/3720007195 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89003a790 con 0x7fe89815a4a0 2023-08-11T15:52:18.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.129+0000 7fe88f7fe640 1 -- 172.21.15.27:0/3720007195 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89003ab40 con 0x7fe89815a4a0 2023-08-11T15:52:18.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.129+0000 7fe89d20d640 1 -- 172.21.15.27:0/3720007195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8980c8330 con 0x7fe89815a4a0 2023-08-11T15:52:18.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.129+0000 7fe89d20d640 1 -- 172.21.15.27:0/3720007195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8980c88f0 con 0x7fe89815a4a0 2023-08-11T15:52:18.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.131+0000 7fe89d20d640 1 -- 172.21.15.27:0/3720007195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8980c8b10 con 0x7fe89815a4a0 2023-08-11T15:52:18.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.132+0000 7fe88f7fe640 1 -- 172.21.15.27:0/3720007195 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe89003aca0 con 0x7fe89815a4a0 2023-08-11T15:52:18.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.132+0000 7fe88f7fe640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe87806a050 0x7fe87806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.133+0000 7fe88f7fe640 1 -- 172.21.15.27:0/3720007195 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe8900aa460 con 0x7fe89815a4a0 2023-08-11T15:52:18.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.133+0000 7fe896d9d640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe87806a050 0x7fe87806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.134+0000 7fe896d9d640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe87806a050 0x7fe87806c510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe888007fd0 tx=0x7fe888009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:18.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.137+0000 7fe88f7fe640 1 -- 172.21.15.27:0/3720007195 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe890075810 con 0x7fe89815a4a0 2023-08-11T15:52:18.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.335+0000 7fe89d20d640 1 -- 172.21.15.27:0/3720007195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe898073990 con 0x7fe89815a4a0 2023-08-11T15:52:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.338+0000 7fe88f7fe640 1 -- 172.21.15.27:0/3720007195 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7fe890078e30 con 0x7fe89815a4a0 2023-08-11T15:52:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:18.341 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.341+0000 7fe89d20d640 1 -- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe87806a050 msgr2=0x7fe87806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.341+0000 7fe89d20d640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe87806a050 0x7fe87806c510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe888007fd0 tx=0x7fe888009040 comp rx=0 tx=0).stop 2023-08-11T15:52:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.341+0000 7fe89d20d640 1 -- 172.21.15.27:0/3720007195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89815a4a0 msgr2=0x7fe8980c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.341+0000 7fe89d20d640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89815a4a0 0x7fe8980c72f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe890009890 tx=0x7fe8900046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.341+0000 7fe89d20d640 1 -- 172.21.15.27:0/3720007195 shutdown_connections 2023-08-11T15:52:18.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.341+0000 7fe89d20d640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe87806a050 0x7fe87806c510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.341+0000 7fe89d20d640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89815a4a0 0x7fe8980c72f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.341+0000 7fe89d20d640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8980b3070 0x7fe89814b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.341+0000 7fe89d20d640 1 --2- 172.21.15.27:0/3720007195 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe89806f0b0 0x7fe8980c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.341+0000 7fe89d20d640 1 -- 172.21.15.27:0/3720007195 >> 172.21.15.27:0/3720007195 conn(0x7fe8980d0220 msgr2=0x7fe8980c45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:18.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.342+0000 7fe89d20d640 1 -- 172.21.15.27:0/3720007195 shutdown_connections 2023-08-11T15:52:18.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.342+0000 7fe89d20d640 1 -- 172.21.15.27:0/3720007195 wait complete. 2023-08-11T15:52:18.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:18.363 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:18.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.606+0000 7fcc6a979640 1 Processor -- start 2023-08-11T15:52:18.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.606+0000 7fcc6a979640 1 -- start start 2023-08-11T15:52:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.607+0000 7fcc6a979640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc6415c4a0 0x7fcc640cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.607+0000 7fcc6a979640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcc640cf7d0 0x7fcc640d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.608+0000 7fcc6a979640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc640d31a0 con 0x7fcc640d1e30 2023-08-11T15:52:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.608+0000 7fcc6a979640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640cfc40 con 0x7fcc640cf7d0 2023-08-11T15:52:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.608+0000 7fcc6a979640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640d2590 con 0x7fcc6415c4a0 2023-08-11T15:52:18.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.608+0000 7fcc637fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcc640cf7d0 0x7fcc640d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.608+0000 7fcc637fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcc640cf7d0 0x7fcc640d18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41100/0 (socket says 172.21.15.27:41100) 2023-08-11T15:52:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.608+0000 7fcc637fe640 1 -- 172.21.15.27:0/194126968 learned_addr learned my addr 172.21.15.27:0/194126968 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.608+0000 7fcc68f16640 1 --1- 172.21.15.27:0/194126968 >> v1:172.21.15.27:6789/0 conn(0x7fcc640d1e30 0x7fcc640d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42054/0 (socket says 172.21.15.27:42054) 2023-08-11T15:52:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.608+0000 7fcc63fff640 1 --2- 172.21.15.27:0/194126968 >> v2:172.21.15.119:3300/0 conn(0x7fcc6415c4a0 0x7fcc640cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.609+0000 7fcc637fe640 1 -- 172.21.15.27:0/194126968 >> v2:172.21.15.119:3300/0 conn(0x7fcc6415c4a0 msgr2=0x7fcc640cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.609+0000 7fcc637fe640 1 --2- 172.21.15.27:0/194126968 >> v2:172.21.15.119:3300/0 conn(0x7fcc6415c4a0 0x7fcc640cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.609+0000 7fcc637fe640 1 -- 172.21.15.27:0/194126968 >> v1:172.21.15.27:6789/0 conn(0x7fcc640d1e30 legacy=0x7fcc640d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.609+0000 7fcc637fe640 1 -- 172.21.15.27:0/194126968 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640d2870 con 0x7fcc640cf7d0 2023-08-11T15:52:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.610+0000 7fcc637fe640 1 --2- 172.21.15.27:0/194126968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 0x7fcc640d18f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcc640ced90 tx=0x7fcc4c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2506c8bbacaea1fa server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.610+0000 7fcc62ffd640 1 -- 172.21.15.27:0/194126968 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c030860 con 0x7fcc640cf7d0 2023-08-11T15:52:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.610+0000 7fcc62ffd640 1 -- 172.21.15.27:0/194126968 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4c0309c0 con 0x7fcc640cf7d0 2023-08-11T15:52:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.610+0000 7fcc62ffd640 1 -- 172.21.15.27:0/194126968 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c030d50 con 0x7fcc640cf7d0 2023-08-11T15:52:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.611+0000 7fcc6a979640 1 -- 172.21.15.27:0/194126968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 msgr2=0x7fcc640d18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.611+0000 7fcc6a979640 1 --2- 172.21.15.27:0/194126968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 0x7fcc640d18f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcc640ced90 tx=0x7fcc4c02b9a0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.611+0000 7fcc6a979640 1 -- 172.21.15.27:0/194126968 shutdown_connections 2023-08-11T15:52:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.611+0000 7fcc6a979640 1 --2- 172.21.15.27:0/194126968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 0x7fcc640d18f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.611+0000 7fcc6a979640 1 --2- 172.21.15.27:0/194126968 >> v2:172.21.15.119:3300/0 conn(0x7fcc6415c4a0 0x7fcc640cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.611+0000 7fcc6a979640 1 -- 172.21.15.27:0/194126968 >> 172.21.15.27:0/194126968 conn(0x7fcc64071320 msgr2=0x7fcc64071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:18.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.611+0000 7fcc6a979640 1 -- 172.21.15.27:0/194126968 shutdown_connections 2023-08-11T15:52:18.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.611+0000 7fcc6a979640 1 -- 172.21.15.27:0/194126968 wait complete. 2023-08-11T15:52:18.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.613+0000 7fcc6a979640 1 Processor -- start 2023-08-11T15:52:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.613+0000 7fcc6a979640 1 -- start start 2023-08-11T15:52:18.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.614+0000 7fcc6a979640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 0x7fcc640b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.614+0000 7fcc6a979640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc640d1e30 0x7fcc6414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.614+0000 7fcc6a979640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcc6415c4a0 0x7fcc6414f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.614+0000 7fcc6a979640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6414fbe0 con 0x7fcc640cf7d0 2023-08-11T15:52:18.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.614+0000 7fcc6a979640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640ca570 con 0x7fcc640d1e30 2023-08-11T15:52:18.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.614+0000 7fcc6a979640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640ca850 con 0x7fcc6415c4a0 2023-08-11T15:52:18.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.614+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 0x7fcc640b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.615+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 0x7fcc640b7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41106/0 (socket says 172.21.15.27:41106) 2023-08-11T15:52:18.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.615+0000 7fcc63fff640 1 -- 172.21.15.27:0/3535470371 learned_addr learned my addr 172.21.15.27:0/3535470371 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:18.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.615+0000 7fcc637fe640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc640d1e30 0x7fcc6414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.615+0000 7fcc68f16640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcc6415c4a0 0x7fcc6414f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.615+0000 7fcc63fff640 1 -- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcc6415c4a0 msgr2=0x7fcc6414f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.615+0000 7fcc63fff640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcc6415c4a0 0x7fcc6414f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.615+0000 7fcc63fff640 1 -- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc640d1e30 msgr2=0x7fcc6414f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.615+0000 7fcc63fff640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc640d1e30 0x7fcc6414f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.615+0000 7fcc63fff640 1 -- 172.21.15.27:0/3535470371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0094d0 con 0x7fcc640cf7d0 2023-08-11T15:52:18.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.615+0000 7fcc63fff640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 0x7fcc640b7220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400efc0 tx=0x7fcc5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:18.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.616+0000 7fcc60ff9640 1 -- 172.21.15.27:0/3535470371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54009b80 con 0x7fcc640cf7d0 2023-08-11T15:52:18.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.616+0000 7fcc60ff9640 1 -- 172.21.15.27:0/3535470371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5400f040 con 0x7fcc640cf7d0 2023-08-11T15:52:18.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.616+0000 7fcc6a979640 1 -- 172.21.15.27:0/3535470371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc640cab30 con 0x7fcc640cf7d0 2023-08-11T15:52:18.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.616+0000 7fcc60ff9640 1 -- 172.21.15.27:0/3535470371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc540146a0 con 0x7fcc640cf7d0 2023-08-11T15:52:18.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.616+0000 7fcc6a979640 1 -- 172.21.15.27:0/3535470371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc640caec0 con 0x7fcc640cf7d0 2023-08-11T15:52:18.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.617+0000 7fcc6a979640 1 -- 172.21.15.27:0/3535470371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6404e420 con 0x7fcc640cf7d0 2023-08-11T15:52:18.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.618+0000 7fcc60ff9640 1 -- 172.21.15.27:0/3535470371 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcc54014800 con 0x7fcc640cf7d0 2023-08-11T15:52:18.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.619+0000 7fcc60ff9640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcc4406a050 0x7fcc4406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:18.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.619+0000 7fcc60ff9640 1 -- 172.21.15.27:0/3535470371 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcc540896f0 con 0x7fcc640cf7d0 2023-08-11T15:52:18.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.620+0000 7fcc637fe640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcc4406a050 0x7fcc4406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:18.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.621+0000 7fcc637fe640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcc4406a050 0x7fcc4406c510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c036000 tx=0x7fcc4c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:18.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.623+0000 7fcc60ff9640 1 -- 172.21.15.27:0/3535470371 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fcc54058540 con 0x7fcc640cf7d0 2023-08-11T15:52:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.817+0000 7fcc6a979640 1 -- 172.21.15.27:0/3535470371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc64072600 con 0x7fcc640cf7d0 2023-08-11T15:52:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.819+0000 7fcc60ff9640 1 -- 172.21.15.27:0/3535470371 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc540580d0 con 0x7fcc640cf7d0 2023-08-11T15:52:18.821 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:18.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.821+0000 7fcc6a979640 1 -- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcc4406a050 msgr2=0x7fcc4406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.821+0000 7fcc6a979640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcc4406a050 0x7fcc4406c510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c036000 tx=0x7fcc4c008030 comp rx=0 tx=0).stop 2023-08-11T15:52:18.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.821+0000 7fcc6a979640 1 -- 172.21.15.27:0/3535470371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 msgr2=0x7fcc640b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.821+0000 7fcc6a979640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 0x7fcc640b7220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400efc0 tx=0x7fcc5400c490 comp rx=0 tx=0).stop 2023-08-11T15:52:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.821+0000 7fcc6a979640 1 -- 172.21.15.27:0/3535470371 shutdown_connections 2023-08-11T15:52:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.821+0000 7fcc6a979640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcc4406a050 0x7fcc4406c510 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.821+0000 7fcc6a979640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcc6415c4a0 0x7fcc6414f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.821+0000 7fcc6a979640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc640d1e30 0x7fcc6414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.821+0000 7fcc6a979640 1 --2- 172.21.15.27:0/3535470371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cf7d0 0x7fcc640b7220 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.821+0000 7fcc6a979640 1 -- 172.21.15.27:0/3535470371 >> 172.21.15.27:0/3535470371 conn(0x7fcc64071320 msgr2=0x7fcc640d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.822+0000 7fcc6a979640 1 -- 172.21.15.27:0/3535470371 shutdown_connections 2023-08-11T15:52:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:18.822+0000 7fcc6a979640 1 -- 172.21.15.27:0/3535470371 wait complete. 2023-08-11T15:52:18.842 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:19.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.044+0000 7fc302ed9640 1 Processor -- start 2023-08-11T15:52:19.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.044+0000 7fc302ed9640 1 -- start start 2023-08-11T15:52:19.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.045+0000 7fc302ed9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc2fc15c4a0 0x7fc2fc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.045+0000 7fc302ed9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2fc0cf7d0 0x7fc2fc0d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.046+0000 7fc302ed9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2fc0d31a0 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.046+0000 7fc302ed9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc0cfc40 con 0x7fc2fc0cf7d0 2023-08-11T15:52:19.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.046+0000 7fc302ed9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc0d2590 con 0x7fc2fc15c4a0 2023-08-11T15:52:19.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.046+0000 7fc2fbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2fc0cf7d0 0x7fc2fc0d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.046+0000 7fc2fbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2fc0cf7d0 0x7fc2fc0d18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41118/0 (socket says 172.21.15.27:41118) 2023-08-11T15:52:19.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.046+0000 7fc2fbfff640 1 -- 172.21.15.27:0/2380093155 learned_addr learned my addr 172.21.15.27:0/2380093155 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:19.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.046+0000 7fc300c75640 1 --2- 172.21.15.27:0/2380093155 >> v2:172.21.15.119:3300/0 conn(0x7fc2fc15c4a0 0x7fc2fc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.047+0000 7fc2fbfff640 1 -- 172.21.15.27:0/2380093155 >> v2:172.21.15.119:3300/0 conn(0x7fc2fc15c4a0 msgr2=0x7fc2fc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:19.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.047+0000 7fc2fbfff640 1 --2- 172.21.15.27:0/2380093155 >> v2:172.21.15.119:3300/0 conn(0x7fc2fc15c4a0 0x7fc2fc0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.047+0000 7fc2fbfff640 1 -- 172.21.15.27:0/2380093155 >> v1:172.21.15.119:6789/0 conn(0x7fc2fc0d1e30 legacy=0x7fc2fc0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:19.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.047+0000 7fc2fbfff640 1 -- 172.21.15.27:0/2380093155 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0d2870 con 0x7fc2fc0cf7d0 2023-08-11T15:52:19.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.048+0000 7fc2fbfff640 1 --2- 172.21.15.27:0/2380093155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0cf7d0 0x7fc2fc0d18f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0ced90 tx=0x7fc2f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9386f816b624b1d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:19.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.048+0000 7fc2fb7fe640 1 -- 172.21.15.27:0/2380093155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0031870 con 0x7fc2fc0cf7d0 2023-08-11T15:52:19.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.048+0000 7fc2fb7fe640 1 -- 172.21.15.27:0/2380093155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f00319d0 con 0x7fc2fc0cf7d0 2023-08-11T15:52:19.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.048+0000 7fc2fb7fe640 1 -- 172.21.15.27:0/2380093155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0031d60 con 0x7fc2fc0cf7d0 2023-08-11T15:52:19.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.049+0000 7fc302ed9640 1 -- 172.21.15.27:0/2380093155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0cf7d0 msgr2=0x7fc2fc0d18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.049+0000 7fc302ed9640 1 --2- 172.21.15.27:0/2380093155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0cf7d0 0x7fc2fc0d18f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0ced90 tx=0x7fc2f002b910 comp rx=0 tx=0).stop 2023-08-11T15:52:19.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.049+0000 7fc302ed9640 1 -- 172.21.15.27:0/2380093155 shutdown_connections 2023-08-11T15:52:19.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.049+0000 7fc302ed9640 1 --2- 172.21.15.27:0/2380093155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0cf7d0 0x7fc2fc0d18f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.049+0000 7fc302ed9640 1 --2- 172.21.15.27:0/2380093155 >> v2:172.21.15.119:3300/0 conn(0x7fc2fc15c4a0 0x7fc2fc0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.049+0000 7fc302ed9640 1 -- 172.21.15.27:0/2380093155 >> 172.21.15.27:0/2380093155 conn(0x7fc2fc071320 msgr2=0x7fc2fc071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:19.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.049+0000 7fc302ed9640 1 -- 172.21.15.27:0/2380093155 shutdown_connections 2023-08-11T15:52:19.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.049+0000 7fc302ed9640 1 -- 172.21.15.27:0/2380093155 wait complete. 2023-08-11T15:52:19.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.051+0000 7fc302ed9640 1 Processor -- start 2023-08-11T15:52:19.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.051+0000 7fc302ed9640 1 -- start start 2023-08-11T15:52:19.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.052+0000 7fc302ed9640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2fc0cf7d0 0x7fc2fc15f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.052+0000 7fc302ed9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0d1e30 0x7fc2fc15c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.052+0000 7fc302ed9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2fc15c4a0 0x7fc2fc145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.052+0000 7fc302ed9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc15fa90 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.052+0000 7fc302ed9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc146740 con 0x7fc2fc15c4a0 2023-08-11T15:52:19.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.052+0000 7fc302ed9640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc146a20 con 0x7fc2fc0cf7d0 2023-08-11T15:52:19.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc2fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0d1e30 0x7fc2fc15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc2fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0d1e30 0x7fc2fc15c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41120/0 (socket says 172.21.15.27:41120) 2023-08-11T15:52:19.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc2fbfff640 1 -- 172.21.15.27:0/1989234753 learned_addr learned my addr 172.21.15.27:0/1989234753 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc300c75640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2fc0cf7d0 0x7fc2fc15f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc301476640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2fc15c4a0 0x7fc2fc145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc2fbfff640 1 -- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2fc0cf7d0 msgr2=0x7fc2fc15f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc2fbfff640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2fc0cf7d0 0x7fc2fc15f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc2fbfff640 1 -- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2fc15c4a0 msgr2=0x7fc2fc145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc2fbfff640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2fc15c4a0 0x7fc2fc145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc2fbfff640 1 -- 172.21.15.27:0/1989234753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f00094d0 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc301476640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2fc15c4a0 0x7fc2fc145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.053+0000 7fc2fbfff640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0d1e30 0x7fc2fc15c880 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.054+0000 7fc2f97fa640 1 -- 172.21.15.27:0/1989234753 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f003abe0 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.054+0000 7fc2f97fa640 1 -- 172.21.15.27:0/1989234753 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f003ad40 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.054+0000 7fc2f97fa640 1 -- 172.21.15.27:0/1989234753 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0044610 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.054+0000 7fc302ed9640 1 -- 172.21.15.27:0/1989234753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc146d00 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.054+0000 7fc302ed9640 1 -- 172.21.15.27:0/1989234753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc1472c0 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.055+0000 7fc302ed9640 1 -- 172.21.15.27:0/1989234753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc1474b0 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.056+0000 7fc2f97fa640 1 -- 172.21.15.27:0/1989234753 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc2f004d450 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.057+0000 7fc2f97fa640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc2e806a080 0x7fc2e806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.057+0000 7fc2f97fa640 1 -- 172.21.15.27:0/1989234753 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc2f002d040 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.058+0000 7fc300c75640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc2e806a080 0x7fc2e806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.059+0000 7fc300c75640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc2e806a080 0x7fc2e806c540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0099c0 tx=0x7fc2ec003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:19.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.062+0000 7fc2f97fa640 1 -- 172.21.15.27:0/1989234753 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fc2f00762e0 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.254+0000 7fc302ed9640 1 -- 172.21.15.27:0/1989234753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc2fc1474b0 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.255+0000 7fc2f97fa640 1 -- 172.21.15.27:0/1989234753 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc2f00350a0 con 0x7fc2fc0d1e30 2023-08-11T15:52:19.257 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:19.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 -- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc2e806a080 msgr2=0x7fc2e806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc2e806a080 0x7fc2e806c540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0099c0 tx=0x7fc2ec003990 comp rx=0 tx=0).stop 2023-08-11T15:52:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 -- 172.21.15.27:0/1989234753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0d1e30 msgr2=0x7fc2fc15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0d1e30 0x7fc2fc15c880 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f00046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 -- 172.21.15.27:0/1989234753 shutdown_connections 2023-08-11T15:52:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc2e806a080 0x7fc2e806c540 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2fc15c4a0 0x7fc2fc145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2fc0d1e30 0x7fc2fc15c880 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 --2- 172.21.15.27:0/1989234753 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2fc0cf7d0 0x7fc2fc15f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 -- 172.21.15.27:0/1989234753 >> 172.21.15.27:0/1989234753 conn(0x7fc2fc071320 msgr2=0x7fc2fc072330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 -- 172.21.15.27:0/1989234753 shutdown_connections 2023-08-11T15:52:19.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.258+0000 7fc302ed9640 1 -- 172.21.15.27:0/1989234753 wait complete. 2023-08-11T15:52:19.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:19.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.519+0000 7f08953b5640 1 Processor -- start 2023-08-11T15:52:19.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.519+0000 7f08953b5640 1 -- start start 2023-08-11T15:52:19.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.520+0000 7f08953b5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f08900b1d10 0x7f08900b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.520+0000 7f08953b5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08900cf1f0 0x7f08900cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.521+0000 7f08953b5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08900b2630 con 0x7f08900cada0 2023-08-11T15:52:19.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.521+0000 7f08953b5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089015a8b0 con 0x7f08900b1d10 2023-08-11T15:52:19.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.521+0000 7f08953b5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08900cfb20 con 0x7f08900cf1f0 2023-08-11T15:52:19.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.521+0000 7f088e7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f08900cada0 0x7f089015b2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47068/0 (socket says 172.21.15.27:47068) 2023-08-11T15:52:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.521+0000 7f088e7fc640 1 -- 172.21.15.27:0/3179748232 learned_addr learned my addr 172.21.15.27:0/3179748232 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.521+0000 7f088effd640 1 --2- 172.21.15.27:0/3179748232 >> v2:172.21.15.119:3300/0 conn(0x7f08900b1d10 0x7f08900b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.521+0000 7f088f7fe640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08900cf1f0 0x7f08900cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.521+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269985318 0 0) 0x7f08900b2630 con 0x7f08900cada0 2023-08-11T15:52:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.521+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0878003520 con 0x7f08900cada0 2023-08-11T15:52:19.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3214623060 0 0) 0x7f0878003520 con 0x7f08900cada0 2023-08-11T15:52:19.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08900cf1f0 msgr2=0x7f08900cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:19.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08900cf1f0 0x7f08900cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 >> v2:172.21.15.119:3300/0 conn(0x7f08900b1d10 msgr2=0x7f08900b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:19.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 --2- 172.21.15.27:0/3179748232 >> v2:172.21.15.119:3300/0 conn(0x7f08900b1d10 0x7f08900b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08900cfe00 con 0x7f08900cada0 2023-08-11T15:52:19.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f0884001670 con 0x7f08900cada0 2023-08-11T15:52:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 >> v1:172.21.15.27:6789/0 conn(0x7f08900cada0 legacy=0x7f089015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0878003970 0x7f0878003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08900cf1f0 0x7f08780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0878004d00 0x7f0878005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089015a8b0 con 0x7f0878003970 2023-08-11T15:52:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08900cfb20 con 0x7f0878004d00 2023-08-11T15:52:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087800fd90 con 0x7f08900cf1f0 2023-08-11T15:52:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.522+0000 7f088effd640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0878003970 0x7f0878003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.523+0000 7f088f7fe640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0878004d00 0x7f0878005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.523+0000 7f088effd640 1 -- 172.21.15.27:0/3179748232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08900cf1f0 msgr2=0x7f08780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:19.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.523+0000 7f088effd640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08900cf1f0 0x7f08780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.523+0000 7f088effd640 1 -- 172.21.15.27:0/3179748232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0878004d00 msgr2=0x7f0878005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:19.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.523+0000 7f088effd640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0878004d00 0x7f0878005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.523+0000 7f088effd640 1 -- 172.21.15.27:0/3179748232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878003520 con 0x7f0878003970 2023-08-11T15:52:19.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.524+0000 7f088effd640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0878003970 0x7f0878003d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0880009ac0 tx=0x7f08780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8838af11e1fb287a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:19.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.524+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088002c710 con 0x7f0878003970 2023-08-11T15:52:19.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.524+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088002c870 con 0x7f0878003970 2023-08-11T15:52:19.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.524+0000 7f088dffb640 1 -- 172.21.15.27:0/3179748232 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088002cc00 con 0x7f0878003970 2023-08-11T15:52:19.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.525+0000 7f08953b5640 1 -- 172.21.15.27:0/3179748232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0878003970 msgr2=0x7f0878003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.525+0000 7f08953b5640 1 --2- 172.21.15.27:0/3179748232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0878003970 0x7f0878003d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0880009ac0 tx=0x7f08780057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.525+0000 7f088f7fe640 1 -- 172.21.15.27:0/3179748232 reap_dead start 2023-08-11T15:52:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.525+0000 7f08953b5640 1 -- 172.21.15.27:0/3179748232 shutdown_connections 2023-08-11T15:52:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.525+0000 7f08953b5640 1 -- 172.21.15.27:0/3179748232 >> 172.21.15.27:0/3179748232 conn(0x7f08900ccd10 msgr2=0x7f08900cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.525+0000 7f08953b5640 1 -- 172.21.15.27:0/3179748232 shutdown_connections 2023-08-11T15:52:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.525+0000 7f08953b5640 1 -- 172.21.15.27:0/3179748232 wait complete. 2023-08-11T15:52:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.527+0000 7f08953b5640 1 Processor -- start 2023-08-11T15:52:19.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.527+0000 7f08953b5640 1 -- start start 2023-08-11T15:52:19.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.528+0000 7f08953b5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0878003970 0x7f0890151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.528+0000 7f08953b5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08900c5780 0x7f08900c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.528+0000 7f08953b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08900c97d0 0x7f08900c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.528+0000 7f08953b5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08900c5360 con 0x7f08900c5780 2023-08-11T15:52:19.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.528+0000 7f08953b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08900c68e0 con 0x7f08900c97d0 2023-08-11T15:52:19.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.528+0000 7f08953b5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08900c9c50 con 0x7f0878003970 2023-08-11T15:52:19.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.528+0000 7f088e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08900c5780 0x7f08900c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.528+0000 7f088e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08900c5780 0x7f08900c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60222/0 (socket says 172.21.15.27:60222) 2023-08-11T15:52:19.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.528+0000 7f088e7fc640 1 -- 172.21.15.27:0/2780142407 learned_addr learned my addr 172.21.15.27:0/2780142407 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:19.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f088f7fe640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08900c97d0 0x7f08900c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f088e7fc640 1 -- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0878003970 msgr2=0x7f0890151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f088effd640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0878003970 0x7f0890151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f088e7fc640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0878003970 0x7f0890151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f088e7fc640 1 -- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08900c97d0 msgr2=0x7f08900c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f088e7fc640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08900c97d0 0x7f08900c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f088e7fc640 1 -- 172.21.15.27:0/2780142407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08900c46e0 con 0x7f08900c5780 2023-08-11T15:52:19.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f088f7fe640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08900c97d0 0x7f08900c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:19.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f088e7fc640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08900c5780 0x7f08900c9290 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0884001820 tx=0x7f088400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:19.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f08777fe640 1 -- 172.21.15.27:0/2780142407 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840072d0 con 0x7f08900c5780 2023-08-11T15:52:19.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f08777fe640 1 -- 172.21.15.27:0/2780142407 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0884007430 con 0x7f08900c5780 2023-08-11T15:52:19.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.529+0000 7f08777fe640 1 -- 172.21.15.27:0/2780142407 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840047e0 con 0x7f08900c5780 2023-08-11T15:52:19.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.530+0000 7f08953b5640 1 -- 172.21.15.27:0/2780142407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08900c49c0 con 0x7f08900c5780 2023-08-11T15:52:19.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.530+0000 7f08953b5640 1 -- 172.21.15.27:0/2780142407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08900c4f80 con 0x7f08900c5780 2023-08-11T15:52:19.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.531+0000 7f08953b5640 1 -- 172.21.15.27:0/2780142407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08900c54a0 con 0x7f08900c5780 2023-08-11T15:52:19.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.532+0000 7f08777fe640 1 -- 172.21.15.27:0/2780142407 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0884004940 con 0x7f08900c5780 2023-08-11T15:52:19.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.533+0000 7f08777fe640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a080 0x7f087c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.533+0000 7f08777fe640 1 -- 172.21.15.27:0/2780142407 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0884087dd0 con 0x7f08900c5780 2023-08-11T15:52:19.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.533+0000 7f088effd640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a080 0x7f087c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.534+0000 7f088effd640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a080 0x7f087c06c540 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0880036000 tx=0x7f0880030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:19.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.537+0000 7f08777fe640 1 -- 172.21.15.27:0/2780142407 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f08900c54a0 con 0x7f08900c5780 2023-08-11T15:52:19.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.733+0000 7f08953b5640 1 -- 172.21.15.27:0/2780142407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08900c54a0 con 0x7f08900c5780 2023-08-11T15:52:19.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.733+0000 7f08777fe640 1 -- 172.21.15.27:0/2780142407 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f08900c54a0 con 0x7f08900c5780 2023-08-11T15:52:19.735 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:19.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.736+0000 7f08953b5640 1 -- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a080 msgr2=0x7f087c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.736+0000 7f08953b5640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a080 0x7f087c06c540 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0880036000 tx=0x7f0880030000 comp rx=0 tx=0).stop 2023-08-11T15:52:19.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.736+0000 7f08953b5640 1 -- 172.21.15.27:0/2780142407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08900c5780 msgr2=0x7f08900c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:19.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.736+0000 7f08953b5640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08900c5780 0x7f08900c9290 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0884001820 tx=0x7f088400cb90 comp rx=0 tx=0).stop 2023-08-11T15:52:19.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.736+0000 7f08953b5640 1 -- 172.21.15.27:0/2780142407 shutdown_connections 2023-08-11T15:52:19.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.736+0000 7f08953b5640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a080 0x7f087c06c540 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.736+0000 7f08953b5640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08900c97d0 0x7f08900c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.736+0000 7f08953b5640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08900c5780 0x7f08900c9290 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.736+0000 7f08953b5640 1 --2- 172.21.15.27:0/2780142407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0878003970 0x7f0890151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:19.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.736+0000 7f08953b5640 1 -- 172.21.15.27:0/2780142407 >> 172.21.15.27:0/2780142407 conn(0x7f08900ccd10 msgr2=0x7f08900722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:19.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.737+0000 7f08953b5640 1 -- 172.21.15.27:0/2780142407 shutdown_connections 2023-08-11T15:52:19.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.737+0000 7f08953b5640 1 -- 172.21.15.27:0/2780142407 wait complete. 2023-08-11T15:52:19.759 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:19.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.991+0000 7f7ea2c83640 1 Processor -- start 2023-08-11T15:52:19.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.991+0000 7f7ea2c83640 1 -- start start 2023-08-11T15:52:19.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.993+0000 7f7ea2c83640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7e9c0cbbc0 0x7f7e9c0c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.993+0000 7f7ea2c83640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e9c0c8d90 0x7f7e9c0ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:19.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.993+0000 7f7ea2c83640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c06fe70 con 0x7f7e9c06d690 2023-08-11T15:52:19.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.993+0000 7f7ea2c83640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c158a10 con 0x7f7e9c0cbbc0 2023-08-11T15:52:19.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.993+0000 7f7ea2c83640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c06f5a0 con 0x7f7e9c0c8d90 2023-08-11T15:52:19.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.994+0000 7f7ea0a1f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7e9c06d690 0x7f7e9c158300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47070/0 (socket says 172.21.15.27:47070) 2023-08-11T15:52:19.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.994+0000 7f7ea0a1f640 1 -- 172.21.15.27:0/3136002310 learned_addr learned my addr 172.21.15.27:0/3136002310 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.994+0000 7f7e9bfff640 1 --2- 172.21.15.27:0/3136002310 >> v2:172.21.15.27:3300/0 conn(0x7f7e9c0cbbc0 0x7f7e9c0c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.994+0000 7f7ea1220640 1 --2- 172.21.15.27:0/3136002310 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e9c0c8d90 0x7f7e9c0ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.994+0000 7f7e9b7fe640 1 -- 172.21.15.27:0/3136002310 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019984911 0 0) 0x7f7e9c06fe70 con 0x7f7e9c06d690 2023-08-11T15:52:20.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.994+0000 7f7e9b7fe640 1 -- 172.21.15.27:0/3136002310 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e84003520 con 0x7f7e9c06d690 2023-08-11T15:52:20.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.994+0000 7f7e9bfff640 1 -- 172.21.15.27:0/3136002310 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e9c0c8d90 msgr2=0x7f7e9c0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.995+0000 7f7e9bfff640 1 --2- 172.21.15.27:0/3136002310 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e9c0c8d90 0x7f7e9c0ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.995+0000 7f7e9bfff640 1 -- 172.21.15.27:0/3136002310 >> v1:172.21.15.27:6789/0 conn(0x7f7e9c06d690 legacy=0x7f7e9c158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.995+0000 7f7e9bfff640 1 -- 172.21.15.27:0/3136002310 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0c9580 con 0x7f7e9c0cbbc0 2023-08-11T15:52:20.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.996+0000 7f7e9bfff640 1 --2- 172.21.15.27:0/3136002310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c0cbbc0 0x7f7e9c0c8850 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0097a0 tx=0x7f7e8c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966ac29e9c75c326 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.996+0000 7f7e9b7fe640 1 -- 172.21.15.27:0/3136002310 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c02d040 con 0x7f7e9c0cbbc0 2023-08-11T15:52:20.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.996+0000 7f7e9b7fe640 1 -- 172.21.15.27:0/3136002310 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c031810 con 0x7f7e9c0cbbc0 2023-08-11T15:52:20.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.996+0000 7f7e9b7fe640 1 -- 172.21.15.27:0/3136002310 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c031b80 con 0x7f7e9c0cbbc0 2023-08-11T15:52:20.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.997+0000 7f7ea2c83640 1 -- 172.21.15.27:0/3136002310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c0cbbc0 msgr2=0x7f7e9c0c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.997+0000 7f7ea2c83640 1 --2- 172.21.15.27:0/3136002310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c0cbbc0 0x7f7e9c0c8850 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0097a0 tx=0x7f7e8c02b8b0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.997+0000 7f7ea2c83640 1 -- 172.21.15.27:0/3136002310 shutdown_connections 2023-08-11T15:52:20.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.997+0000 7f7ea2c83640 1 --2- 172.21.15.27:0/3136002310 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e9c0c8d90 0x7f7e9c0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.997+0000 7f7ea2c83640 1 --2- 172.21.15.27:0/3136002310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c0cbbc0 0x7f7e9c0c8850 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.997+0000 7f7ea2c83640 1 -- 172.21.15.27:0/3136002310 >> 172.21.15.27:0/3136002310 conn(0x7f7e9c0afbe0 msgr2=0x7f7e9c0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.997+0000 7f7ea2c83640 1 -- 172.21.15.27:0/3136002310 shutdown_connections 2023-08-11T15:52:20.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:19.997+0000 7f7ea2c83640 1 -- 172.21.15.27:0/3136002310 wait complete. 2023-08-11T15:52:20.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.000+0000 7f7ea2c83640 1 Processor -- start 2023-08-11T15:52:20.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.000+0000 7f7ea2c83640 1 -- start start 2023-08-11T15:52:20.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.001+0000 7f7ea2c83640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c06d690 0x7f7e9c1440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea2c83640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e9c0c8d90 0x7f7e9c158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea2c83640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c0cbbc0 0x7f7e9c0cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea2c83640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1e5190 con 0x7f7e9c06d690 2023-08-11T15:52:20.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea2c83640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0c2190 con 0x7f7e9c0cbbc0 2023-08-11T15:52:20.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea2c83640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c159140 con 0x7f7e9c0c8d90 2023-08-11T15:52:20.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea0a1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c06d690 0x7f7e9c1440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea0a1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c06d690 0x7f7e9c1440a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60236/0 (socket says 172.21.15.27:60236) 2023-08-11T15:52:20.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea0a1f640 1 -- 172.21.15.27:0/587216439 learned_addr learned my addr 172.21.15.27:0/587216439 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:20.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7e9bfff640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e9c0c8d90 0x7f7e9c158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea1220640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c0cbbc0 0x7f7e9c0cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea0a1f640 1 -- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e9c0c8d90 msgr2=0x7f7e9c158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea0a1f640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e9c0c8d90 0x7f7e9c158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea0a1f640 1 -- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c0cbbc0 msgr2=0x7f7e9c0cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea0a1f640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c0cbbc0 0x7f7e9c0cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.002+0000 7f7ea0a1f640 1 -- 172.21.15.27:0/587216439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c009400 con 0x7f7e9c06d690 2023-08-11T15:52:20.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.003+0000 7f7ea0a1f640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c06d690 0x7f7e9c1440a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000c460 tx=0x7f7e9000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.003+0000 7f7e997fa640 1 -- 172.21.15.27:0/587216439 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90005ce0 con 0x7f7e9c06d690 2023-08-11T15:52:20.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.003+0000 7f7e997fa640 1 -- 172.21.15.27:0/587216439 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e90005e40 con 0x7f7e9c06d690 2023-08-11T15:52:20.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.003+0000 7f7e997fa640 1 -- 172.21.15.27:0/587216439 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90007270 con 0x7f7e9c06d690 2023-08-11T15:52:20.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.003+0000 7f7ea2c83640 1 -- 172.21.15.27:0/587216439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c0c1860 con 0x7f7e9c06d690 2023-08-11T15:52:20.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.003+0000 7f7ea2c83640 1 -- 172.21.15.27:0/587216439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c0c1e20 con 0x7f7e9c06d690 2023-08-11T15:52:20.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.005+0000 7f7ea2c83640 1 -- 172.21.15.27:0/587216439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c04e420 con 0x7f7e9c06d690 2023-08-11T15:52:20.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.005+0000 7f7e997fa640 1 -- 172.21.15.27:0/587216439 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7e90007490 con 0x7f7e9c06d690 2023-08-11T15:52:20.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.006+0000 7f7e997fa640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e8806a050 0x7f7e8806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.007+0000 7f7e997fa640 1 -- 172.21.15.27:0/587216439 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7e900884b0 con 0x7f7e9c06d690 2023-08-11T15:52:20.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.007+0000 7f7e9bfff640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e8806a050 0x7f7e8806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.008+0000 7f7e9bfff640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e8806a050 0x7f7e8806c510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c036000 tx=0x7f7e8c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.011+0000 7f7e997fa640 1 -- 172.21.15.27:0/587216439 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f7e900537e0 con 0x7f7e9c06d690 2023-08-11T15:52:20.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.207+0000 7f7ea2c83640 1 -- 172.21.15.27:0/587216439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e9c157b90 con 0x7f7e9c06d690 2023-08-11T15:52:20.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.209+0000 7f7e997fa640 1 -- 172.21.15.27:0/587216439 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e90056e00 con 0x7f7e9c06d690 2023-08-11T15:52:20.210 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:20.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.211+0000 7f7ea2c83640 1 -- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e8806a050 msgr2=0x7f7e8806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.211+0000 7f7ea2c83640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e8806a050 0x7f7e8806c510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c036000 tx=0x7f7e8c03c040 comp rx=0 tx=0).stop 2023-08-11T15:52:20.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.211+0000 7f7ea2c83640 1 -- 172.21.15.27:0/587216439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c06d690 msgr2=0x7f7e9c1440a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.211+0000 7f7ea2c83640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c06d690 0x7f7e9c1440a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000c460 tx=0x7f7e9000c920 comp rx=0 tx=0).stop 2023-08-11T15:52:20.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.212+0000 7f7ea2c83640 1 -- 172.21.15.27:0/587216439 shutdown_connections 2023-08-11T15:52:20.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.212+0000 7f7ea2c83640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e8806a050 0x7f7e8806c510 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.212+0000 7f7ea2c83640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e9c0c8d90 0x7f7e9c158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.212+0000 7f7ea2c83640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c0cbbc0 0x7f7e9c0cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.212+0000 7f7ea2c83640 1 --2- 172.21.15.27:0/587216439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c06d690 0x7f7e9c1440a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.212+0000 7f7ea2c83640 1 -- 172.21.15.27:0/587216439 >> 172.21.15.27:0/587216439 conn(0x7f7e9c0afbe0 msgr2=0x7f7e9c0c7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.212+0000 7f7ea2c83640 1 -- 172.21.15.27:0/587216439 shutdown_connections 2023-08-11T15:52:20.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.212+0000 7f7ea2c83640 1 -- 172.21.15.27:0/587216439 wait complete. 2023-08-11T15:52:20.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:20.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.470+0000 7f05f8d24640 1 Processor -- start 2023-08-11T15:52:20.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.470+0000 7f05f8d24640 1 -- start start 2023-08-11T15:52:20.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.471+0000 7f05f8d24640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f05f415c5b0 0x7f05f40d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.471+0000 7f05f8d24640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05f40cf310 0x7f05f40b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.471+0000 7f05f8d24640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f40d0dc0 con 0x7f05f40d0790 2023-08-11T15:52:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.471+0000 7f05f8d24640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f40d10a0 con 0x7f05f40cf310 2023-08-11T15:52:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.471+0000 7f05f8d24640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f40d1380 con 0x7f05f415c5b0 2023-08-11T15:52:20.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.472+0000 7f05f3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05f40cf310 0x7f05f40b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.472+0000 7f05f2ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f05f40d0790 0x7f05f40cec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47072/0 (socket says 172.21.15.27:47072) 2023-08-11T15:52:20.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.472+0000 7f05f2ffd640 1 -- 172.21.15.27:0/2638051024 learned_addr learned my addr 172.21.15.27:0/2638051024 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.472+0000 7f05f3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05f40cf310 0x7f05f40b51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60250/0 (socket says 172.21.15.27:60250) 2023-08-11T15:52:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.472+0000 7f05f37fe640 1 --2- 172.21.15.27:0/2638051024 >> v2:172.21.15.119:3300/0 conn(0x7f05f415c5b0 0x7f05f40d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.472+0000 7f05f3fff640 1 -- 172.21.15.27:0/2638051024 >> v2:172.21.15.119:3300/0 conn(0x7f05f415c5b0 msgr2=0x7f05f40d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.472+0000 7f05f3fff640 1 --2- 172.21.15.27:0/2638051024 >> v2:172.21.15.119:3300/0 conn(0x7f05f415c5b0 0x7f05f40d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.472+0000 7f05f3fff640 1 -- 172.21.15.27:0/2638051024 >> v1:172.21.15.27:6789/0 conn(0x7f05f40d0790 legacy=0x7f05f40cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.472+0000 7f05f3fff640 1 -- 172.21.15.27:0/2638051024 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40d1660 con 0x7f05f40cf310 2023-08-11T15:52:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.472+0000 7f05f27fc640 1 -- 172.21.15.27:0/2638051024 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335980325 0 0) 0x7f05f40d0dc0 con 0x7f05f40d0790 2023-08-11T15:52:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.473+0000 7f05f3fff640 1 --2- 172.21.15.27:0/2638051024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f40cf310 0x7f05f40b51f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f05f4071f10 tx=0x7f05ec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95c4ff9d19fde2f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f27fc640 1 -- 172.21.15.27:0/2638051024 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec030610 con 0x7f05f40cf310 2023-08-11T15:52:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f27fc640 1 -- 172.21.15.27:0/2638051024 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec030770 con 0x7f05f40cf310 2023-08-11T15:52:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f27fc640 1 -- 172.21.15.27:0/2638051024 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec030b00 con 0x7f05f40cf310 2023-08-11T15:52:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f8d24640 1 -- 172.21.15.27:0/2638051024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f40cf310 msgr2=0x7f05f40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f8d24640 1 --2- 172.21.15.27:0/2638051024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f40cf310 0x7f05f40b51f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f05f4071f10 tx=0x7f05ec02b960 comp rx=0 tx=0).stop 2023-08-11T15:52:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f8d24640 1 -- 172.21.15.27:0/2638051024 shutdown_connections 2023-08-11T15:52:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f8d24640 1 --2- 172.21.15.27:0/2638051024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f40cf310 0x7f05f40b51f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f8d24640 1 --2- 172.21.15.27:0/2638051024 >> v2:172.21.15.119:3300/0 conn(0x7f05f415c5b0 0x7f05f40d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f8d24640 1 -- 172.21.15.27:0/2638051024 >> 172.21.15.27:0/2638051024 conn(0x7f05f406a270 msgr2=0x7f05f406a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f8d24640 1 -- 172.21.15.27:0/2638051024 shutdown_connections 2023-08-11T15:52:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.474+0000 7f05f8d24640 1 -- 172.21.15.27:0/2638051024 wait complete. 2023-08-11T15:52:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.476+0000 7f05f8d24640 1 Processor -- start 2023-08-11T15:52:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.476+0000 7f05f8d24640 1 -- start start 2023-08-11T15:52:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.477+0000 7f05f8d24640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05f40cf310 0x7f05f40c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.477+0000 7f05f8d24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f40d0790 0x7f05f41f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.477+0000 7f05f8d24640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f415c5b0 0x7f05f41540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.477+0000 7f05f8d24640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f4154890 con 0x7f05f415c5b0 2023-08-11T15:52:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.477+0000 7f05f8d24640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f41545f0 con 0x7f05f40d0790 2023-08-11T15:52:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.477+0000 7f05f8d24640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f4156940 con 0x7f05f40cf310 2023-08-11T15:52:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f415c5b0 0x7f05f41540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f415c5b0 0x7f05f41540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60260/0 (socket says 172.21.15.27:60260) 2023-08-11T15:52:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f3fff640 1 -- 172.21.15.27:0/956159987 learned_addr learned my addr 172.21.15.27:0/956159987 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f2ffd640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f40d0790 0x7f05f41f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f37fe640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05f40cf310 0x7f05f40c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f3fff640 1 -- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05f40cf310 msgr2=0x7f05f40c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f3fff640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05f40cf310 0x7f05f40c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f3fff640 1 -- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f40d0790 msgr2=0x7f05f41f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f3fff640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f40d0790 0x7f05f41f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f3fff640 1 -- 172.21.15.27:0/956159987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec009400 con 0x7f05f415c5b0 2023-08-11T15:52:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f2ffd640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f40d0790 0x7f05f41f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.478+0000 7f05f3fff640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f415c5b0 0x7f05f41540b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f05ec02bf70 tx=0x7f05ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.479+0000 7f05ebfff640 1 -- 172.21.15.27:0/956159987 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec03e070 con 0x7f05f415c5b0 2023-08-11T15:52:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.479+0000 7f05ebfff640 1 -- 172.21.15.27:0/956159987 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec006e10 con 0x7f05f415c5b0 2023-08-11T15:52:20.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.479+0000 7f05ebfff640 1 -- 172.21.15.27:0/956159987 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec039bb0 con 0x7f05f415c5b0 2023-08-11T15:52:20.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.479+0000 7f05f8d24640 1 -- 172.21.15.27:0/956159987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f4156c20 con 0x7f05f415c5b0 2023-08-11T15:52:20.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.479+0000 7f05f8d24640 1 -- 172.21.15.27:0/956159987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f41571e0 con 0x7f05f415c5b0 2023-08-11T15:52:20.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.480+0000 7f05f8d24640 1 -- 172.21.15.27:0/956159987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f404e420 con 0x7f05f415c5b0 2023-08-11T15:52:20.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.481+0000 7f05ebfff640 1 -- 172.21.15.27:0/956159987 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f05ec039d10 con 0x7f05f415c5b0 2023-08-11T15:52:20.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.482+0000 7f05ebfff640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f05d406a050 0x7f05d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.483+0000 7f05ebfff640 1 -- 172.21.15.27:0/956159987 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f05ec0aa640 con 0x7f05f415c5b0 2023-08-11T15:52:20.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.483+0000 7f05f37fe640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f05d406a050 0x7f05d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.484+0000 7f05f37fe640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f05d406a050 0x7f05d406c510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f05e4006fd0 tx=0x7f05e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.487+0000 7f05ebfff640 1 -- 172.21.15.27:0/956159987 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f05ec0758d0 con 0x7f05f415c5b0 2023-08-11T15:52:20.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.681+0000 7f05f8d24640 1 -- 172.21.15.27:0/956159987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f05f41573a0 con 0x7f05f415c5b0 2023-08-11T15:52:20.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.682+0000 7f05ebfff640 1 -- 172.21.15.27:0/956159987 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f05ec078ef0 con 0x7f05f415c5b0 2023-08-11T15:52:20.684 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:20.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 -- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f05d406a050 msgr2=0x7f05d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f05d406a050 0x7f05d406c510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f05e4006fd0 tx=0x7f05e4008040 comp rx=0 tx=0).stop 2023-08-11T15:52:20.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 -- 172.21.15.27:0/956159987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f415c5b0 msgr2=0x7f05f41540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f415c5b0 0x7f05f41540b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f05ec02bf70 tx=0x7f05ec0046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 -- 172.21.15.27:0/956159987 shutdown_connections 2023-08-11T15:52:20.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f05d406a050 0x7f05d406c510 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f415c5b0 0x7f05f41540b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05f40d0790 0x7f05f41f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 --2- 172.21.15.27:0/956159987 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05f40cf310 0x7f05f40c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 -- 172.21.15.27:0/956159987 >> 172.21.15.27:0/956159987 conn(0x7f05f406a270 msgr2=0x7f05f4158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 -- 172.21.15.27:0/956159987 shutdown_connections 2023-08-11T15:52:20.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.685+0000 7f05f8d24640 1 -- 172.21.15.27:0/956159987 wait complete. 2023-08-11T15:52:20.707 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:52:20.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.948+0000 7f2d2a66a640 1 Processor -- start 2023-08-11T15:52:20.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.948+0000 7f2d2a66a640 1 -- start start 2023-08-11T15:52:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.949+0000 7f2d2a66a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d240cc8c0 0x7f2d240ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.949+0000 7f2d2a66a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240c2c00 0x7f2d240c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.949+0000 7f2d2a66a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240cd260 con 0x7f2d24142bd0 2023-08-11T15:52:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.949+0000 7f2d2a66a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2415a910 con 0x7f2d240cc8c0 2023-08-11T15:52:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.949+0000 7f2d2a66a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2406a670 con 0x7f2d240c2c00 2023-08-11T15:52:20.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d29668640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2d24142bd0 0x7f2d2415a200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47080/0 (socket says 172.21.15.27:47080) 2023-08-11T15:52:20.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d29668640 1 -- 172.21.15.27:0/2055884053 learned_addr learned my addr 172.21.15.27:0/2055884053 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:20.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d28e67640 1 --2- 172.21.15.27:0/2055884053 >> v2:172.21.15.119:3300/0 conn(0x7f2d240cc8c0 0x7f2d240ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d29e69640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240c2c00 0x7f2d240c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038657645 0 0) 0x7f2d240cd260 con 0x7f2d24142bd0 2023-08-11T15:52:20.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d10003520 con 0x7f2d24142bd0 2023-08-11T15:52:20.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 785336450 0 0) 0x7f2d10003520 con 0x7f2d24142bd0 2023-08-11T15:52:20.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240c2c00 msgr2=0x7f2d240c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d1bfff640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240c2c00 0x7f2d240c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 >> v2:172.21.15.119:3300/0 conn(0x7f2d240cc8c0 msgr2=0x7f2d240ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d1bfff640 1 --2- 172.21.15.27:0/2055884053 >> v2:172.21.15.119:3300/0 conn(0x7f2d240cc8c0 0x7f2d240ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d240c9440 con 0x7f2d24142bd0 2023-08-11T15:52:20.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.950+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f2d20001670 con 0x7f2d24142bd0 2023-08-11T15:52:20.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 >> v1:172.21.15.27:6789/0 conn(0x7f2d24142bd0 legacy=0x7f2d2415a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d1bfff640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d10003970 0x7f2d10003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d1bfff640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240c2c00 0x7f2d100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d1bfff640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 0x7f2d10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2415a910 con 0x7f2d10004d00 2023-08-11T15:52:20.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2406a670 con 0x7f2d10003970 2023-08-11T15:52:20.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1000b980 con 0x7f2d240c2c00 2023-08-11T15:52:20.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d29e69640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 0x7f2d10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d29668640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d10003970 0x7f2d10003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d28e67640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240c2c00 0x7f2d100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d29e69640 1 -- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240c2c00 msgr2=0x7f2d100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d29e69640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240c2c00 0x7f2d100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d29e69640 1 -- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d10003970 msgr2=0x7f2d10003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d29e69640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d10003970 0x7f2d10003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d29e69640 1 -- 172.21.15.27:0/2055884053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d10003520 con 0x7f2d10004d00 2023-08-11T15:52:20.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.951+0000 7f2d28e67640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240c2c00 0x7f2d100047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:20.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.952+0000 7f2d29668640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d10003970 0x7f2d10003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:20.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.953+0000 7f2d29e69640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 0x7f2d10005180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c007390 tx=0x7f2d100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0611ab08cbcba38 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.953+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c02ccb0 con 0x7f2d10004d00 2023-08-11T15:52:20.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.953+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c02ce10 con 0x7f2d10004d00 2023-08-11T15:52:20.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.953+0000 7f2d1bfff640 1 -- 172.21.15.27:0/2055884053 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c034660 con 0x7f2d10004d00 2023-08-11T15:52:20.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.953+0000 7f2d2a66a640 1 -- 172.21.15.27:0/2055884053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 msgr2=0x7f2d10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.953+0000 7f2d2a66a640 1 --2- 172.21.15.27:0/2055884053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 0x7f2d10005180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c007390 tx=0x7f2d100057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.953+0000 7f2d29e69640 1 -- 172.21.15.27:0/2055884053 reap_dead start 2023-08-11T15:52:20.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.954+0000 7f2d2a66a640 1 -- 172.21.15.27:0/2055884053 shutdown_connections 2023-08-11T15:52:20.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.954+0000 7f2d2a66a640 1 -- 172.21.15.27:0/2055884053 >> 172.21.15.27:0/2055884053 conn(0x7f2d24069800 msgr2=0x7f2d24069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.954+0000 7f2d2a66a640 1 -- 172.21.15.27:0/2055884053 shutdown_connections 2023-08-11T15:52:20.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.954+0000 7f2d2a66a640 1 -- 172.21.15.27:0/2055884053 wait complete. 2023-08-11T15:52:20.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.956+0000 7f2d2a66a640 1 Processor -- start 2023-08-11T15:52:20.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.956+0000 7f2d2a66a640 1 -- start start 2023-08-11T15:52:20.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d2a66a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 0x7f2d241f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d2a66a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d240a96a0 0x7f2d24141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d2a66a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240a74c0 0x7f2d240c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d2a66a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241f5100 con 0x7f2d10004d00 2023-08-11T15:52:20.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d2a66a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240a9b20 con 0x7f2d240a96a0 2023-08-11T15:52:20.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d2a66a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240a9200 con 0x7f2d240a74c0 2023-08-11T15:52:20.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d29668640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 0x7f2d241f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d29668640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 0x7f2d241f19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60284/0 (socket says 172.21.15.27:60284) 2023-08-11T15:52:20.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d29668640 1 -- 172.21.15.27:0/4286119238 learned_addr learned my addr 172.21.15.27:0/4286119238 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:20.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d29668640 1 -- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240a74c0 msgr2=0x7f2d240c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d29668640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240a74c0 0x7f2d240c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d29668640 1 -- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d240a96a0 msgr2=0x7f2d24141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d29668640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d240a96a0 0x7f2d24141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.957+0000 7f2d29668640 1 -- 172.21.15.27:0/4286119238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d241f1ff0 con 0x7f2d10004d00 2023-08-11T15:52:20.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.958+0000 7f2d29668640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 0x7f2d241f19e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d8b0 tx=0x7f2d2000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.958+0000 7f2d19ffb640 1 -- 172.21.15.27:0/4286119238 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200074e0 con 0x7f2d10004d00 2023-08-11T15:52:20.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.958+0000 7f2d19ffb640 1 -- 172.21.15.27:0/4286119238 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20007640 con 0x7f2d10004d00 2023-08-11T15:52:20.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.958+0000 7f2d19ffb640 1 -- 172.21.15.27:0/4286119238 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200049e0 con 0x7f2d10004d00 2023-08-11T15:52:20.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.958+0000 7f2d2a66a640 1 -- 172.21.15.27:0/4286119238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d241f22d0 con 0x7f2d10004d00 2023-08-11T15:52:20.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.958+0000 7f2d2a66a640 1 -- 172.21.15.27:0/4286119238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d241f2890 con 0x7f2d10004d00 2023-08-11T15:52:20.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.959+0000 7f2d2a66a640 1 -- 172.21.15.27:0/4286119238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d24066e80 con 0x7f2d10004d00 2023-08-11T15:52:20.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.960+0000 7f2d19ffb640 1 -- 172.21.15.27:0/4286119238 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2d20004b40 con 0x7f2d10004d00 2023-08-11T15:52:20.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.962+0000 7f2d19ffb640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2d0806a050 0x7f2d0806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.962+0000 7f2d19ffb640 1 -- 172.21.15.27:0/4286119238 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2d20088310 con 0x7f2d10004d00 2023-08-11T15:52:20.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.962+0000 7f2d28e67640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2d0806a050 0x7f2d0806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.963+0000 7f2d28e67640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2d0806a050 0x7f2d0806c510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d240ae580 tx=0x7f2d14008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:20.966+0000 7f2d19ffb640 1 -- 172.21.15.27:0/4286119238 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2d20053770 con 0x7f2d10004d00 2023-08-11T15:52:21.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.174+0000 7f2d2a66a640 1 -- 172.21.15.27:0/4286119238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2d240ae2d0 con 0x7f2d10004d00 2023-08-11T15:52:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.175+0000 7f2d19ffb640 1 -- 172.21.15.27:0/4286119238 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2d20056d90 con 0x7f2d10004d00 2023-08-11T15:52:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:52:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.177+0000 7f2d2a66a640 1 -- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2d0806a050 msgr2=0x7f2d0806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.177+0000 7f2d2a66a640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2d0806a050 0x7f2d0806c510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d240ae580 tx=0x7f2d14008040 comp rx=0 tx=0).stop 2023-08-11T15:52:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.177+0000 7f2d2a66a640 1 -- 172.21.15.27:0/4286119238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 msgr2=0x7f2d241f19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.177+0000 7f2d2a66a640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 0x7f2d241f19e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d8b0 tx=0x7f2d2000dd70 comp rx=0 tx=0).stop 2023-08-11T15:52:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.178+0000 7f2d2a66a640 1 -- 172.21.15.27:0/4286119238 shutdown_connections 2023-08-11T15:52:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.178+0000 7f2d2a66a640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2d0806a050 0x7f2d0806c510 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.178+0000 7f2d2a66a640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d240a74c0 0x7f2d240c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.178+0000 7f2d2a66a640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d240a96a0 0x7f2d24141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.178+0000 7f2d2a66a640 1 --2- 172.21.15.27:0/4286119238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d10004d00 0x7f2d241f19e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.178+0000 7f2d2a66a640 1 -- 172.21.15.27:0/4286119238 >> 172.21.15.27:0/4286119238 conn(0x7f2d24069800 msgr2=0x7f2d240cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.178+0000 7f2d2a66a640 1 -- 172.21.15.27:0/4286119238 shutdown_connections 2023-08-11T15:52:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.178+0000 7f2d2a66a640 1 -- 172.21.15.27:0/4286119238 wait complete. 2023-08-11T15:52:21.200 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-08-11T15:52:21.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:52:21.200 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-11T15:52:21.316 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:52:21.316 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:52:21.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:52:21.317 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-11T15:52:21.377 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:52:21.377 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:21.438 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:52:21.439 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:21.493 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:21.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.728+0000 7fc4ab414640 1 Processor -- start 2023-08-11T15:52:21.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.728+0000 7fc4ab414640 1 -- start start 2023-08-11T15:52:21.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.730+0000 7fc4ab414640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4a415a140 0x7fc4a406b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:21.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.730+0000 7fc4ab414640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc4a406f5d0 0x7fc4a406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:21.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.730+0000 7fc4ab414640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a40c8a30 con 0x7fc4a40cb430 2023-08-11T15:52:21.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.730+0000 7fc4ab414640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a406ffa0 con 0x7fc4a406f5d0 2023-08-11T15:52:21.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.730+0000 7fc4ab414640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a40b1580 con 0x7fc4a415a140 2023-08-11T15:52:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.730+0000 7fc4a89af640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc4a40cb430 0x7fc4a40b0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47082/0 (socket says 172.21.15.27:47082) 2023-08-11T15:52:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.730+0000 7fc4a89af640 1 -- 172.21.15.27:0/2551356110 learned_addr learned my addr 172.21.15.27:0/2551356110 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.731+0000 7fc4a99b1640 1 --2- 172.21.15.27:0/2551356110 >> v2:172.21.15.27:3300/0 conn(0x7fc4a406f5d0 0x7fc4a406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.731+0000 7fc4a91b0640 1 --2- 172.21.15.27:0/2551356110 >> v2:172.21.15.119:3300/0 conn(0x7fc4a415a140 0x7fc4a406b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.731+0000 7fc49bfff640 1 -- 172.21.15.27:0/2551356110 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245451661 0 0) 0x7fc4a40c8a30 con 0x7fc4a40cb430 2023-08-11T15:52:21.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.731+0000 7fc49bfff640 1 -- 172.21.15.27:0/2551356110 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc490003520 con 0x7fc4a40cb430 2023-08-11T15:52:21.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.731+0000 7fc4a99b1640 1 -- 172.21.15.27:0/2551356110 >> v2:172.21.15.119:3300/0 conn(0x7fc4a415a140 msgr2=0x7fc4a406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.731+0000 7fc4a99b1640 1 --2- 172.21.15.27:0/2551356110 >> v2:172.21.15.119:3300/0 conn(0x7fc4a415a140 0x7fc4a406b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.731+0000 7fc4a99b1640 1 -- 172.21.15.27:0/2551356110 >> v1:172.21.15.27:6789/0 conn(0x7fc4a40cb430 legacy=0x7fc4a40b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:21.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.731+0000 7fc4a99b1640 1 -- 172.21.15.27:0/2551356110 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a406c0c0 con 0x7fc4a406f5d0 2023-08-11T15:52:21.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.732+0000 7fc4a99b1640 1 --2- 172.21.15.27:0/2551356110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a406f5d0 0x7fc4a406fa60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc4a406dae0 tx=0x7fc49c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222be71bd7f486cc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:21.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.733+0000 7fc49bfff640 1 -- 172.21.15.27:0/2551356110 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c02cc60 con 0x7fc4a406f5d0 2023-08-11T15:52:21.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.733+0000 7fc49bfff640 1 -- 172.21.15.27:0/2551356110 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c02cdc0 con 0x7fc4a406f5d0 2023-08-11T15:52:21.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.733+0000 7fc49bfff640 1 -- 172.21.15.27:0/2551356110 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c034660 con 0x7fc4a406f5d0 2023-08-11T15:52:21.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.733+0000 7fc4ab414640 1 -- 172.21.15.27:0/2551356110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a406f5d0 msgr2=0x7fc4a406fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:21.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.733+0000 7fc4ab414640 1 --2- 172.21.15.27:0/2551356110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a406f5d0 0x7fc4a406fa60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc4a406dae0 tx=0x7fc49c02beb0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.733+0000 7fc4ab414640 1 -- 172.21.15.27:0/2551356110 shutdown_connections 2023-08-11T15:52:21.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.733+0000 7fc4ab414640 1 --2- 172.21.15.27:0/2551356110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a406f5d0 0x7fc4a406fa60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.733+0000 7fc4ab414640 1 --2- 172.21.15.27:0/2551356110 >> v2:172.21.15.119:3300/0 conn(0x7fc4a415a140 0x7fc4a406b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.733+0000 7fc4ab414640 1 -- 172.21.15.27:0/2551356110 >> 172.21.15.27:0/2551356110 conn(0x7fc4a40afb70 msgr2=0x7fc4a40aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:21.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.733+0000 7fc4ab414640 1 -- 172.21.15.27:0/2551356110 shutdown_connections 2023-08-11T15:52:21.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.734+0000 7fc4ab414640 1 -- 172.21.15.27:0/2551356110 wait complete. 2023-08-11T15:52:21.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.735+0000 7fc4ab414640 1 Processor -- start 2023-08-11T15:52:21.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.735+0000 7fc4ab414640 1 -- start start 2023-08-11T15:52:21.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.736+0000 7fc4ab414640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a406f5d0 0x7fc4a40c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:21.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.736+0000 7fc4ab414640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4a40cb430 0x7fc4a40c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:21.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.736+0000 7fc4ab414640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a415a140 0x7fc4a415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:21.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.736+0000 7fc4ab414640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40c7fd0 con 0x7fc4a415a140 2023-08-11T15:52:21.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.736+0000 7fc4ab414640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40c73c0 con 0x7fc4a406f5d0 2023-08-11T15:52:21.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.736+0000 7fc4ab414640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40c76a0 con 0x7fc4a40cb430 2023-08-11T15:52:21.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.737+0000 7fc4a99b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a415a140 0x7fc4a415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:21.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.737+0000 7fc4a99b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a415a140 0x7fc4a415c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60304/0 (socket says 172.21.15.27:60304) 2023-08-11T15:52:21.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.737+0000 7fc4a99b1640 1 -- 172.21.15.27:0/3261853540 learned_addr learned my addr 172.21.15.27:0/3261853540 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:21.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.737+0000 7fc4a91b0640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a406f5d0 0x7fc4a40c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:21.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.737+0000 7fc4a89af640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4a40cb430 0x7fc4a40c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:21.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.737+0000 7fc4a99b1640 1 -- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4a40cb430 msgr2=0x7fc4a40c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.737+0000 7fc4a99b1640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4a40cb430 0x7fc4a40c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.737+0000 7fc4a99b1640 1 -- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a406f5d0 msgr2=0x7fc4a40c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.737+0000 7fc4a99b1640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a406f5d0 0x7fc4a40c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.737+0000 7fc4a99b1640 1 -- 172.21.15.27:0/3261853540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c009a70 con 0x7fc4a415a140 2023-08-11T15:52:21.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.738+0000 7fc4a99b1640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a415a140 0x7fc4a415c030 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009d90 tx=0x7fc49c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:21.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.738+0000 7fc499ffb640 1 -- 172.21.15.27:0/3261853540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c040650 con 0x7fc4a415a140 2023-08-11T15:52:21.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.738+0000 7fc499ffb640 1 -- 172.21.15.27:0/3261853540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c0407b0 con 0x7fc4a415a140 2023-08-11T15:52:21.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.738+0000 7fc499ffb640 1 -- 172.21.15.27:0/3261853540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c040b60 con 0x7fc4a415a140 2023-08-11T15:52:21.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.738+0000 7fc4ab414640 1 -- 172.21.15.27:0/3261853540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a40c7980 con 0x7fc4a415a140 2023-08-11T15:52:21.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.738+0000 7fc4ab414640 1 -- 172.21.15.27:0/3261853540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a40c47f0 con 0x7fc4a415a140 2023-08-11T15:52:21.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.740+0000 7fc4ab414640 1 -- 172.21.15.27:0/3261853540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a4073990 con 0x7fc4a415a140 2023-08-11T15:52:21.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.741+0000 7fc499ffb640 1 -- 172.21.15.27:0/3261853540 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc49c040cc0 con 0x7fc4a415a140 2023-08-11T15:52:21.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.742+0000 7fc499ffb640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc48806a050 0x7fc48806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:21.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.742+0000 7fc499ffb640 1 -- 172.21.15.27:0/3261853540 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc49c0ab630 con 0x7fc4a415a140 2023-08-11T15:52:21.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.743+0000 7fc4a91b0640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc48806a050 0x7fc48806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:21.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.744+0000 7fc4a91b0640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc48806a050 0x7fc48806c510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009960 tx=0x7fc4a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:21.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.746+0000 7fc499ffb640 1 -- 172.21.15.27:0/3261853540 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fc49c076a70 con 0x7fc4a415a140 2023-08-11T15:52:21.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.947+0000 7fc4ab414640 1 -- 172.21.15.27:0/3261853540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4a40af950 con 0x7fc4a415a140 2023-08-11T15:52:21.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.948+0000 7fc499ffb640 1 -- 172.21.15.27:0/3261853540 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7fc49c009450 con 0x7fc4a415a140 2023-08-11T15:52:21.951 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:21.952 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:21.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 -- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc48806a050 msgr2=0x7fc48806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:21.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc48806a050 0x7fc48806c510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009960 tx=0x7fc4a0008040 comp rx=0 tx=0).stop 2023-08-11T15:52:21.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 -- 172.21.15.27:0/3261853540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a415a140 msgr2=0x7fc4a415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:21.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a415a140 0x7fc4a415c030 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009d90 tx=0x7fc49c004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:21.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 -- 172.21.15.27:0/3261853540 shutdown_connections 2023-08-11T15:52:21.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc48806a050 0x7fc48806c510 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4a40cb430 0x7fc4a40c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a415a140 0x7fc4a415c030 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 --2- 172.21.15.27:0/3261853540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4a406f5d0 0x7fc4a40c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:21.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 -- 172.21.15.27:0/3261853540 >> 172.21.15.27:0/3261853540 conn(0x7fc4a40afb70 msgr2=0x7fc4a40b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:21.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.951+0000 7fc4ab414640 1 -- 172.21.15.27:0/3261853540 shutdown_connections 2023-08-11T15:52:21.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:21.952+0000 7fc4ab414640 1 -- 172.21.15.27:0/3261853540 wait complete. 2023-08-11T15:52:21.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:21.974 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.214+0000 7fde3a5cf640 1 Processor -- start 2023-08-11T15:52:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.214+0000 7fde3a5cf640 1 -- start start 2023-08-11T15:52:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.216+0000 7fde3a5cf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde34071680 0x7fde3415cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.216+0000 7fde3a5cf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde340cbae0 con 0x7fde34070780 2023-08-11T15:52:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.216+0000 7fde3a5cf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3415d1a0 con 0x7fde34069800 2023-08-11T15:52:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.216+0000 7fde3a5cf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3415d480 con 0x7fde34071680 2023-08-11T15:52:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.216+0000 7fde337fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fde34070780 0x7fde340b3fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47088/0 (socket says 172.21.15.27:47088) 2023-08-11T15:52:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.216+0000 7fde337fe640 1 -- 172.21.15.27:0/812065367 learned_addr learned my addr 172.21.15.27:0/812065367 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.217+0000 7fde33fff640 1 --2- 172.21.15.27:0/812065367 >> v2:172.21.15.119:3300/0 conn(0x7fde34071680 0x7fde3415cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.217+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906743200 0 0) 0x7fde340cbae0 con 0x7fde34070780 2023-08-11T15:52:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.217+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde20003520 con 0x7fde34070780 2023-08-11T15:52:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.217+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1251167139 0 0) 0x7fde20003520 con 0x7fde34070780 2023-08-11T15:52:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.217+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 >> v2:172.21.15.119:3300/0 conn(0x7fde34071680 msgr2=0x7fde3415cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:22.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.217+0000 7fde32ffd640 1 --2- 172.21.15.27:0/812065367 >> v2:172.21.15.119:3300/0 conn(0x7fde34071680 0x7fde3415cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.217+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 >> v1:172.21.15.119:6789/0 conn(0x7fde34069800 legacy=0x7fde340cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.217+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3415d760 con 0x7fde34070780 2023-08-11T15:52:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fde1c001670 con 0x7fde34070780 2023-08-11T15:52:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 >> v1:172.21.15.27:6789/0 conn(0x7fde34070780 legacy=0x7fde340b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde32ffd640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde20003970 0x7fde20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde32ffd640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fde200047c0 0x7fde20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde32ffd640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde20005180 0x7fde2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3415d480 con 0x7fde20005180 2023-08-11T15:52:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2000c020 con 0x7fde20003970 2023-08-11T15:52:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2000c1e0 con 0x7fde200047c0 2023-08-11T15:52:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde38b6c640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde20005180 0x7fde2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde33fff640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde20003970 0x7fde20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde337fe640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fde200047c0 0x7fde20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde38b6c640 1 -- 172.21.15.27:0/812065367 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fde200047c0 msgr2=0x7fde20004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.218+0000 7fde38b6c640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fde200047c0 0x7fde20004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.219+0000 7fde38b6c640 1 -- 172.21.15.27:0/812065367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde20003970 msgr2=0x7fde20003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.219+0000 7fde38b6c640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde20003970 0x7fde20003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.219+0000 7fde38b6c640 1 -- 172.21.15.27:0/812065367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde20003520 con 0x7fde20005180 2023-08-11T15:52:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.219+0000 7fde38b6c640 1 -- 172.21.15.27:0/812065367 reap_dead start 2023-08-11T15:52:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.220+0000 7fde38b6c640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde20005180 0x7fde2000b980 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fde2c001750 tx=0x7fde200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6db9db89d4d1c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.220+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c0052d0 con 0x7fde20005180 2023-08-11T15:52:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.220+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c02e050 con 0x7fde20005180 2023-08-11T15:52:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.220+0000 7fde32ffd640 1 -- 172.21.15.27:0/812065367 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c003830 con 0x7fde20005180 2023-08-11T15:52:22.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.220+0000 7fde3a5cf640 1 -- 172.21.15.27:0/812065367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde20005180 msgr2=0x7fde2000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.220+0000 7fde3a5cf640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde20005180 0x7fde2000b980 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fde2c001750 tx=0x7fde200057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.221+0000 7fde3a5cf640 1 -- 172.21.15.27:0/812065367 shutdown_connections 2023-08-11T15:52:22.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.221+0000 7fde3a5cf640 1 --2- 172.21.15.27:0/812065367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde20005180 0x7fde2000b980 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.221+0000 7fde3a5cf640 1 -- 172.21.15.27:0/812065367 >> 172.21.15.27:0/812065367 conn(0x7fde340b0dd0 msgr2=0x7fde340b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:22.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.221+0000 7fde3a5cf640 1 -- 172.21.15.27:0/812065367 shutdown_connections 2023-08-11T15:52:22.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.221+0000 7fde3a5cf640 1 -- 172.21.15.27:0/812065367 wait complete. 2023-08-11T15:52:22.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.223+0000 7fde3a5cf640 1 Processor -- start 2023-08-11T15:52:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.223+0000 7fde3a5cf640 1 -- start start 2023-08-11T15:52:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.224+0000 7fde3a5cf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fde20005180 0x7fde340ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.224+0000 7fde3a5cf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde34140770 0x7fde34140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.224+0000 7fde3a5cf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde341f5aa0 0x7fde341f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.224+0000 7fde3a5cf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34141130 con 0x7fde34140770 2023-08-11T15:52:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.224+0000 7fde3a5cf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde341583f0 con 0x7fde341f5aa0 2023-08-11T15:52:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.224+0000 7fde3a5cf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde341589b0 con 0x7fde20005180 2023-08-11T15:52:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.224+0000 7fde337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde34140770 0x7fde34140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.224+0000 7fde337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde34140770 0x7fde34140bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60330/0 (socket says 172.21.15.27:60330) 2023-08-11T15:52:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.224+0000 7fde337fe640 1 -- 172.21.15.27:0/1641219229 learned_addr learned my addr 172.21.15.27:0/1641219229 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.224+0000 7fde33fff640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fde20005180 0x7fde340ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde38b6c640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde341f5aa0 0x7fde341f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde337fe640 1 -- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fde20005180 msgr2=0x7fde340ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde337fe640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fde20005180 0x7fde340ad3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde337fe640 1 -- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde341f5aa0 msgr2=0x7fde341f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde337fe640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde341f5aa0 0x7fde341f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde337fe640 1 -- 172.21.15.27:0/1641219229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde340adae0 con 0x7fde34140770 2023-08-11T15:52:22.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde337fe640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde34140770 0x7fde34140bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00d770 tx=0x7fde1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:22.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde30ff9640 1 -- 172.21.15.27:0/1641219229 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0074e0 con 0x7fde34140770 2023-08-11T15:52:22.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde30ff9640 1 -- 172.21.15.27:0/1641219229 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c007660 con 0x7fde34140770 2023-08-11T15:52:22.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde30ff9640 1 -- 172.21.15.27:0/1641219229 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c004a20 con 0x7fde34140770 2023-08-11T15:52:22.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.225+0000 7fde3a5cf640 1 -- 172.21.15.27:0/1641219229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde340addc0 con 0x7fde34140770 2023-08-11T15:52:22.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.226+0000 7fde3a5cf640 1 -- 172.21.15.27:0/1641219229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3414eb50 con 0x7fde34140770 2023-08-11T15:52:22.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.227+0000 7fde3a5cf640 1 -- 172.21.15.27:0/1641219229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3404e420 con 0x7fde34140770 2023-08-11T15:52:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.228+0000 7fde30ff9640 1 -- 172.21.15.27:0/1641219229 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fde1c004b80 con 0x7fde34140770 2023-08-11T15:52:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.229+0000 7fde30ff9640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fde1406a050 0x7fde1406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.229+0000 7fde30ff9640 1 -- 172.21.15.27:0/1641219229 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fde1c0881a0 con 0x7fde34140770 2023-08-11T15:52:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.230+0000 7fde33fff640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fde1406a050 0x7fde1406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.231+0000 7fde33fff640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fde1406a050 0x7fde1406c510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fde24009fd0 tx=0x7fde2400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:22.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.233+0000 7fde30ff9640 1 -- 172.21.15.27:0/1641219229 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fde1c0534c0 con 0x7fde34140770 2023-08-11T15:52:22.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.426+0000 7fde3a5cf640 1 -- 172.21.15.27:0/1641219229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde34158af0 con 0x7fde34140770 2023-08-11T15:52:22.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.427+0000 7fde30ff9640 1 -- 172.21.15.27:0/1641219229 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde1c056ae0 con 0x7fde34140770 2023-08-11T15:52:22.429 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:22.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 -- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fde1406a050 msgr2=0x7fde1406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fde1406a050 0x7fde1406c510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fde24009fd0 tx=0x7fde2400b040 comp rx=0 tx=0).stop 2023-08-11T15:52:22.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 -- 172.21.15.27:0/1641219229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde34140770 msgr2=0x7fde34140bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde34140770 0x7fde34140bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00d770 tx=0x7fde1c00dc30 comp rx=0 tx=0).stop 2023-08-11T15:52:22.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 -- 172.21.15.27:0/1641219229 shutdown_connections 2023-08-11T15:52:22.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fde1406a050 0x7fde1406c510 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde341f5aa0 0x7fde341f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde34140770 0x7fde34140bf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 --2- 172.21.15.27:0/1641219229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fde20005180 0x7fde340ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 -- 172.21.15.27:0/1641219229 >> 172.21.15.27:0/1641219229 conn(0x7fde340b0dd0 msgr2=0x7fde340b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:22.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 -- 172.21.15.27:0/1641219229 shutdown_connections 2023-08-11T15:52:22.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.430+0000 7fde3a5cf640 1 -- 172.21.15.27:0/1641219229 wait complete. 2023-08-11T15:52:22.450 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.693+0000 7fbc78cec640 1 Processor -- start 2023-08-11T15:52:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.693+0000 7fbc78cec640 1 -- start start 2023-08-11T15:52:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.695+0000 7fbc78cec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbc740cb9c0 0x7fbc740cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.695+0000 7fbc78cec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc740cc530 con 0x7fbc74143e40 2023-08-11T15:52:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.695+0000 7fbc78cec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc740cc810 con 0x7fbc74069800 2023-08-11T15:52:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.695+0000 7fbc78cec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc740ce090 con 0x7fbc740cb9c0 2023-08-11T15:52:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.696+0000 7fbc7259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbc74143e40 0x7fbc740ce890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47092/0 (socket says 172.21.15.27:47092) 2023-08-11T15:52:22.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.696+0000 7fbc7259c640 1 -- 172.21.15.27:0/3146333265 learned_addr learned my addr 172.21.15.27:0/3146333265 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:22.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.696+0000 7fbc6bfff640 1 --2- 172.21.15.27:0/3146333265 >> v2:172.21.15.27:3300/0 conn(0x7fbc740cb9c0 0x7fbc740cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.696+0000 7fbc6bfff640 1 -- 172.21.15.27:0/3146333265 >> v1:172.21.15.119:6789/0 conn(0x7fbc74069800 legacy=0x7fbc74069c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:22.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.696+0000 7fbc6bfff640 1 -- 172.21.15.27:0/3146333265 >> v1:172.21.15.27:6789/0 conn(0x7fbc74143e40 legacy=0x7fbc740ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.696+0000 7fbc6bfff640 1 -- 172.21.15.27:0/3146333265 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc740ce370 con 0x7fbc740cb9c0 2023-08-11T15:52:22.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.697+0000 7fbc6bfff640 1 --2- 172.21.15.27:0/3146333265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc740cb9c0 0x7fbc740cbe50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0097a0 tx=0x7fbc5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0e3b249a61651a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:22.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.698+0000 7fbc71d9b640 1 -- 172.21.15.27:0/3146333265 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c031830 con 0x7fbc740cb9c0 2023-08-11T15:52:22.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.698+0000 7fbc71d9b640 1 -- 172.21.15.27:0/3146333265 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c031990 con 0x7fbc740cb9c0 2023-08-11T15:52:22.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.698+0000 7fbc71d9b640 1 -- 172.21.15.27:0/3146333265 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c031d20 con 0x7fbc740cb9c0 2023-08-11T15:52:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.698+0000 7fbc78cec640 1 -- 172.21.15.27:0/3146333265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc740cb9c0 msgr2=0x7fbc740cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.698+0000 7fbc78cec640 1 --2- 172.21.15.27:0/3146333265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc740cb9c0 0x7fbc740cbe50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0097a0 tx=0x7fbc5c02b8d0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.698+0000 7fbc78cec640 1 -- 172.21.15.27:0/3146333265 shutdown_connections 2023-08-11T15:52:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.698+0000 7fbc78cec640 1 --2- 172.21.15.27:0/3146333265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc740cb9c0 0x7fbc740cbe50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.698+0000 7fbc78cec640 1 -- 172.21.15.27:0/3146333265 >> 172.21.15.27:0/3146333265 conn(0x7fbc740b0dd0 msgr2=0x7fbc740b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.698+0000 7fbc78cec640 1 -- 172.21.15.27:0/3146333265 shutdown_connections 2023-08-11T15:52:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.699+0000 7fbc78cec640 1 -- 172.21.15.27:0/3146333265 wait complete. 2023-08-11T15:52:22.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.700+0000 7fbc78cec640 1 Processor -- start 2023-08-11T15:52:22.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.700+0000 7fbc78cec640 1 -- start start 2023-08-11T15:52:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.701+0000 7fbc78cec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74069800 0x7fbc741ef550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.701+0000 7fbc78cec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc740cb9c0 0x7fbc7414c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.701+0000 7fbc78cec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc74143e40 0x7fbc740c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.701+0000 7fbc78cec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7415bf70 con 0x7fbc74069800 2023-08-11T15:52:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.701+0000 7fbc78cec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7415c250 con 0x7fbc740cb9c0 2023-08-11T15:52:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.701+0000 7fbc78cec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7415c530 con 0x7fbc74143e40 2023-08-11T15:52:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.702+0000 7fbc7259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74069800 0x7fbc741ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.702+0000 7fbc7259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74069800 0x7fbc741ef550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60352/0 (socket says 172.21.15.27:60352) 2023-08-11T15:52:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.702+0000 7fbc7259c640 1 -- 172.21.15.27:0/2408654852 learned_addr learned my addr 172.21.15.27:0/2408654852 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.702+0000 7fbc7259c640 1 -- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc74143e40 msgr2=0x7fbc740c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.702+0000 7fbc7259c640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc74143e40 0x7fbc740c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.702+0000 7fbc7259c640 1 -- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc740cb9c0 msgr2=0x7fbc7414c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.702+0000 7fbc7259c640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc740cb9c0 0x7fbc7414c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.702+0000 7fbc7259c640 1 -- 172.21.15.27:0/2408654852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c009400 con 0x7fbc74069800 2023-08-11T15:52:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.703+0000 7fbc7259c640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74069800 0x7fbc741ef550 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400c480 tx=0x7fbc6400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.703+0000 7fbc6affd640 1 -- 172.21.15.27:0/2408654852 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64005ce0 con 0x7fbc74069800 2023-08-11T15:52:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.703+0000 7fbc6affd640 1 -- 172.21.15.27:0/2408654852 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc64005e40 con 0x7fbc74069800 2023-08-11T15:52:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.703+0000 7fbc6affd640 1 -- 172.21.15.27:0/2408654852 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc640072f0 con 0x7fbc74069800 2023-08-11T15:52:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.703+0000 7fbc78cec640 1 -- 172.21.15.27:0/2408654852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7415c810 con 0x7fbc74069800 2023-08-11T15:52:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.703+0000 7fbc78cec640 1 -- 172.21.15.27:0/2408654852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7415bc20 con 0x7fbc74069800 2023-08-11T15:52:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.705+0000 7fbc78cec640 1 -- 172.21.15.27:0/2408654852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7415cd30 con 0x7fbc74069800 2023-08-11T15:52:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.705+0000 7fbc6affd640 1 -- 172.21.15.27:0/2408654852 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbc64007510 con 0x7fbc74069800 2023-08-11T15:52:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.706+0000 7fbc6affd640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc5406a050 0x7fbc5406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.706+0000 7fbc6affd640 1 -- 172.21.15.27:0/2408654852 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbc64087d10 con 0x7fbc74069800 2023-08-11T15:52:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.707+0000 7fbc6bfff640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc5406a050 0x7fbc5406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.708+0000 7fbc6bfff640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc5406a050 0x7fbc5406c510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c036000 tx=0x7fbc5c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.711+0000 7fbc6affd640 1 -- 172.21.15.27:0/2408654852 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fbc7415cd30 con 0x7fbc74069800 2023-08-11T15:52:22.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.904+0000 7fbc78cec640 1 -- 172.21.15.27:0/2408654852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbc7415cd30 con 0x7fbc74069800 2023-08-11T15:52:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.905+0000 7fbc6affd640 1 -- 172.21.15.27:0/2408654852 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbc640170a0 con 0x7fbc74069800 2023-08-11T15:52:22.907 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:22.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 -- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc5406a050 msgr2=0x7fbc5406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc5406a050 0x7fbc5406c510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c036000 tx=0x7fbc5c030000 comp rx=0 tx=0).stop 2023-08-11T15:52:22.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 -- 172.21.15.27:0/2408654852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74069800 msgr2=0x7fbc741ef550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:22.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74069800 0x7fbc741ef550 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400c480 tx=0x7fbc6400c940 comp rx=0 tx=0).stop 2023-08-11T15:52:22.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 -- 172.21.15.27:0/2408654852 shutdown_connections 2023-08-11T15:52:22.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc5406a050 0x7fbc5406c510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc74143e40 0x7fbc740c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc740cb9c0 0x7fbc7414c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 --2- 172.21.15.27:0/2408654852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74069800 0x7fbc741ef550 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:22.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 -- 172.21.15.27:0/2408654852 >> 172.21.15.27:0/2408654852 conn(0x7fbc740b0dd0 msgr2=0x7fbc740b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:22.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.908+0000 7fbc78cec640 1 -- 172.21.15.27:0/2408654852 shutdown_connections 2023-08-11T15:52:22.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:22.909+0000 7fbc78cec640 1 -- 172.21.15.27:0/2408654852 wait complete. 2023-08-11T15:52:22.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:23.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.159+0000 7f9664960640 1 Processor -- start 2023-08-11T15:52:23.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.159+0000 7f9664960640 1 -- start start 2023-08-11T15:52:23.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.161+0000 7f9664960640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96600ccf40 0x7f96601435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.161+0000 7f9664960640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966006c500 con 0x7f966015a0f0 2023-08-11T15:52:23.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.161+0000 7f9664960640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966006b8f0 con 0x7f966015a4d0 2023-08-11T15:52:23.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.161+0000 7f9664960640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966006bbd0 con 0x7f96600ccf40 2023-08-11T15:52:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.162+0000 7f965f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f966015a0f0 0x7f9660071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47106/0 (socket says 172.21.15.27:47106) 2023-08-11T15:52:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.162+0000 7f965f7fe640 1 -- 172.21.15.27:0/1104805120 learned_addr learned my addr 172.21.15.27:0/1104805120 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.162+0000 7f965ffff640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96600ccf40 0x7f96601435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.162+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478186517 0 0) 0x7f966006c500 con 0x7f966015a0f0 2023-08-11T15:52:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.162+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9648003520 con 0x7f966015a0f0 2023-08-11T15:52:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 760576264 0 0) 0x7f9648003520 con 0x7f966015a0f0 2023-08-11T15:52:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96600ccf40 msgr2=0x7f96601435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96600ccf40 0x7f96601435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 >> v1:172.21.15.119:6789/0 conn(0x7f966015a4d0 legacy=0x7f966006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966006beb0 con 0x7f966015a0f0 2023-08-11T15:52:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f9650001670 con 0x7f966015a0f0 2023-08-11T15:52:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 >> v1:172.21.15.27:6789/0 conn(0x7f966015a0f0 legacy=0x7f9660071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480039a0 0x7f9648003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96600ccf40 0x7f96480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9648004d00 0x7f9648005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966006bbd0 con 0x7f96480039a0 2023-08-11T15:52:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964800ba60 con 0x7f9648004d00 2023-08-11T15:52:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964800bc20 con 0x7f96600ccf40 2023-08-11T15:52:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.163+0000 7f965f7fe640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480039a0 0x7f9648003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.164+0000 7f965effd640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96600ccf40 0x7f96480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.164+0000 7f965ffff640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9648004d00 0x7f9648005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.164+0000 7f965f7fe640 1 -- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96600ccf40 msgr2=0x7f96480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.164+0000 7f965f7fe640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96600ccf40 0x7f96480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.164+0000 7f965f7fe640 1 -- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9648004d00 msgr2=0x7f9648005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.164+0000 7f965f7fe640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9648004d00 0x7f9648005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.164+0000 7f965f7fe640 1 -- 172.21.15.27:0/1104805120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648003520 con 0x7f96480039a0 2023-08-11T15:52:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.164+0000 7f965ffff640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9648004d00 0x7f9648005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.165+0000 7f965f7fe640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480039a0 0x7f9648003dc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f965000c210 tx=0x7f96480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802d671c48e5b2d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.165+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650005ad0 con 0x7f96480039a0 2023-08-11T15:52:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.165+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9650005c30 con 0x7f96480039a0 2023-08-11T15:52:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.165+0000 7f965e7fc640 1 -- 172.21.15.27:0/1104805120 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965002c890 con 0x7f96480039a0 2023-08-11T15:52:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.166+0000 7f9664960640 1 -- 172.21.15.27:0/1104805120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480039a0 msgr2=0x7f9648003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.166+0000 7f9664960640 1 --2- 172.21.15.27:0/1104805120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480039a0 0x7f9648003dc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f965000c210 tx=0x7f96480057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.166+0000 7f965ffff640 1 -- 172.21.15.27:0/1104805120 reap_dead start 2023-08-11T15:52:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.166+0000 7f9664960640 1 -- 172.21.15.27:0/1104805120 shutdown_connections 2023-08-11T15:52:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.166+0000 7f9664960640 1 -- 172.21.15.27:0/1104805120 >> 172.21.15.27:0/1104805120 conn(0x7f96600caab0 msgr2=0x7f96600cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.166+0000 7f9664960640 1 -- 172.21.15.27:0/1104805120 shutdown_connections 2023-08-11T15:52:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.166+0000 7f9664960640 1 -- 172.21.15.27:0/1104805120 wait complete. 2023-08-11T15:52:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.168+0000 7f9664960640 1 Processor -- start 2023-08-11T15:52:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.168+0000 7f9664960640 1 -- start start 2023-08-11T15:52:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.169+0000 7f9664960640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96480039a0 0x7f96600c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.169+0000 7f9664960640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c34b0 0x7f9660157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.169+0000 7f9664960640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96601577c0 0x7f966015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.169+0000 7f9664960640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96600c32f0 con 0x7f96600c34b0 2023-08-11T15:52:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.169+0000 7f9664960640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96601561e0 con 0x7f96601577c0 2023-08-11T15:52:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.169+0000 7f9664960640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96601564c0 con 0x7f96480039a0 2023-08-11T15:52:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c34b0 0x7f9660157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c34b0 0x7f9660157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60378/0 (socket says 172.21.15.27:60378) 2023-08-11T15:52:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965effd640 1 -- 172.21.15.27:0/2896233467 learned_addr learned my addr 172.21.15.27:0/2896233467 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965f7fe640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96480039a0 0x7f96600c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965ffff640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96601577c0 0x7f966015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965effd640 1 -- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96480039a0 msgr2=0x7f96600c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965effd640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96480039a0 0x7f96600c2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965effd640 1 -- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96601577c0 msgr2=0x7f966015b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965effd640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96601577c0 0x7f966015b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965effd640 1 -- 172.21.15.27:0/2896233467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96601567a0 con 0x7f96600c34b0 2023-08-11T15:52:23.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.170+0000 7f965effd640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c34b0 0x7f9660157280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f965400e550 tx=0x7f965400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:23.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.171+0000 7f9647fff640 1 -- 172.21.15.27:0/2896233467 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540052e0 con 0x7f96600c34b0 2023-08-11T15:52:23.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.171+0000 7f9647fff640 1 -- 172.21.15.27:0/2896233467 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9654005440 con 0x7f96600c34b0 2023-08-11T15:52:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.171+0000 7f9647fff640 1 -- 172.21.15.27:0/2896233467 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654003810 con 0x7f96600c34b0 2023-08-11T15:52:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.171+0000 7f9664960640 1 -- 172.21.15.27:0/2896233467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9660156a80 con 0x7f96600c34b0 2023-08-11T15:52:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.171+0000 7f9664960640 1 -- 172.21.15.27:0/2896233467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9660155e90 con 0x7f96600c34b0 2023-08-11T15:52:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.172+0000 7f9664960640 1 -- 172.21.15.27:0/2896233467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966004e420 con 0x7f96600c34b0 2023-08-11T15:52:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.173+0000 7f9647fff640 1 -- 172.21.15.27:0/2896233467 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9654003970 con 0x7f96600c34b0 2023-08-11T15:52:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.174+0000 7f9647fff640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f964c06a080 0x7f964c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.175+0000 7f9647fff640 1 -- 172.21.15.27:0/2896233467 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f96540886a0 con 0x7f96600c34b0 2023-08-11T15:52:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.175+0000 7f965f7fe640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f964c06a080 0x7f964c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.176+0000 7f965f7fe640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f964c06a080 0x7f964c06c540 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9650034000 tx=0x7f9650009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:23.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.179+0000 7f9647fff640 1 -- 172.21.15.27:0/2896233467 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f96540539c0 con 0x7f96600c34b0 2023-08-11T15:52:23.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.372+0000 7f9664960640 1 -- 172.21.15.27:0/2896233467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96600cb380 con 0x7f96600c34b0 2023-08-11T15:52:23.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.373+0000 7f9647fff640 1 -- 172.21.15.27:0/2896233467 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9654056fe0 con 0x7f96600c34b0 2023-08-11T15:52:23.375 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:23.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 -- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f964c06a080 msgr2=0x7f964c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f964c06a080 0x7f964c06c540 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9650034000 tx=0x7f9650009030 comp rx=0 tx=0).stop 2023-08-11T15:52:23.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 -- 172.21.15.27:0/2896233467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c34b0 msgr2=0x7f9660157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c34b0 0x7f9660157280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f965400e550 tx=0x7f965400ea10 comp rx=0 tx=0).stop 2023-08-11T15:52:23.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 -- 172.21.15.27:0/2896233467 shutdown_connections 2023-08-11T15:52:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f964c06a080 0x7f964c06c540 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96601577c0 0x7f966015b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c34b0 0x7f9660157280 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 --2- 172.21.15.27:0/2896233467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96480039a0 0x7f96600c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 -- 172.21.15.27:0/2896233467 >> 172.21.15.27:0/2896233467 conn(0x7f96600caab0 msgr2=0x7f966015e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 -- 172.21.15.27:0/2896233467 shutdown_connections 2023-08-11T15:52:23.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.376+0000 7f9664960640 1 -- 172.21.15.27:0/2896233467 wait complete. 2023-08-11T15:52:23.396 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:23.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.638+0000 7f7ae82b8640 1 Processor -- start 2023-08-11T15:52:23.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.638+0000 7f7ae82b8640 1 -- start start 2023-08-11T15:52:23.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.639+0000 7f7ae82b8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ae00b38f0 0x7f7ae00d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.639+0000 7f7ae82b8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 0x7f7ae0160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.640+0000 7f7ae82b8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae00af250 con 0x7f7ae00b3cd0 2023-08-11T15:52:23.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.640+0000 7f7ae82b8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae00b0190 con 0x7f7ae00b38f0 2023-08-11T15:52:23.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.640+0000 7f7ae82b8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00af510 con 0x7f7ae00b3090 2023-08-11T15:52:23.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.640+0000 7f7ae5853640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 0x7f7ae0160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.640+0000 7f7ae6054640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ae00b38f0 0x7f7ae00d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.640+0000 7f7ae5853640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 0x7f7ae0160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:52386/0 (socket says 172.21.15.27:52386) 2023-08-11T15:52:23.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.640+0000 7f7ae5853640 1 -- 172.21.15.27:0/3757289148 learned_addr learned my addr 172.21.15.27:0/3757289148 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:23.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.640+0000 7f7ae6054640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ae00b38f0 0x7f7ae00d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:42500/0 (socket says 172.21.15.27:42500) 2023-08-11T15:52:23.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.641+0000 7f7ae6054640 1 -- 172.21.15.27:0/3757289148 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 msgr2=0x7f7ae0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:23.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.641+0000 7f7ae6054640 1 --2- 172.21.15.27:0/3757289148 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 0x7f7ae0160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.641+0000 7f7ae6054640 1 -- 172.21.15.27:0/3757289148 >> v1:172.21.15.119:6789/0 conn(0x7f7ae00b3cd0 legacy=0x7f7ae00b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:23.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.641+0000 7f7ae6054640 1 -- 172.21.15.27:0/3757289148 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae00af7f0 con 0x7f7ae00b38f0 2023-08-11T15:52:23.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.643+0000 7f7ae6054640 1 --2- 172.21.15.27:0/3757289148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ae00b38f0 0x7f7ae00d30e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009790 tx=0x7f7adc02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5e5beeba51e7a58 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:23.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.643+0000 7f7ae5052640 1 -- 172.21.15.27:0/3757289148 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc02d040 con 0x7f7ae00b38f0 2023-08-11T15:52:23.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.644+0000 7f7ae5052640 1 -- 172.21.15.27:0/3757289148 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc031810 con 0x7f7ae00b38f0 2023-08-11T15:52:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.644+0000 7f7ae5052640 1 -- 172.21.15.27:0/3757289148 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc031ba0 con 0x7f7ae00b38f0 2023-08-11T15:52:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.644+0000 7f7ae82b8640 1 -- 172.21.15.27:0/3757289148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ae00b38f0 msgr2=0x7f7ae00d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.644+0000 7f7ae82b8640 1 --2- 172.21.15.27:0/3757289148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ae00b38f0 0x7f7ae00d30e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009790 tx=0x7f7adc02b8b0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.644+0000 7f7ae82b8640 1 -- 172.21.15.27:0/3757289148 shutdown_connections 2023-08-11T15:52:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.644+0000 7f7ae82b8640 1 --2- 172.21.15.27:0/3757289148 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 0x7f7ae0160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.644+0000 7f7ae82b8640 1 --2- 172.21.15.27:0/3757289148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ae00b38f0 0x7f7ae00d30e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.644+0000 7f7ae82b8640 1 -- 172.21.15.27:0/3757289148 >> 172.21.15.27:0/3757289148 conn(0x7f7ae009b960 msgr2=0x7f7ae00d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.644+0000 7f7ae82b8640 1 -- 172.21.15.27:0/3757289148 shutdown_connections 2023-08-11T15:52:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.644+0000 7f7ae82b8640 1 -- 172.21.15.27:0/3757289148 wait complete. 2023-08-11T15:52:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.646+0000 7f7ae82b8640 1 Processor -- start 2023-08-11T15:52:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.646+0000 7f7ae82b8640 1 -- start start 2023-08-11T15:52:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.647+0000 7f7ae82b8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 0x7f7ae00ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.647+0000 7f7ae82b8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae00b38f0 0x7f7ae0156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.647+0000 7f7ae82b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ae00b3cd0 0x7f7ae00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.647+0000 7f7ae82b8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae0156e60 con 0x7f7ae00b38f0 2023-08-11T15:52:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.647+0000 7f7ae82b8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae015d900 con 0x7f7ae00b3cd0 2023-08-11T15:52:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.647+0000 7f7ae82b8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae015d5d0 con 0x7f7ae00b3090 2023-08-11T15:52:23.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.647+0000 7f7ae5853640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae00b38f0 0x7f7ae0156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae5853640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae00b38f0 0x7f7ae0156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60394/0 (socket says 172.21.15.27:60394) 2023-08-11T15:52:23.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae5853640 1 -- 172.21.15.27:0/4268394161 learned_addr learned my addr 172.21.15.27:0/4268394161 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:23.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae6054640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 0x7f7ae00ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae5853640 1 -- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 msgr2=0x7f7ae00ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae5853640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 0x7f7ae00ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae5853640 1 -- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ae00b3cd0 msgr2=0x7f7ae00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae6855640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ae00b3cd0 0x7f7ae00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae5853640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ae00b3cd0 0x7f7ae00cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae5853640 1 -- 172.21.15.27:0/4268394161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0093a0 con 0x7f7ae00b38f0 2023-08-11T15:52:23.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae6054640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 0x7f7ae00ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:23.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.648+0000 7f7ae6855640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ae00b3cd0 0x7f7ae00cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:23.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.649+0000 7f7ae5853640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae00b38f0 0x7f7ae0156920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000e900 tx=0x7f7ad000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:23.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.649+0000 7f7ad6ffd640 1 -- 172.21.15.27:0/4268394161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000ca00 con 0x7f7ae00b38f0 2023-08-11T15:52:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.649+0000 7f7ad6ffd640 1 -- 172.21.15.27:0/4268394161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad000cb60 con 0x7f7ae00b38f0 2023-08-11T15:52:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.649+0000 7f7ad6ffd640 1 -- 172.21.15.27:0/4268394161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0013470 con 0x7f7ae00b38f0 2023-08-11T15:52:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.649+0000 7f7ae82b8640 1 -- 172.21.15.27:0/4268394161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae00ca8a0 con 0x7f7ae00b38f0 2023-08-11T15:52:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.649+0000 7f7ae82b8640 1 -- 172.21.15.27:0/4268394161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae00b52c0 con 0x7f7ae00b38f0 2023-08-11T15:52:23.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.651+0000 7f7ae82b8640 1 -- 172.21.15.27:0/4268394161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae015da90 con 0x7f7ae00b38f0 2023-08-11T15:52:23.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.652+0000 7f7ad6ffd640 1 -- 172.21.15.27:0/4268394161 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7ad00135d0 con 0x7f7ae00b38f0 2023-08-11T15:52:23.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.652+0000 7f7ad6ffd640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ac406a080 0x7f7ac406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:23.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.653+0000 7f7ad6ffd640 1 -- 172.21.15.27:0/4268394161 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7ad0088ae0 con 0x7f7ae00b38f0 2023-08-11T15:52:23.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.653+0000 7f7ae6054640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ac406a080 0x7f7ac406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:23.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.655+0000 7f7ae6054640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ac406a080 0x7f7ac406c540 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7adc036000 tx=0x7f7adc03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:23.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.657+0000 7f7ad6ffd640 1 -- 172.21.15.27:0/4268394161 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f7ad0053ea0 con 0x7f7ae00b38f0 2023-08-11T15:52:23.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.855+0000 7f7ae82b8640 1 -- 172.21.15.27:0/4268394161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ae004e420 con 0x7f7ae00b38f0 2023-08-11T15:52:23.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.856+0000 7f7ad6ffd640 1 -- 172.21.15.27:0/4268394161 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ad00574c0 con 0x7f7ae00b38f0 2023-08-11T15:52:23.859 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:23.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 -- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ac406a080 msgr2=0x7f7ac406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ac406a080 0x7f7ac406c540 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7adc036000 tx=0x7f7adc03c040 comp rx=0 tx=0).stop 2023-08-11T15:52:23.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 -- 172.21.15.27:0/4268394161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae00b38f0 msgr2=0x7f7ae0156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:23.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae00b38f0 0x7f7ae0156920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000e900 tx=0x7f7ad000edc0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 -- 172.21.15.27:0/4268394161 shutdown_connections 2023-08-11T15:52:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ac406a080 0x7f7ac406c540 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ae00b3cd0 0x7f7ae00cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae00b38f0 0x7f7ae0156920 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 --2- 172.21.15.27:0/4268394161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ae00b3090 0x7f7ae00ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 -- 172.21.15.27:0/4268394161 >> 172.21.15.27:0/4268394161 conn(0x7f7ae009b960 msgr2=0x7f7ae00d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.859+0000 7f7ae82b8640 1 -- 172.21.15.27:0/4268394161 shutdown_connections 2023-08-11T15:52:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:23.860+0000 7f7ae82b8640 1 -- 172.21.15.27:0/4268394161 wait complete. 2023-08-11T15:52:23.881 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:24.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.117+0000 7f59d8da2640 1 Processor -- start 2023-08-11T15:52:24.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.117+0000 7f59d8da2640 1 -- start start 2023-08-11T15:52:24.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.119+0000 7f59d8da2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f59d415a4d0 0x7f59d41435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.119+0000 7f59d8da2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59d406f5d0 0x7f59d406fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.119+0000 7f59d8da2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d406ff60 con 0x7f59d415a0f0 2023-08-11T15:52:24.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.119+0000 7f59d8da2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d406c500 con 0x7f59d415a4d0 2023-08-11T15:52:24.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.119+0000 7f59d8da2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d406b8f0 con 0x7f59d406f5d0 2023-08-11T15:52:24.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.120+0000 7f59d2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f59d415a4d0 0x7f59d41435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.120+0000 7f59d2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f59d415a4d0 0x7f59d41435e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60406/0 (socket says 172.21.15.27:60406) 2023-08-11T15:52:24.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.120+0000 7f59d2ffd640 1 -- 172.21.15.27:0/3807573581 learned_addr learned my addr 172.21.15.27:0/3807573581 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:24.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.120+0000 7f59d3fff640 1 --2- 172.21.15.27:0/3807573581 >> v2:172.21.15.119:3300/0 conn(0x7f59d406f5d0 0x7f59d406fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.120+0000 7f59d2ffd640 1 -- 172.21.15.27:0/3807573581 >> v2:172.21.15.119:3300/0 conn(0x7f59d406f5d0 msgr2=0x7f59d406fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:24.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.120+0000 7f59d2ffd640 1 --2- 172.21.15.27:0/3807573581 >> v2:172.21.15.119:3300/0 conn(0x7f59d406f5d0 0x7f59d406fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.120+0000 7f59d2ffd640 1 -- 172.21.15.27:0/3807573581 >> v1:172.21.15.119:6789/0 conn(0x7f59d415a0f0 legacy=0x7f59d4071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:24.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.120+0000 7f59d2ffd640 1 -- 172.21.15.27:0/3807573581 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d406bbd0 con 0x7f59d415a4d0 2023-08-11T15:52:24.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.121+0000 7f59d2ffd640 1 --2- 172.21.15.27:0/3807573581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a4d0 0x7f59d41435e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f59d4070240 tx=0x7f59c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c1c235c46795e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:24.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.122+0000 7f59d27fc640 1 -- 172.21.15.27:0/3807573581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8031870 con 0x7f59d415a4d0 2023-08-11T15:52:24.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.122+0000 7f59d27fc640 1 -- 172.21.15.27:0/3807573581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c80319d0 con 0x7f59d415a4d0 2023-08-11T15:52:24.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.122+0000 7f59d27fc640 1 -- 172.21.15.27:0/3807573581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8031d60 con 0x7f59d415a4d0 2023-08-11T15:52:24.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.122+0000 7f59d8da2640 1 -- 172.21.15.27:0/3807573581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a4d0 msgr2=0x7f59d41435e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.122+0000 7f59d8da2640 1 --2- 172.21.15.27:0/3807573581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a4d0 0x7f59d41435e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f59d4070240 tx=0x7f59c802b910 comp rx=0 tx=0).stop 2023-08-11T15:52:24.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.122+0000 7f59d8da2640 1 -- 172.21.15.27:0/3807573581 shutdown_connections 2023-08-11T15:52:24.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.122+0000 7f59d8da2640 1 --2- 172.21.15.27:0/3807573581 >> v2:172.21.15.119:3300/0 conn(0x7f59d406f5d0 0x7f59d406fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.122+0000 7f59d8da2640 1 --2- 172.21.15.27:0/3807573581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a4d0 0x7f59d41435e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.122+0000 7f59d8da2640 1 -- 172.21.15.27:0/3807573581 >> 172.21.15.27:0/3807573581 conn(0x7f59d40caab0 msgr2=0x7f59d40cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:24.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.123+0000 7f59d8da2640 1 -- 172.21.15.27:0/3807573581 shutdown_connections 2023-08-11T15:52:24.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.123+0000 7f59d8da2640 1 -- 172.21.15.27:0/3807573581 wait complete. 2023-08-11T15:52:24.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.124+0000 7f59d8da2640 1 Processor -- start 2023-08-11T15:52:24.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.125+0000 7f59d8da2640 1 -- start start 2023-08-11T15:52:24.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.125+0000 7f59d8da2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59d406f5d0 0x7f59d4154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.125+0000 7f59d8da2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a0f0 0x7f59d40c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.125+0000 7f59d8da2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d415a4d0 0x7f59d415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.125+0000 7f59d8da2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40c3ba0 con 0x7f59d415a0f0 2023-08-11T15:52:24.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.125+0000 7f59d8da2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41553f0 con 0x7f59d415a4d0 2023-08-11T15:52:24.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.125+0000 7f59d8da2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40c3fc0 con 0x7f59d406f5d0 2023-08-11T15:52:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a0f0 0x7f59d40c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a0f0 0x7f59d40c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60418/0 (socket says 172.21.15.27:60418) 2023-08-11T15:52:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d2ffd640 1 -- 172.21.15.27:0/2402436379 learned_addr learned my addr 172.21.15.27:0/2402436379 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d37fe640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59d406f5d0 0x7f59d4154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d3fff640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d415a4d0 0x7f59d415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d2ffd640 1 -- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59d406f5d0 msgr2=0x7f59d4154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d2ffd640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59d406f5d0 0x7f59d4154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d2ffd640 1 -- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d415a4d0 msgr2=0x7f59d415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d2ffd640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d415a4d0 0x7f59d415cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d2ffd640 1 -- 172.21.15.27:0/2402436379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c80094d0 con 0x7f59d415a0f0 2023-08-11T15:52:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d37fe640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59d406f5d0 0x7f59d4154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.126+0000 7f59d3fff640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d415a4d0 0x7f59d415cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.127+0000 7f59d2ffd640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a0f0 0x7f59d40c4900 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f59c8000c00 tx=0x7f59c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.127+0000 7f59bbfff640 1 -- 172.21.15.27:0/2402436379 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c803abe0 con 0x7f59d415a0f0 2023-08-11T15:52:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.127+0000 7f59bbfff640 1 -- 172.21.15.27:0/2402436379 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c803ad40 con 0x7f59d415a0f0 2023-08-11T15:52:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.127+0000 7f59bbfff640 1 -- 172.21.15.27:0/2402436379 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8044610 con 0x7f59d415a0f0 2023-08-11T15:52:24.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.127+0000 7f59d8da2640 1 -- 172.21.15.27:0/2402436379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d40c2f20 con 0x7f59d415a0f0 2023-08-11T15:52:24.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.127+0000 7f59d8da2640 1 -- 172.21.15.27:0/2402436379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d40c34e0 con 0x7f59d415a0f0 2023-08-11T15:52:24.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.128+0000 7f59d8da2640 1 -- 172.21.15.27:0/2402436379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d40728d0 con 0x7f59d415a0f0 2023-08-11T15:52:24.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.129+0000 7f59bbfff640 1 -- 172.21.15.27:0/2402436379 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f59c804d450 con 0x7f59d415a0f0 2023-08-11T15:52:24.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.130+0000 7f59bbfff640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f59c006a080 0x7f59c006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.130+0000 7f59bbfff640 1 -- 172.21.15.27:0/2402436379 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f59c802d040 con 0x7f59d415a0f0 2023-08-11T15:52:24.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.131+0000 7f59d37fe640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f59c006a080 0x7f59c006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.132+0000 7f59d37fe640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f59c006a080 0x7f59c006c540 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f59d415d950 tx=0x7f59c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:24.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.135+0000 7f59bbfff640 1 -- 172.21.15.27:0/2402436379 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f59c80760a0 con 0x7f59d415a0f0 2023-08-11T15:52:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.329+0000 7f59d8da2640 1 -- 172.21.15.27:0/2402436379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f59d406ff60 con 0x7f59d415a0f0 2023-08-11T15:52:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.330+0000 7f59bbfff640 1 -- 172.21.15.27:0/2402436379 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f59d406ff60 con 0x7f59d415a0f0 2023-08-11T15:52:24.333 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:24.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 -- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f59c006a080 msgr2=0x7f59c006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f59c006a080 0x7f59c006c540 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f59d415d950 tx=0x7f59c4009040 comp rx=0 tx=0).stop 2023-08-11T15:52:24.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 -- 172.21.15.27:0/2402436379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a0f0 msgr2=0x7f59d40c4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a0f0 0x7f59d40c4900 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f59c8000c00 tx=0x7f59c80046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 -- 172.21.15.27:0/2402436379 shutdown_connections 2023-08-11T15:52:24.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f59c006a080 0x7f59c006c540 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d415a4d0 0x7f59d415cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59d415a0f0 0x7f59d40c4900 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 --2- 172.21.15.27:0/2402436379 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59d406f5d0 0x7f59d4154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 -- 172.21.15.27:0/2402436379 >> 172.21.15.27:0/2402436379 conn(0x7f59d40caab0 msgr2=0x7f59d40cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:24.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 -- 172.21.15.27:0/2402436379 shutdown_connections 2023-08-11T15:52:24.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.333+0000 7f59d8da2640 1 -- 172.21.15.27:0/2402436379 wait complete. 2023-08-11T15:52:24.356 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:52:24.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.593+0000 7fcb00542640 1 Processor -- start 2023-08-11T15:52:24.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.593+0000 7fcb00542640 1 -- start start 2023-08-11T15:52:24.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.595+0000 7fcb00542640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf815b780 0x7fcaf815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.595+0000 7fcb00542640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf815f050 con 0x7fcaf80c27d0 2023-08-11T15:52:24.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.595+0000 7fcb00542640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf815e0c0 con 0x7fcaf8069f10 2023-08-11T15:52:24.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.595+0000 7fcb00542640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf80cb990 con 0x7fcaf815b780 2023-08-11T15:52:24.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.595+0000 7fcafdadd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcaf80c27d0 0x7fcaf8069800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47118/0 (socket says 172.21.15.27:47118) 2023-08-11T15:52:24.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.595+0000 7fcafdadd640 1 -- 172.21.15.27:0/1843989444 learned_addr learned my addr 172.21.15.27:0/1843989444 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:24.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafe2de640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf815b780 0x7fcaf815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821824829 0 0) 0x7fcaec001110 con 0x7fcaf80c27d0 2023-08-11T15:52:24.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae0003520 con 0x7fcaf80c27d0 2023-08-11T15:52:24.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2431015592 0 0) 0x7fcae0003520 con 0x7fcaf80c27d0 2023-08-11T15:52:24.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf815b780 msgr2=0x7fcaf815eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:24.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf815b780 0x7fcaf815eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 >> v1:172.21.15.119:6789/0 conn(0x7fcaf8069f10 legacy=0x7fcaf8070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf80cc850 con 0x7fcaf80c27d0 2023-08-11T15:52:24.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fcaec001800 con 0x7fcaf80c27d0 2023-08-11T15:52:24.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 >> v1:172.21.15.27:6789/0 conn(0x7fcaf80c27d0 legacy=0x7fcaf8069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf815b780 0x7fcae00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae0003f20 0x7fcae00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 0x7fcae0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf80cb990 con 0x7fcae0004d00 2023-08-11T15:52:24.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae000bbc0 con 0x7fcae0003f20 2023-08-11T15:52:24.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.596+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae000bd80 con 0x7fcaf815b780 2023-08-11T15:52:24.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.597+0000 7fcafeadf640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 0x7fcae0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.597+0000 7fcafe2de640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf815b780 0x7fcae00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.597+0000 7fcafeadf640 1 -- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf815b780 msgr2=0x7fcae00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:24.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.597+0000 7fcafeadf640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf815b780 0x7fcae00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.597+0000 7fcafeadf640 1 -- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae0003f20 msgr2=0x7fcae00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:24.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.597+0000 7fcafdadd640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae0003f20 0x7fcae00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.597+0000 7fcafeadf640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae0003f20 0x7fcae00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.597+0000 7fcafeadf640 1 -- 172.21.15.27:0/1843989444 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae0003520 con 0x7fcae0004d00 2023-08-11T15:52:24.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.597+0000 7fcafdadd640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae0003f20 0x7fcae00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:24.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.598+0000 7fcafeadf640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 0x7fcae0005180 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0001750 tx=0x7fcae00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eede5880d3b4f77 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:24.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.599+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00052d0 con 0x7fcae0004d00 2023-08-11T15:52:24.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.599+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf002e050 con 0x7fcae0004d00 2023-08-11T15:52:24.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.599+0000 7fcafd2dc640 1 -- 172.21.15.27:0/1843989444 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0003830 con 0x7fcae0004d00 2023-08-11T15:52:24.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.599+0000 7fcb00542640 1 -- 172.21.15.27:0/1843989444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 msgr2=0x7fcae0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.599+0000 7fcb00542640 1 --2- 172.21.15.27:0/1843989444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 0x7fcae0005180 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0001750 tx=0x7fcae00057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.599+0000 7fcafeadf640 1 -- 172.21.15.27:0/1843989444 reap_dead start 2023-08-11T15:52:24.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.599+0000 7fcb00542640 1 -- 172.21.15.27:0/1843989444 shutdown_connections 2023-08-11T15:52:24.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.599+0000 7fcb00542640 1 -- 172.21.15.27:0/1843989444 >> 172.21.15.27:0/1843989444 conn(0x7fcaf80cbbc0 msgr2=0x7fcaf80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:24.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.600+0000 7fcb00542640 1 -- 172.21.15.27:0/1843989444 shutdown_connections 2023-08-11T15:52:24.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.600+0000 7fcb00542640 1 -- 172.21.15.27:0/1843989444 wait complete. 2023-08-11T15:52:24.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.601+0000 7fcb00542640 1 Processor -- start 2023-08-11T15:52:24.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.602+0000 7fcb00542640 1 -- start start 2023-08-11T15:52:24.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcb00542640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 0x7fcaf81f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcb00542640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaf80c4f60 0x7fcaf80c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcb00542640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf81f2310 0x7fcaf815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcb00542640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81f5f40 con 0x7fcae0004d00 2023-08-11T15:52:24.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcb00542640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf80b4e20 con 0x7fcaf80c4f60 2023-08-11T15:52:24.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcb00542640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81f1e80 con 0x7fcaf81f2310 2023-08-11T15:52:24.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcafe2de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 0x7fcaf81f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcafe2de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 0x7fcaf81f5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60438/0 (socket says 172.21.15.27:60438) 2023-08-11T15:52:24.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcafe2de640 1 -- 172.21.15.27:0/4275370248 learned_addr learned my addr 172.21.15.27:0/4275370248 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:24.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcafe2de640 1 -- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf81f2310 msgr2=0x7fcaf815c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcafdadd640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaf80c4f60 0x7fcaf80c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcafeadf640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf81f2310 0x7fcaf815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcafe2de640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf81f2310 0x7fcaf815c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.603+0000 7fcafe2de640 1 -- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaf80c4f60 msgr2=0x7fcaf80c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.604+0000 7fcafe2de640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaf80c4f60 0x7fcaf80c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.604+0000 7fcafe2de640 1 -- 172.21.15.27:0/4275370248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf80c5490 con 0x7fcae0004d00 2023-08-11T15:52:24.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.604+0000 7fcafdadd640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaf80c4f60 0x7fcaf80c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:24.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.604+0000 7fcafe2de640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 0x7fcaf81f5a00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcae8007f60 tx=0x7fcae8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:24.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.604+0000 7fcaf6ffd640 1 -- 172.21.15.27:0/4275370248 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae800e030 con 0x7fcae0004d00 2023-08-11T15:52:24.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.604+0000 7fcaf6ffd640 1 -- 172.21.15.27:0/4275370248 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae8016470 con 0x7fcae0004d00 2023-08-11T15:52:24.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.604+0000 7fcaf6ffd640 1 -- 172.21.15.27:0/4275370248 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8016820 con 0x7fcae0004d00 2023-08-11T15:52:24.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.604+0000 7fcb00542640 1 -- 172.21.15.27:0/4275370248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf80c5770 con 0x7fcae0004d00 2023-08-11T15:52:24.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.604+0000 7fcb00542640 1 -- 172.21.15.27:0/4275370248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf80c9f10 con 0x7fcae0004d00 2023-08-11T15:52:24.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.605+0000 7fcb00542640 1 -- 172.21.15.27:0/4275370248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf8071c50 con 0x7fcae0004d00 2023-08-11T15:52:24.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.607+0000 7fcaf6ffd640 1 -- 172.21.15.27:0/4275370248 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcae8016980 con 0x7fcae0004d00 2023-08-11T15:52:24.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.608+0000 7fcaf6ffd640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcae406a080 0x7fcae406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:24.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.608+0000 7fcaf6ffd640 1 -- 172.21.15.27:0/4275370248 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcae800f070 con 0x7fcae0004d00 2023-08-11T15:52:24.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.608+0000 7fcafdadd640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcae406a080 0x7fcae406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:24.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.609+0000 7fcafdadd640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcae406a080 0x7fcae406c540 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcaec001930 tx=0x7fcaec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:24.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.612+0000 7fcaf6ffd640 1 -- 172.21.15.27:0/4275370248 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fcae8058610 con 0x7fcae0004d00 2023-08-11T15:52:24.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.824+0000 7fcb00542640 1 -- 172.21.15.27:0/4275370248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcaf8073990 con 0x7fcae0004d00 2023-08-11T15:52:24.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.826+0000 7fcaf6ffd640 1 -- 172.21.15.27:0/4275370248 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcae80581a0 con 0x7fcae0004d00 2023-08-11T15:52:24.828 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:52:24.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.828+0000 7fcb00542640 1 -- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcae406a080 msgr2=0x7fcae406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.828+0000 7fcb00542640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcae406a080 0x7fcae406c540 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcaec001930 tx=0x7fcaec009040 comp rx=0 tx=0).stop 2023-08-11T15:52:24.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.828+0000 7fcb00542640 1 -- 172.21.15.27:0/4275370248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 msgr2=0x7fcaf81f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:24.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.828+0000 7fcb00542640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 0x7fcaf81f5a00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcae8007f60 tx=0x7fcae8007f90 comp rx=0 tx=0).stop 2023-08-11T15:52:24.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.829+0000 7fcb00542640 1 -- 172.21.15.27:0/4275370248 shutdown_connections 2023-08-11T15:52:24.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.829+0000 7fcb00542640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcae406a080 0x7fcae406c540 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.829+0000 7fcb00542640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcaf81f2310 0x7fcaf815c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.829+0000 7fcb00542640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaf80c4f60 0x7fcaf80c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.829+0000 7fcb00542640 1 --2- 172.21.15.27:0/4275370248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae0004d00 0x7fcaf81f5a00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:24.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.829+0000 7fcb00542640 1 -- 172.21.15.27:0/4275370248 >> 172.21.15.27:0/4275370248 conn(0x7fcaf80cbbc0 msgr2=0x7fcaf80cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:24.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.829+0000 7fcb00542640 1 -- 172.21.15.27:0/4275370248 shutdown_connections 2023-08-11T15:52:24.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:24.829+0000 7fcb00542640 1 -- 172.21.15.27:0/4275370248 wait complete. 2023-08-11T15:52:24.849 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:52:24.850 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:24.911 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:25.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.154+0000 7f9fce0a7640 1 Processor -- start 2023-08-11T15:52:25.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.154+0000 7f9fce0a7640 1 -- start start 2023-08-11T15:52:25.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.156+0000 7f9fce0a7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fc80cc520 0x7f9fc815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.156+0000 7f9fce0a7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9fc815d1e0 0x7f9fc80c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.156+0000 7f9fce0a7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc815da30 con 0x7f9fc8160d20 2023-08-11T15:52:25.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.156+0000 7f9fce0a7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc8156fa0 con 0x7f9fc815d1e0 2023-08-11T15:52:25.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.156+0000 7f9fce0a7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80ca570 con 0x7f9fc80cc520 2023-08-11T15:52:25.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.156+0000 7f9fc77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9fc8160d20 0x7f9fc80cbe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47126/0 (socket says 172.21.15.27:47126) 2023-08-11T15:52:25.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.156+0000 7f9fc77fe640 1 -- 172.21.15.27:0/838996794 learned_addr learned my addr 172.21.15.27:0/838996794 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:25.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc7fff640 1 --2- 172.21.15.27:0/838996794 >> v2:172.21.15.119:3300/0 conn(0x7f9fc815d1e0 0x7f9fc80c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc6ffd640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fc80cc520 0x7f9fc815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492503353 0 0) 0x7f9fc815da30 con 0x7f9fc8160d20 2023-08-11T15:52:25.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb0003520 con 0x7f9fc8160d20 2023-08-11T15:52:25.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 609177485 0 0) 0x7f9fb0003520 con 0x7f9fc8160d20 2023-08-11T15:52:25.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fc80cc520 msgr2=0x7f9fc815cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:25.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fc80cc520 0x7f9fc815cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 >> v2:172.21.15.119:3300/0 conn(0x7f9fc815d1e0 msgr2=0x7f9fc80c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:25.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 --2- 172.21.15.27:0/838996794 >> v2:172.21.15.119:3300/0 conn(0x7f9fc815d1e0 0x7f9fc80c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc80ca850 con 0x7f9fc8160d20 2023-08-11T15:52:25.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f9fb8001810 con 0x7f9fc8160d20 2023-08-11T15:52:25.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 >> v1:172.21.15.27:6789/0 conn(0x7f9fc8160d20 legacy=0x7f9fc80cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fb0003970 0x7f9fb0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.157+0000 7f9fc67fc640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fc80cc520 0x7f9fb00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc67fc640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fb0004d00 0x7f9fb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc8156fa0 con 0x7f9fb0004d00 2023-08-11T15:52:25.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80ca570 con 0x7f9fb0003970 2023-08-11T15:52:25.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb000b980 con 0x7f9fc80cc520 2023-08-11T15:52:25.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc7fff640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fb0004d00 0x7f9fb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc77fe640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fb0003970 0x7f9fb0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc6ffd640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fc80cc520 0x7f9fb00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc7fff640 1 -- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fc80cc520 msgr2=0x7f9fb00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:25.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc7fff640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fc80cc520 0x7f9fb00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc7fff640 1 -- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fb0003970 msgr2=0x7f9fb0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:25.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc7fff640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fb0003970 0x7f9fb0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc7fff640 1 -- 172.21.15.27:0/838996794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb0003520 con 0x7f9fb0004d00 2023-08-11T15:52:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.158+0000 7f9fc77fe640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fb0003970 0x7f9fb0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.160+0000 7f9fc7fff640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fb0004d00 0x7f9fb0005180 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009fc0 tx=0x7f9fb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5369e76d4e0b4a50 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.160+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc002cc60 con 0x7f9fb0004d00 2023-08-11T15:52:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.160+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc000f040 con 0x7f9fb0004d00 2023-08-11T15:52:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.160+0000 7f9fc67fc640 1 -- 172.21.15.27:0/838996794 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0034440 con 0x7f9fb0004d00 2023-08-11T15:52:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.160+0000 7f9fce0a7640 1 -- 172.21.15.27:0/838996794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fb0004d00 msgr2=0x7f9fb0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.160+0000 7f9fce0a7640 1 --2- 172.21.15.27:0/838996794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fb0004d00 0x7f9fb0005180 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009fc0 tx=0x7f9fb00057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.160+0000 7f9fc7fff640 1 -- 172.21.15.27:0/838996794 reap_dead start 2023-08-11T15:52:25.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.160+0000 7f9fce0a7640 1 -- 172.21.15.27:0/838996794 shutdown_connections 2023-08-11T15:52:25.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.160+0000 7f9fce0a7640 1 -- 172.21.15.27:0/838996794 >> 172.21.15.27:0/838996794 conn(0x7f9fc80afb70 msgr2=0x7f9fc80aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:25.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.161+0000 7f9fce0a7640 1 -- 172.21.15.27:0/838996794 shutdown_connections 2023-08-11T15:52:25.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.161+0000 7f9fce0a7640 1 -- 172.21.15.27:0/838996794 wait complete. 2023-08-11T15:52:25.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.162+0000 7f9fce0a7640 1 Processor -- start 2023-08-11T15:52:25.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.163+0000 7f9fce0a7640 1 -- start start 2023-08-11T15:52:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fce0a7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fb0004d00 0x7f9fc81f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fce0a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fc81f1a30 0x7f9fc8155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fce0a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fc8152ff0 0x7f9fc8153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fce0a7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc81f2bf0 con 0x7f9fc81f1a30 2023-08-11T15:52:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fce0a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc81f5f70 con 0x7f9fc8152ff0 2023-08-11T15:52:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fce0a7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80c7c80 con 0x7f9fb0004d00 2023-08-11T15:52:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fc81f1a30 0x7f9fc8155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fc81f1a30 0x7f9fc8155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60466/0 (socket says 172.21.15.27:60466) 2023-08-11T15:52:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fc6ffd640 1 -- 172.21.15.27:0/2207829511 learned_addr learned my addr 172.21.15.27:0/2207829511 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fc77fe640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fb0004d00 0x7f9fc81f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fc7fff640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fc8152ff0 0x7f9fc8153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fc6ffd640 1 -- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fb0004d00 msgr2=0x7f9fc81f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fc6ffd640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fb0004d00 0x7f9fc81f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fc6ffd640 1 -- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fc8152ff0 msgr2=0x7f9fc8153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fc6ffd640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fc8152ff0 0x7f9fc8153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.164+0000 7f9fc6ffd640 1 -- 172.21.15.27:0/2207829511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc81539b0 con 0x7f9fc81f1a30 2023-08-11T15:52:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.165+0000 7f9fc6ffd640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fc81f1a30 0x7f9fc8155120 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc011600 tx=0x7f9fbc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.165+0000 7f9faffff640 1 -- 172.21.15.27:0/2207829511 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc00f920 con 0x7f9fc81f1a30 2023-08-11T15:52:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.165+0000 7f9faffff640 1 -- 172.21.15.27:0/2207829511 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fbc00fa80 con 0x7f9fc81f1a30 2023-08-11T15:52:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.165+0000 7f9faffff640 1 -- 172.21.15.27:0/2207829511 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc00fdf0 con 0x7f9fc81f1a30 2023-08-11T15:52:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.165+0000 7f9fce0a7640 1 -- 172.21.15.27:0/2207829511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc81f1f60 con 0x7f9fc81f1a30 2023-08-11T15:52:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.165+0000 7f9fce0a7640 1 -- 172.21.15.27:0/2207829511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc81f2520 con 0x7f9fc81f1a30 2023-08-11T15:52:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.166+0000 7f9fce0a7640 1 -- 172.21.15.27:0/2207829511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc8073990 con 0x7f9fc81f1a30 2023-08-11T15:52:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.167+0000 7f9faffff640 1 -- 172.21.15.27:0/2207829511 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9fbc015460 con 0x7f9fc81f1a30 2023-08-11T15:52:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.168+0000 7f9faffff640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9fb406a080 0x7f9fb406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.168+0000 7f9faffff640 1 -- 172.21.15.27:0/2207829511 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9fbc088dc0 con 0x7f9fc81f1a30 2023-08-11T15:52:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.169+0000 7f9fc77fe640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9fb406a080 0x7f9fb406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.170+0000 7f9fc77fe640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9fb406a080 0x7f9fb406c540 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8001940 tx=0x7f9fb8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.173+0000 7f9faffff640 1 -- 172.21.15.27:0/2207829511 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9fbc054170 con 0x7f9fc81f1a30 2023-08-11T15:52:25.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.383+0000 7f9fce0a7640 1 -- 172.21.15.27:0/2207829511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fc80cb7f0 con 0x7f9fc81f1a30 2023-08-11T15:52:25.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.384+0000 7f9faffff640 1 -- 172.21.15.27:0/2207829511 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f9fbc057790 con 0x7f9fc81f1a30 2023-08-11T15:52:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:25.387 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:25.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.387+0000 7f9fce0a7640 1 -- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9fb406a080 msgr2=0x7f9fb406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.387+0000 7f9fce0a7640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9fb406a080 0x7f9fb406c540 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8001940 tx=0x7f9fb8009040 comp rx=0 tx=0).stop 2023-08-11T15:52:25.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.387+0000 7f9fce0a7640 1 -- 172.21.15.27:0/2207829511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fc81f1a30 msgr2=0x7f9fc8155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.387+0000 7f9fce0a7640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fc81f1a30 0x7f9fc8155120 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc011600 tx=0x7f9fbc011ac0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.388+0000 7f9fce0a7640 1 -- 172.21.15.27:0/2207829511 shutdown_connections 2023-08-11T15:52:25.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.388+0000 7f9fce0a7640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9fb406a080 0x7f9fb406c540 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.388+0000 7f9fce0a7640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fc8152ff0 0x7f9fc8153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.388+0000 7f9fce0a7640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fc81f1a30 0x7f9fc8155120 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.388+0000 7f9fce0a7640 1 --2- 172.21.15.27:0/2207829511 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fb0004d00 0x7f9fc81f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.388+0000 7f9fce0a7640 1 -- 172.21.15.27:0/2207829511 >> 172.21.15.27:0/2207829511 conn(0x7f9fc80afb70 msgr2=0x7f9fc80cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:25.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.388+0000 7f9fce0a7640 1 -- 172.21.15.27:0/2207829511 shutdown_connections 2023-08-11T15:52:25.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.388+0000 7f9fce0a7640 1 -- 172.21.15.27:0/2207829511 wait complete. 2023-08-11T15:52:25.412 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:25.414 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.639+0000 7f06fb63f640 1 Processor -- start 2023-08-11T15:52:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.639+0000 7f06fb63f640 1 -- start start 2023-08-11T15:52:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.641+0000 7f06fb63f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06f415c760 0x7f06f40d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.641+0000 7f06fb63f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f406b670 0x7f06f40d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.641+0000 7f06fb63f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f406bc10 con 0x7f06f40d2850 2023-08-11T15:52:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.641+0000 7f06fb63f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f406c790 con 0x7f06f415c760 2023-08-11T15:52:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.641+0000 7f06fb63f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f406bef0 con 0x7f06f406b670 2023-08-11T15:52:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.641+0000 7f06f9bdc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f06f40d2850 0x7f06f41602b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47138/0 (socket says 172.21.15.27:47138) 2023-08-11T15:52:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.641+0000 7f06f9bdc640 1 -- 172.21.15.27:0/4213606668 learned_addr learned my addr 172.21.15.27:0/4213606668 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.641+0000 7f06f93db640 1 --2- 172.21.15.27:0/4213606668 >> v2:172.21.15.119:3300/0 conn(0x7f06f415c760 0x7f06f40d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06f8bda640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f406b670 0x7f06f40d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266126099 0 0) 0x7f06f406bc10 con 0x7f06f40d2850 2023-08-11T15:52:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06e0003520 con 0x7f06f40d2850 2023-08-11T15:52:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3324902817 0 0) 0x7f06e0003520 con 0x7f06f40d2850 2023-08-11T15:52:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f406b670 msgr2=0x7f06f40d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f406b670 0x7f06f40d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 >> v2:172.21.15.119:3300/0 conn(0x7f06f415c760 msgr2=0x7f06f40d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 --2- 172.21.15.27:0/4213606668 >> v2:172.21.15.119:3300/0 conn(0x7f06f415c760 0x7f06f40d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f406c1d0 con 0x7f06f40d2850 2023-08-11T15:52:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f06ec0019f0 con 0x7f06f40d2850 2023-08-11T15:52:25.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 >> v1:172.21.15.27:6789/0 conn(0x7f06f40d2850 legacy=0x7f06f41602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f406b670 0x7f06e0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06e0003ef0 0x7f06e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e0004d00 0x7f06e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f406bef0 con 0x7f06e0003ef0 2023-08-11T15:52:25.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e000b980 con 0x7f06e0004d00 2023-08-11T15:52:25.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.642+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e000bac0 con 0x7f06f406b670 2023-08-11T15:52:25.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.643+0000 7f06f8bda640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06e0003ef0 0x7f06e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.643+0000 7f06f93db640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f406b670 0x7f06e0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.643+0000 7f06f9bdc640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e0004d00 0x7f06e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.643+0000 7f06f8bda640 1 -- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f406b670 msgr2=0x7f06e0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:25.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.643+0000 7f06f8bda640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f406b670 0x7f06e0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.643+0000 7f06f8bda640 1 -- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e0004d00 msgr2=0x7f06e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:25.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.643+0000 7f06f8bda640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e0004d00 0x7f06e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.643+0000 7f06f8bda640 1 -- 172.21.15.27:0/4213606668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e0003520 con 0x7f06e0003ef0 2023-08-11T15:52:25.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.643+0000 7f06f9bdc640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e0004d00 0x7f06e0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:25.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.644+0000 7f06f8bda640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06e0003ef0 0x7f06e00047c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009ac0 tx=0x7f06e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4963ed4ed9531611 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:25.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.644+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e402c6b0 con 0x7f06e0003ef0 2023-08-11T15:52:25.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.645+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e402c810 con 0x7f06e0003ef0 2023-08-11T15:52:25.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.645+0000 7f06ebfff640 1 -- 172.21.15.27:0/4213606668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e402cba0 con 0x7f06e0003ef0 2023-08-11T15:52:25.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.645+0000 7f06fb63f640 1 -- 172.21.15.27:0/4213606668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06e0003ef0 msgr2=0x7f06e00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.645+0000 7f06fb63f640 1 --2- 172.21.15.27:0/4213606668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06e0003ef0 0x7f06e00047c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009ac0 tx=0x7f06e00057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.645+0000 7f06f9bdc640 1 -- 172.21.15.27:0/4213606668 reap_dead start 2023-08-11T15:52:25.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.645+0000 7f06fb63f640 1 -- 172.21.15.27:0/4213606668 shutdown_connections 2023-08-11T15:52:25.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.645+0000 7f06fb63f640 1 -- 172.21.15.27:0/4213606668 >> 172.21.15.27:0/4213606668 conn(0x7f06f4089ff0 msgr2=0x7f06f409b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:25.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.645+0000 7f06fb63f640 1 -- 172.21.15.27:0/4213606668 shutdown_connections 2023-08-11T15:52:25.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.645+0000 7f06fb63f640 1 -- 172.21.15.27:0/4213606668 wait complete. 2023-08-11T15:52:25.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.647+0000 7f06fb63f640 1 Processor -- start 2023-08-11T15:52:25.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.647+0000 7f06fb63f640 1 -- start start 2023-08-11T15:52:25.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.648+0000 7f06fb63f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06e0003ef0 0x7f06f40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06fb63f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f40cc3f0 0x7f06f4157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06fb63f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f40b2900 0x7f06f40b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06fb63f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f40b32c0 con 0x7f06f40cc3f0 2023-08-11T15:52:25.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06fb63f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f4158500 con 0x7f06f40b2900 2023-08-11T15:52:25.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06fb63f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f40c9760 con 0x7f06e0003ef0 2023-08-11T15:52:25.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f8bda640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f40cc3f0 0x7f06f4157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f8bda640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f40cc3f0 0x7f06f4157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60492/0 (socket says 172.21.15.27:60492) 2023-08-11T15:52:25.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f8bda640 1 -- 172.21.15.27:0/3280451993 learned_addr learned my addr 172.21.15.27:0/3280451993 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:25.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f93db640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06e0003ef0 0x7f06f40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f9bdc640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f40b2900 0x7f06f40b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f8bda640 1 -- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06e0003ef0 msgr2=0x7f06f40c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f8bda640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06e0003ef0 0x7f06f40c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f8bda640 1 -- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f40b2900 msgr2=0x7f06f40b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f8bda640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f40b2900 0x7f06f40b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f8bda640 1 -- 172.21.15.27:0/3280451993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f40c93d0 con 0x7f06f40cc3f0 2023-08-11T15:52:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.649+0000 7f06f9bdc640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f40b2900 0x7f06f40b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.650+0000 7f06f8bda640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f40cc3f0 0x7f06f4157bd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009be0 tx=0x7f06e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.650+0000 7f06e9ffb640 1 -- 172.21.15.27:0/3280451993 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4003810 con 0x7f06f40cc3f0 2023-08-11T15:52:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.650+0000 7f06e9ffb640 1 -- 172.21.15.27:0/3280451993 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e4003970 con 0x7f06f40cc3f0 2023-08-11T15:52:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.650+0000 7f06e9ffb640 1 -- 172.21.15.27:0/3280451993 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4003ce0 con 0x7f06f40cc3f0 2023-08-11T15:52:25.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.650+0000 7f06fb63f640 1 -- 172.21.15.27:0/3280451993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f40cd830 con 0x7f06f40cc3f0 2023-08-11T15:52:25.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.650+0000 7f06fb63f640 1 -- 172.21.15.27:0/3280451993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f40cddf0 con 0x7f06f40cc3f0 2023-08-11T15:52:25.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.652+0000 7f06fb63f640 1 -- 172.21.15.27:0/3280451993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f404e420 con 0x7f06f40cc3f0 2023-08-11T15:52:25.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.652+0000 7f06e9ffb640 1 -- 172.21.15.27:0/3280451993 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f06e4034440 con 0x7f06f40cc3f0 2023-08-11T15:52:25.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.653+0000 7f06e9ffb640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f06d806a050 0x7f06d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:25.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.653+0000 7f06e9ffb640 1 -- 172.21.15.27:0/3280451993 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f06e40aa730 con 0x7f06f40cc3f0 2023-08-11T15:52:25.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.654+0000 7f06f93db640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f06d806a050 0x7f06d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:25.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.655+0000 7f06f93db640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f06d806a050 0x7f06d806c510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f06f415fed0 tx=0x7f06f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:25.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.658+0000 7f06e9ffb640 1 -- 172.21.15.27:0/3280451993 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f06e4075d50 con 0x7f06f40cc3f0 2023-08-11T15:52:25.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.856+0000 7f06fb63f640 1 -- 172.21.15.27:0/3280451993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06f40d34b0 con 0x7f06f40cc3f0 2023-08-11T15:52:25.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.856+0000 7f06e9ffb640 1 -- 172.21.15.27:0/3280451993 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06e40044e0 con 0x7f06f40cc3f0 2023-08-11T15:52:25.860 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:25.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 -- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f06d806a050 msgr2=0x7f06d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f06d806a050 0x7f06d806c510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f06f415fed0 tx=0x7f06f0008040 comp rx=0 tx=0).stop 2023-08-11T15:52:25.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 -- 172.21.15.27:0/3280451993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f40cc3f0 msgr2=0x7f06f4157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:25.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f40cc3f0 0x7f06f4157bd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009be0 tx=0x7f06e4004730 comp rx=0 tx=0).stop 2023-08-11T15:52:25.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 -- 172.21.15.27:0/3280451993 shutdown_connections 2023-08-11T15:52:25.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f06d806a050 0x7f06d806c510 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f40b2900 0x7f06f40b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f40cc3f0 0x7f06f4157bd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 --2- 172.21.15.27:0/3280451993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06e0003ef0 0x7f06f40c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:25.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 -- 172.21.15.27:0/3280451993 >> 172.21.15.27:0/3280451993 conn(0x7f06f4089ff0 msgr2=0x7f06f4160240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:25.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.859+0000 7f06fb63f640 1 -- 172.21.15.27:0/3280451993 shutdown_connections 2023-08-11T15:52:25.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:25.860+0000 7f06fb63f640 1 -- 172.21.15.27:0/3280451993 wait complete. 2023-08-11T15:52:25.880 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:26.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.107+0000 7f3219dcb640 1 Processor -- start 2023-08-11T15:52:26.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.108+0000 7f3219dcb640 1 -- start start 2023-08-11T15:52:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3219dcb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f321415c2a0 0x7f32140b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3219dcb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f32140b45d0 0x7f32140ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3219dcb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32140cd890 con 0x7f32140ceb70 2023-08-11T15:52:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3219dcb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32140b4140 con 0x7f321415c2a0 2023-08-11T15:52:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3219dcb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32140cd050 con 0x7f32140b45d0 2023-08-11T15:52:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3218dc9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f321415c2a0 0x7f32140b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3218dc9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f321415c2a0 0x7f32140b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60504/0 (socket says 172.21.15.27:60504) 2023-08-11T15:52:26.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3218dc9640 1 -- 172.21.15.27:0/89768580 learned_addr learned my addr 172.21.15.27:0/89768580 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3213fff640 1 --2- 172.21.15.27:0/89768580 >> v2:172.21.15.119:3300/0 conn(0x7f32140b45d0 0x7f32140ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3218dc9640 1 -- 172.21.15.27:0/89768580 >> v2:172.21.15.119:3300/0 conn(0x7f32140b45d0 msgr2=0x7f32140ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.111+0000 7f3218dc9640 1 --2- 172.21.15.27:0/89768580 >> v2:172.21.15.119:3300/0 conn(0x7f32140b45d0 0x7f32140ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.112+0000 7f3218dc9640 1 -- 172.21.15.27:0/89768580 >> v1:172.21.15.27:6789/0 conn(0x7f32140ceb70 legacy=0x7f32140b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.112+0000 7f3218dc9640 1 -- 172.21.15.27:0/89768580 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32140cd2d0 con 0x7f321415c2a0 2023-08-11T15:52:26.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.113+0000 7f3218dc9640 1 --2- 172.21.15.27:0/89768580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321415c2a0 0x7f32140b1580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f32140b1ac0 tx=0x7f320802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b30e5ed01cc73b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:26.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.113+0000 7f32137fe640 1 -- 172.21.15.27:0/89768580 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208031870 con 0x7f321415c2a0 2023-08-11T15:52:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.113+0000 7f32137fe640 1 -- 172.21.15.27:0/89768580 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32080319d0 con 0x7f321415c2a0 2023-08-11T15:52:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.113+0000 7f32137fe640 1 -- 172.21.15.27:0/89768580 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208031d60 con 0x7f321415c2a0 2023-08-11T15:52:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.114+0000 7f3219dcb640 1 -- 172.21.15.27:0/89768580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321415c2a0 msgr2=0x7f32140b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.114+0000 7f3219dcb640 1 --2- 172.21.15.27:0/89768580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321415c2a0 0x7f32140b1580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f32140b1ac0 tx=0x7f320802b910 comp rx=0 tx=0).stop 2023-08-11T15:52:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.114+0000 7f3219dcb640 1 -- 172.21.15.27:0/89768580 shutdown_connections 2023-08-11T15:52:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.114+0000 7f3219dcb640 1 --2- 172.21.15.27:0/89768580 >> v2:172.21.15.119:3300/0 conn(0x7f32140b45d0 0x7f32140ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.114+0000 7f3219dcb640 1 --2- 172.21.15.27:0/89768580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321415c2a0 0x7f32140b1580 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.114+0000 7f3219dcb640 1 -- 172.21.15.27:0/89768580 >> 172.21.15.27:0/89768580 conn(0x7f321406a250 msgr2=0x7f321406a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.114+0000 7f3219dcb640 1 -- 172.21.15.27:0/89768580 shutdown_connections 2023-08-11T15:52:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.114+0000 7f3219dcb640 1 -- 172.21.15.27:0/89768580 wait complete. 2023-08-11T15:52:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.116+0000 7f3219dcb640 1 Processor -- start 2023-08-11T15:52:26.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.116+0000 7f3219dcb640 1 -- start start 2023-08-11T15:52:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.117+0000 7f3219dcb640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32140b45d0 0x7f32140b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.117+0000 7f3219dcb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32140ceb70 0x7f321415c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.117+0000 7f3219dcb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f321415c2a0 0x7f32140b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.117+0000 7f3219dcb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32141f1ac0 con 0x7f32140ceb70 2023-08-11T15:52:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.117+0000 7f3219dcb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32141f1d60 con 0x7f321415c2a0 2023-08-11T15:52:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.117+0000 7f3219dcb640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321415cff0 con 0x7f32140b45d0 2023-08-11T15:52:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.117+0000 7f3213fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32140ceb70 0x7f321415c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.117+0000 7f3213fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32140ceb70 0x7f321415c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60508/0 (socket says 172.21.15.27:60508) 2023-08-11T15:52:26.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.117+0000 7f3213fff640 1 -- 172.21.15.27:0/2301587040 learned_addr learned my addr 172.21.15.27:0/2301587040 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:26.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.118+0000 7f3218dc9640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32140b45d0 0x7f32140b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.118+0000 7f32195ca640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f321415c2a0 0x7f32140b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.118+0000 7f3213fff640 1 -- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32140b45d0 msgr2=0x7f32140b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.118+0000 7f3213fff640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32140b45d0 0x7f32140b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.118+0000 7f3213fff640 1 -- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f321415c2a0 msgr2=0x7f32140b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.118+0000 7f3213fff640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f321415c2a0 0x7f32140b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.118+0000 7f3213fff640 1 -- 172.21.15.27:0/2301587040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080094d0 con 0x7f32140ceb70 2023-08-11T15:52:26.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.118+0000 7f3218dc9640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32140b45d0 0x7f32140b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:26.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.118+0000 7f32195ca640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f321415c2a0 0x7f32140b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:26.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.118+0000 7f3213fff640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32140ceb70 0x7f321415c680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f320000c7f0 tx=0x7f320000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:26.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.119+0000 7f32117fa640 1 -- 172.21.15.27:0/2301587040 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3200004220 con 0x7f32140ceb70 2023-08-11T15:52:26.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.119+0000 7f3219dcb640 1 -- 172.21.15.27:0/2301587040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321415d2d0 con 0x7f32140ceb70 2023-08-11T15:52:26.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.119+0000 7f32117fa640 1 -- 172.21.15.27:0/2301587040 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32000043a0 con 0x7f32140ceb70 2023-08-11T15:52:26.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.119+0000 7f32117fa640 1 -- 172.21.15.27:0/2301587040 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3200007860 con 0x7f32140ceb70 2023-08-11T15:52:26.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.119+0000 7f3219dcb640 1 -- 172.21.15.27:0/2301587040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32140b0750 con 0x7f32140ceb70 2023-08-11T15:52:26.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.120+0000 7f3219dcb640 1 -- 172.21.15.27:0/2301587040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32140cdae0 con 0x7f32140ceb70 2023-08-11T15:52:26.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.121+0000 7f32117fa640 1 -- 172.21.15.27:0/2301587040 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3200007bd0 con 0x7f32140ceb70 2023-08-11T15:52:26.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.122+0000 7f32117fa640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f31f806a050 0x7f31f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.122+0000 7f32117fa640 1 -- 172.21.15.27:0/2301587040 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3200088940 con 0x7f32140ceb70 2023-08-11T15:52:26.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.123+0000 7f3218dc9640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f31f806a050 0x7f31f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.124+0000 7f3218dc9640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f31f806a050 0x7f31f806c510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3208036000 tx=0x7f320803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:26.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.126+0000 7f32117fa640 1 -- 172.21.15.27:0/2301587040 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f3200053c60 con 0x7f32140ceb70 2023-08-11T15:52:26.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.318+0000 7f3219dcb640 1 -- 172.21.15.27:0/2301587040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f321415d7f0 con 0x7f32140ceb70 2023-08-11T15:52:26.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.319+0000 7f32117fa640 1 -- 172.21.15.27:0/2301587040 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f320001e020 con 0x7f32140ceb70 2023-08-11T15:52:26.322 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:26.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 -- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f31f806a050 msgr2=0x7f31f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f31f806a050 0x7f31f806c510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3208036000 tx=0x7f320803c040 comp rx=0 tx=0).stop 2023-08-11T15:52:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 -- 172.21.15.27:0/2301587040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32140ceb70 msgr2=0x7f321415c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32140ceb70 0x7f321415c680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f320000c7f0 tx=0x7f320000ccb0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 -- 172.21.15.27:0/2301587040 shutdown_connections 2023-08-11T15:52:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f31f806a050 0x7f31f806c510 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f321415c2a0 0x7f32140b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32140ceb70 0x7f321415c680 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 --2- 172.21.15.27:0/2301587040 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32140b45d0 0x7f32140b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 -- 172.21.15.27:0/2301587040 >> 172.21.15.27:0/2301587040 conn(0x7f321406a250 msgr2=0x7f32140b6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:26.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 -- 172.21.15.27:0/2301587040 shutdown_connections 2023-08-11T15:52:26.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.322+0000 7f3219dcb640 1 -- 172.21.15.27:0/2301587040 wait complete. 2023-08-11T15:52:26.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:26.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.584+0000 7fa87b59e640 1 Processor -- start 2023-08-11T15:52:26.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.584+0000 7fa87b59e640 1 -- start start 2023-08-11T15:52:26.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.586+0000 7fa87b59e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa87c15c430 0x7fa87c0b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.586+0000 7fa87b59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c0d0c90 con 0x7fa87c0d1d30 2023-08-11T15:52:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.586+0000 7fa87b59e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c0d0f70 con 0x7fa87c0cf340 2023-08-11T15:52:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.586+0000 7fa87b59e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0d1910 con 0x7fa87c15c430 2023-08-11T15:52:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa871d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa87c0d1d30 0x7fa87c0cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47160/0 (socket says 172.21.15.27:47160) 2023-08-11T15:52:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa871d9b640 1 -- 172.21.15.27:0/2885082346 learned_addr learned my addr 172.21.15.27:0/2885082346 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa87a59c640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa87c15c430 0x7fa87c0b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410328862 0 0) 0x7fa87c0d0c90 con 0x7fa87c0d1d30 2023-08-11T15:52:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa868003520 con 0x7fa87c0d1d30 2023-08-11T15:52:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1170682793 0 0) 0x7fa868003520 con 0x7fa87c0d1d30 2023-08-11T15:52:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa87c15c430 msgr2=0x7fa87c0b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa879d9b640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa87c15c430 0x7fa87c0b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 >> v1:172.21.15.119:6789/0 conn(0x7fa87c0cf340 legacy=0x7fa87c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0d1250 con 0x7fa87c0d1d30 2023-08-11T15:52:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.587+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fa864001670 con 0x7fa87c0d1d30 2023-08-11T15:52:26.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 >> v1:172.21.15.27:6789/0 conn(0x7fa87c0d1d30 legacy=0x7fa87c0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa879d9b640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8680039a0 0x7fa868003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa879d9b640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa8680047c0 0x7fa868004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa879d9b640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680051d0 0x7fa86800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0d1910 con 0x7fa8680039a0 2023-08-11T15:52:26.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86800c0d0 con 0x7fa8680051d0 2023-08-11T15:52:26.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86800c290 con 0x7fa8680047c0 2023-08-11T15:52:26.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa87a59c640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8680039a0 0x7fa868003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa871d9b640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa8680047c0 0x7fa868004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa87ad9d640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680051d0 0x7fa86800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa87a59c640 1 -- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa8680047c0 msgr2=0x7fa868004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:26.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa87a59c640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa8680047c0 0x7fa868004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa87a59c640 1 -- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680051d0 msgr2=0x7fa86800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:26.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa87a59c640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680051d0 0x7fa86800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa87a59c640 1 -- 172.21.15.27:0/2885082346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa868003520 con 0x7fa8680039a0 2023-08-11T15:52:26.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa871d9b640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa8680047c0 0x7fa868004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:26.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.588+0000 7fa87ad9d640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680051d0 0x7fa86800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:26.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.589+0000 7fa87a59c640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8680039a0 0x7fa868003dc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0d32d0 tx=0x7fa8680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7ae25e735e87e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:26.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.590+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c034610 con 0x7fa8680039a0 2023-08-11T15:52:26.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.590+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c02e070 con 0x7fa8680039a0 2023-08-11T15:52:26.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.590+0000 7fa879d9b640 1 -- 172.21.15.27:0/2885082346 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c034990 con 0x7fa8680039a0 2023-08-11T15:52:26.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.590+0000 7fa87b59e640 1 -- 172.21.15.27:0/2885082346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8680039a0 msgr2=0x7fa868003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.590+0000 7fa87b59e640 1 --2- 172.21.15.27:0/2885082346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8680039a0 0x7fa868003dc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0d32d0 tx=0x7fa8680057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.590+0000 7fa87ad9d640 1 -- 172.21.15.27:0/2885082346 reap_dead start 2023-08-11T15:52:26.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.590+0000 7fa87b59e640 1 -- 172.21.15.27:0/2885082346 shutdown_connections 2023-08-11T15:52:26.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.590+0000 7fa87b59e640 1 -- 172.21.15.27:0/2885082346 >> 172.21.15.27:0/2885082346 conn(0x7fa87c06a2b0 msgr2=0x7fa87c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:26.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.591+0000 7fa87b59e640 1 -- 172.21.15.27:0/2885082346 shutdown_connections 2023-08-11T15:52:26.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.591+0000 7fa87b59e640 1 -- 172.21.15.27:0/2885082346 wait complete. 2023-08-11T15:52:26.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.592+0000 7fa87b59e640 1 Processor -- start 2023-08-11T15:52:26.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.593+0000 7fa87b59e640 1 -- start start 2023-08-11T15:52:26.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.593+0000 7fa87b59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680039a0 0x7fa87c0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.593+0000 7fa87b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa87c0cc3a0 0x7fa87c154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.593+0000 7fa87b59e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa87c1531d0 0x7fa87c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.593+0000 7fa87b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0b8280 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.593+0000 7fa87b59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0cace0 con 0x7fa8680039a0 2023-08-11T15:52:26.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.593+0000 7fa87b59e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0c9c40 con 0x7fa87c1531d0 2023-08-11T15:52:26.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa871d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa87c0cc3a0 0x7fa87c154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa871d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa87c0cc3a0 0x7fa87c154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60530/0 (socket says 172.21.15.27:60530) 2023-08-11T15:52:26.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa871d9b640 1 -- 172.21.15.27:0/3309848834 learned_addr learned my addr 172.21.15.27:0/3309848834 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:26.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa87a59c640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680039a0 0x7fa87c0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa87ad9d640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa87c1531d0 0x7fa87c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa871d9b640 1 -- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa87c1531d0 msgr2=0x7fa87c154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa871d9b640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa87c1531d0 0x7fa87c154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa871d9b640 1 -- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680039a0 msgr2=0x7fa87c0c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa871d9b640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680039a0 0x7fa87c0c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa871d9b640 1 -- 172.21.15.27:0/3309848834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0c9f20 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa87a59c640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680039a0 0x7fa87c0c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:26.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.594+0000 7fa87ad9d640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa87c1531d0 0x7fa87c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:26.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.595+0000 7fa871d9b640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa87c0cc3a0 0x7fa87c154420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa86400d810 tx=0x7fa86400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:26.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.595+0000 7fa8737fe640 1 -- 172.21.15.27:0/3309848834 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640074e0 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.595+0000 7fa8737fe640 1 -- 172.21.15.27:0/3309848834 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa864007640 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.595+0000 7fa8737fe640 1 -- 172.21.15.27:0/3309848834 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640049e0 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.595+0000 7fa87b59e640 1 -- 172.21.15.27:0/3309848834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c0ca200 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.595+0000 7fa87b59e640 1 -- 172.21.15.27:0/3309848834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c0ca7c0 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.596+0000 7fa87b59e640 1 -- 172.21.15.27:0/3309848834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87c04e420 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.597+0000 7fa8737fe640 1 -- 172.21.15.27:0/3309848834 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa864004b40 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.598+0000 7fa8737fe640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa85c06a080 0x7fa85c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:26.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.598+0000 7fa8737fe640 1 -- 172.21.15.27:0/3309848834 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa864088350 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.599+0000 7fa87a59c640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa85c06a080 0x7fa85c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:26.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.600+0000 7fa87a59c640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa85c06a080 0x7fa85c06c540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa87c071f60 tx=0x7fa86c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:26.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.603+0000 7fa8737fe640 1 -- 172.21.15.27:0/3309848834 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fa864053670 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.802+0000 7fa87b59e640 1 -- 172.21.15.27:0/3309848834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa87c0b2000 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.803+0000 7fa8737fe640 1 -- 172.21.15.27:0/3309848834 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa864056c90 con 0x7fa87c0cc3a0 2023-08-11T15:52:26.806 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:26.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.805+0000 7fa87b59e640 1 -- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa85c06a080 msgr2=0x7fa85c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.805+0000 7fa87b59e640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa85c06a080 0x7fa85c06c540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa87c071f60 tx=0x7fa86c030000 comp rx=0 tx=0).stop 2023-08-11T15:52:26.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.806+0000 7fa87b59e640 1 -- 172.21.15.27:0/3309848834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa87c0cc3a0 msgr2=0x7fa87c154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:26.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.806+0000 7fa87b59e640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa87c0cc3a0 0x7fa87c154420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa86400d810 tx=0x7fa86400dcd0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.806+0000 7fa87b59e640 1 -- 172.21.15.27:0/3309848834 shutdown_connections 2023-08-11T15:52:26.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.806+0000 7fa87b59e640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa85c06a080 0x7fa85c06c540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.806+0000 7fa87b59e640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa87c1531d0 0x7fa87c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.806+0000 7fa87b59e640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa87c0cc3a0 0x7fa87c154420 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.806+0000 7fa87b59e640 1 --2- 172.21.15.27:0/3309848834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8680039a0 0x7fa87c0c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:26.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.806+0000 7fa87b59e640 1 -- 172.21.15.27:0/3309848834 >> 172.21.15.27:0/3309848834 conn(0x7fa87c06a2b0 msgr2=0x7fa87c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:26.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.806+0000 7fa87b59e640 1 -- 172.21.15.27:0/3309848834 shutdown_connections 2023-08-11T15:52:26.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:26.806+0000 7fa87b59e640 1 -- 172.21.15.27:0/3309848834 wait complete. 2023-08-11T15:52:26.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:27.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.055+0000 7f66e23e1640 1 Processor -- start 2023-08-11T15:52:27.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.055+0000 7f66e23e1640 1 -- start start 2023-08-11T15:52:27.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.057+0000 7f66e23e1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66dc15a310 0x7f66dc06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.057+0000 7f66e23e1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66dc0ccb00 con 0x7f66dc0ca690 2023-08-11T15:52:27.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.057+0000 7f66e23e1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66dc0ccde0 con 0x7f66dc06cf00 2023-08-11T15:52:27.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.057+0000 7f66e23e1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc06d370 con 0x7f66dc15a310 2023-08-11T15:52:27.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.057+0000 7f66dbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f66dc0ca690 0x7f66dc0cd8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47168/0 (socket says 172.21.15.27:47168) 2023-08-11T15:52:27.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.057+0000 7f66dbfff640 1 -- 172.21.15.27:0/1091103092 learned_addr learned my addr 172.21.15.27:0/1091103092 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66db7fe640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66dc15a310 0x7f66dc06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273366738 0 0) 0x7f66dc0ccb00 con 0x7f66dc0ca690 2023-08-11T15:52:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c8003520 con 0x7f66dc0ca690 2023-08-11T15:52:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2823292349 0 0) 0x7f66c8003520 con 0x7f66dc0ca690 2023-08-11T15:52:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66dc15a310 msgr2=0x7f66dc06c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66dc15a310 0x7f66dc06c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 >> v1:172.21.15.119:6789/0 conn(0x7f66dc06cf00 legacy=0x7f66dc0c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc0cd0c0 con 0x7f66dc0ca690 2023-08-11T15:52:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f66cc001a10 con 0x7f66dc0ca690 2023-08-11T15:52:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 >> v1:172.21.15.27:6789/0 conn(0x7f66dc0ca690 legacy=0x7f66dc0cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66c80039a0 0x7f66c8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c80047c0 0x7f66c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66dc15a310 0x7f66c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc06d370 con 0x7f66c80047c0 2023-08-11T15:52:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c800ba30 con 0x7f66c80039a0 2023-08-11T15:52:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.058+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c800bbf0 con 0x7f66dc15a310 2023-08-11T15:52:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.059+0000 7f66db7fe640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c80047c0 0x7f66c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.059+0000 7f66dbfff640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66c80039a0 0x7f66c8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.059+0000 7f66db7fe640 1 -- 172.21.15.27:0/1091103092 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66dc15a310 msgr2=0x7f66c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.059+0000 7f66db7fe640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66dc15a310 0x7f66c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.059+0000 7f66db7fe640 1 -- 172.21.15.27:0/1091103092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66c80039a0 msgr2=0x7f66c8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.059+0000 7f66db7fe640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66c80039a0 0x7f66c8003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.059+0000 7f66db7fe640 1 -- 172.21.15.27:0/1091103092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8003520 con 0x7f66c80047c0 2023-08-11T15:52:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.060+0000 7f66db7fe640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c80047c0 0x7f66c8004c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f66c400bd30 tx=0x7f66c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11fa65be549ed78 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.060+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c40306e0 con 0x7f66c80047c0 2023-08-11T15:52:27.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.060+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c4030840 con 0x7f66c80047c0 2023-08-11T15:52:27.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.060+0000 7f66daffd640 1 -- 172.21.15.27:0/1091103092 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4030bd0 con 0x7f66c80047c0 2023-08-11T15:52:27.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.061+0000 7f66e23e1640 1 -- 172.21.15.27:0/1091103092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c80047c0 msgr2=0x7f66c8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.061+0000 7f66e23e1640 1 --2- 172.21.15.27:0/1091103092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c80047c0 0x7f66c8004c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f66c400bd30 tx=0x7f66c80057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.061+0000 7f66e097e640 1 -- 172.21.15.27:0/1091103092 reap_dead start 2023-08-11T15:52:27.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.061+0000 7f66e23e1640 1 -- 172.21.15.27:0/1091103092 shutdown_connections 2023-08-11T15:52:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.061+0000 7f66e23e1640 1 -- 172.21.15.27:0/1091103092 >> 172.21.15.27:0/1091103092 conn(0x7f66dc0afbe0 msgr2=0x7f66dc0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.061+0000 7f66e23e1640 1 -- 172.21.15.27:0/1091103092 shutdown_connections 2023-08-11T15:52:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.061+0000 7f66e23e1640 1 -- 172.21.15.27:0/1091103092 wait complete. 2023-08-11T15:52:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.063+0000 7f66e23e1640 1 Processor -- start 2023-08-11T15:52:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.063+0000 7f66e23e1640 1 -- start start 2023-08-11T15:52:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.064+0000 7f66e23e1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66c80047c0 0x7f66dc0c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.064+0000 7f66e23e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66dc151280 0x7f66dc151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.064+0000 7f66e23e1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66dc154730 0x7f66dc154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.064+0000 7f66e23e1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc151c40 con 0x7f66dc154730 2023-08-11T15:52:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.064+0000 7f66e23e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc1f2fd0 con 0x7f66dc151280 2023-08-11T15:52:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.064+0000 7f66e23e1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc1f32b0 con 0x7f66c80047c0 2023-08-11T15:52:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.064+0000 7f66e097e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66dc154730 0x7f66dc154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.065+0000 7f66e097e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66dc154730 0x7f66dc154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60554/0 (socket says 172.21.15.27:60554) 2023-08-11T15:52:27.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.065+0000 7f66e097e640 1 -- 172.21.15.27:0/3485476907 learned_addr learned my addr 172.21.15.27:0/3485476907 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:27.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.065+0000 7f66e097e640 1 -- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66c80047c0 msgr2=0x7f66dc0c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.065+0000 7f66e097e640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66c80047c0 0x7f66dc0c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.065+0000 7f66e097e640 1 -- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66dc151280 msgr2=0x7f66dc151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.065+0000 7f66e097e640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66dc151280 0x7f66dc151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.065+0000 7f66e097e640 1 -- 172.21.15.27:0/3485476907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc1f3590 con 0x7f66dc154730 2023-08-11T15:52:27.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.065+0000 7f66e097e640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66dc154730 0x7f66dc154bb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f66dc157310 tx=0x7f66d400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:27.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.065+0000 7f66d8ff9640 1 -- 172.21.15.27:0/3485476907 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d40072e0 con 0x7f66dc154730 2023-08-11T15:52:27.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.066+0000 7f66d8ff9640 1 -- 172.21.15.27:0/3485476907 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d4007440 con 0x7f66dc154730 2023-08-11T15:52:27.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.066+0000 7f66d8ff9640 1 -- 172.21.15.27:0/3485476907 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4004020 con 0x7f66dc154730 2023-08-11T15:52:27.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.066+0000 7f66e23e1640 1 -- 172.21.15.27:0/3485476907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66dc1f3870 con 0x7f66dc154730 2023-08-11T15:52:27.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.066+0000 7f66e23e1640 1 -- 172.21.15.27:0/3485476907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66dc0c4b30 con 0x7f66dc154730 2023-08-11T15:52:27.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.067+0000 7f66e23e1640 1 -- 172.21.15.27:0/3485476907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66dc1f3d90 con 0x7f66dc154730 2023-08-11T15:52:27.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.068+0000 7f66d8ff9640 1 -- 172.21.15.27:0/3485476907 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f66d40041a0 con 0x7f66dc154730 2023-08-11T15:52:27.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.069+0000 7f66d8ff9640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f66bc06a050 0x7f66bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.070+0000 7f66d8ff9640 1 -- 172.21.15.27:0/3485476907 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f66d4087aa0 con 0x7f66dc154730 2023-08-11T15:52:27.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.070+0000 7f66dbfff640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f66bc06a050 0x7f66bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:27.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.071+0000 7f66dbfff640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f66bc06a050 0x7f66bc06c510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f66cc007c40 tx=0x7f66cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:27.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.074+0000 7f66d8ff9640 1 -- 172.21.15.27:0/3485476907 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f66dc1f3d90 con 0x7f66dc154730 2023-08-11T15:52:27.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.275+0000 7f66e23e1640 1 -- 172.21.15.27:0/3485476907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f66dc1f3d90 con 0x7f66dc154730 2023-08-11T15:52:27.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.276+0000 7f66d8ff9640 1 -- 172.21.15.27:0/3485476907 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66dc1f3d90 con 0x7f66dc154730 2023-08-11T15:52:27.279 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:27.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.279+0000 7f66e23e1640 1 -- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f66bc06a050 msgr2=0x7f66bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.279+0000 7f66e23e1640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f66bc06a050 0x7f66bc06c510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f66cc007c40 tx=0x7f66cc009040 comp rx=0 tx=0).stop 2023-08-11T15:52:27.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.279+0000 7f66e23e1640 1 -- 172.21.15.27:0/3485476907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66dc154730 msgr2=0x7f66dc154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.279+0000 7f66e23e1640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66dc154730 0x7f66dc154bb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f66dc157310 tx=0x7f66d400c820 comp rx=0 tx=0).stop 2023-08-11T15:52:27.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.279+0000 7f66e23e1640 1 -- 172.21.15.27:0/3485476907 shutdown_connections 2023-08-11T15:52:27.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.279+0000 7f66e23e1640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f66bc06a050 0x7f66bc06c510 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.279+0000 7f66e23e1640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66dc154730 0x7f66dc154bb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.279+0000 7f66e23e1640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66dc151280 0x7f66dc151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.279+0000 7f66e23e1640 1 --2- 172.21.15.27:0/3485476907 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66c80047c0 0x7f66dc0c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.280+0000 7f66e23e1640 1 -- 172.21.15.27:0/3485476907 >> 172.21.15.27:0/3485476907 conn(0x7f66dc0afbe0 msgr2=0x7f66dc06b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:27.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.280+0000 7f66e23e1640 1 -- 172.21.15.27:0/3485476907 shutdown_connections 2023-08-11T15:52:27.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.280+0000 7f66e23e1640 1 -- 172.21.15.27:0/3485476907 wait complete. 2023-08-11T15:52:27.303 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:27.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.529+0000 7f04a3804640 1 Processor -- start 2023-08-11T15:52:27.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.529+0000 7f04a3804640 1 -- start start 2023-08-11T15:52:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.531+0000 7f04a3804640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f049c151220 0x7f049c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.531+0000 7f04a3804640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049c06c850 con 0x7f049c06bdf0 2023-08-11T15:52:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.531+0000 7f04a3804640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049c06c290 con 0x7f049c15e680 2023-08-11T15:52:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.531+0000 7f04a3804640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049c06c570 con 0x7f049c151220 2023-08-11T15:52:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.531+0000 7f04a15a0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f049c151220 0x7f049c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.531+0000 7f04a15a0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f049c151220 0x7f049c06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60570/0 (socket says 172.21.15.27:60570) 2023-08-11T15:52:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.531+0000 7f04a15a0640 1 -- 172.21.15.27:0/3026558091 learned_addr learned my addr 172.21.15.27:0/3026558091 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.531+0000 7f04a0d9f640 1 --1- 172.21.15.27:0/3026558091 >> v1:172.21.15.27:6789/0 conn(0x7f049c06bdf0 0x7f049c15df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47174/0 (socket says 172.21.15.27:47174) 2023-08-11T15:52:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.532+0000 7f04a15a0640 1 -- 172.21.15.27:0/3026558091 >> v1:172.21.15.119:6789/0 conn(0x7f049c15e680 legacy=0x7f049c0ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.532+0000 7f04a15a0640 1 -- 172.21.15.27:0/3026558091 >> v1:172.21.15.27:6789/0 conn(0x7f049c06bdf0 legacy=0x7f049c15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.532+0000 7f04a15a0640 1 -- 172.21.15.27:0/3026558091 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c0ca050 con 0x7f049c151220 2023-08-11T15:52:27.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.532+0000 7f0493fff640 1 -- 172.21.15.27:0/3026558091 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152402814 0 0) 0x7f049c06c850 con 0x7f049c06bdf0 2023-08-11T15:52:27.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.533+0000 7f04a15a0640 1 --2- 172.21.15.27:0/3026558091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c151220 0x7f049c06b8b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0498009870 tx=0x7f049802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9862f752b1a314 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.533+0000 7f0493fff640 1 -- 172.21.15.27:0/3026558091 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04980317e0 con 0x7f049c151220 2023-08-11T15:52:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.533+0000 7f0493fff640 1 -- 172.21.15.27:0/3026558091 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0498031940 con 0x7f049c151220 2023-08-11T15:52:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.533+0000 7f0493fff640 1 -- 172.21.15.27:0/3026558091 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498031cd0 con 0x7f049c151220 2023-08-11T15:52:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.534+0000 7f04a3804640 1 -- 172.21.15.27:0/3026558091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c151220 msgr2=0x7f049c06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.534+0000 7f04a3804640 1 --2- 172.21.15.27:0/3026558091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c151220 0x7f049c06b8b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0498009870 tx=0x7f049802b880 comp rx=0 tx=0).stop 2023-08-11T15:52:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.534+0000 7f04a3804640 1 -- 172.21.15.27:0/3026558091 shutdown_connections 2023-08-11T15:52:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.534+0000 7f04a3804640 1 --2- 172.21.15.27:0/3026558091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c151220 0x7f049c06b8b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.534+0000 7f04a3804640 1 -- 172.21.15.27:0/3026558091 >> 172.21.15.27:0/3026558091 conn(0x7f049c0afbe0 msgr2=0x7f049c0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.534+0000 7f04a3804640 1 -- 172.21.15.27:0/3026558091 shutdown_connections 2023-08-11T15:52:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.534+0000 7f04a3804640 1 -- 172.21.15.27:0/3026558091 wait complete. 2023-08-11T15:52:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.536+0000 7f04a3804640 1 Processor -- start 2023-08-11T15:52:27.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.536+0000 7f04a3804640 1 -- start start 2023-08-11T15:52:27.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.537+0000 7f04a3804640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c06bdf0 0x7f049c1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.537+0000 7f04a3804640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c151220 0x7f049c0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.537+0000 7f04a3804640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f049c15e680 0x7f049c159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.537+0000 7f04a3804640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c150af0 con 0x7f049c06bdf0 2023-08-11T15:52:27.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.537+0000 7f04a3804640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c0cdc00 con 0x7f049c151220 2023-08-11T15:52:27.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.537+0000 7f04a3804640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049c0c4b70 con 0x7f049c15e680 2023-08-11T15:52:27.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.538+0000 7f04a15a0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c06bdf0 0x7f049c1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:27.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.538+0000 7f04a15a0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c06bdf0 0x7f049c1505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60586/0 (socket says 172.21.15.27:60586) 2023-08-11T15:52:27.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.538+0000 7f04a15a0640 1 -- 172.21.15.27:0/2631147929 learned_addr learned my addr 172.21.15.27:0/2631147929 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:27.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.538+0000 7f04a0d9f640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c151220 0x7f049c0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:27.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.538+0000 7f04a15a0640 1 -- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f049c15e680 msgr2=0x7f049c159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.538+0000 7f04a15a0640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f049c15e680 0x7f049c159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.538+0000 7f04a15a0640 1 -- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c151220 msgr2=0x7f049c0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.538+0000 7f04a15a0640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c151220 0x7f049c0cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.538+0000 7f04a15a0640 1 -- 172.21.15.27:0/2631147929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04980094d0 con 0x7f049c06bdf0 2023-08-11T15:52:27.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.539+0000 7f04a15a0640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c06bdf0 0x7f049c1505b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f04980090b0 tx=0x7f04980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:27.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.539+0000 7f0491ffb640 1 -- 172.21.15.27:0/2631147929 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498004230 con 0x7f049c06bdf0 2023-08-11T15:52:27.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.539+0000 7f0491ffb640 1 -- 172.21.15.27:0/2631147929 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0498004390 con 0x7f049c06bdf0 2023-08-11T15:52:27.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.539+0000 7f0491ffb640 1 -- 172.21.15.27:0/2631147929 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498003830 con 0x7f049c06bdf0 2023-08-11T15:52:27.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.539+0000 7f04a3804640 1 -- 172.21.15.27:0/2631147929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049c15c4c0 con 0x7f049c06bdf0 2023-08-11T15:52:27.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.539+0000 7f04a3804640 1 -- 172.21.15.27:0/2631147929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049c0ccde0 con 0x7f049c06bdf0 2023-08-11T15:52:27.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.540+0000 7f04a3804640 1 -- 172.21.15.27:0/2631147929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049c04e420 con 0x7f049c06bdf0 2023-08-11T15:52:27.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.541+0000 7f0491ffb640 1 -- 172.21.15.27:0/2631147929 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0498043070 con 0x7f049c06bdf0 2023-08-11T15:52:27.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.542+0000 7f0491ffb640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f048006a0d0 0x7f048006c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.542+0000 7f0491ffb640 1 -- 172.21.15.27:0/2631147929 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f04980aa870 con 0x7f049c06bdf0 2023-08-11T15:52:27.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.543+0000 7f04a0d9f640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f048006a0d0 0x7f048006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:27.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.544+0000 7f04a0d9f640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f048006a0d0 0x7f048006c590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f049c0cd340 tx=0x7f048c007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:27.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.547+0000 7f0491ffb640 1 -- 172.21.15.27:0/2631147929 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0498075b90 con 0x7f049c06bdf0 2023-08-11T15:52:27.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.740+0000 7f04a3804640 1 -- 172.21.15.27:0/2631147929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f049c06d380 con 0x7f049c06bdf0 2023-08-11T15:52:27.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.741+0000 7f0491ffb640 1 -- 172.21.15.27:0/2631147929 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f04980791b0 con 0x7f049c06bdf0 2023-08-11T15:52:27.743 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:27.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 -- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f048006a0d0 msgr2=0x7f048006c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f048006a0d0 0x7f048006c590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f049c0cd340 tx=0x7f048c007b50 comp rx=0 tx=0).stop 2023-08-11T15:52:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 -- 172.21.15.27:0/2631147929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c06bdf0 msgr2=0x7f049c1505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c06bdf0 0x7f049c1505b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f04980090b0 tx=0x7f04980046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 -- 172.21.15.27:0/2631147929 shutdown_connections 2023-08-11T15:52:27.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f048006a0d0 0x7f048006c590 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f049c15e680 0x7f049c159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c151220 0x7f049c0cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 --2- 172.21.15.27:0/2631147929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f049c06bdf0 0x7f049c1505b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:27.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 -- 172.21.15.27:0/2631147929 >> 172.21.15.27:0/2631147929 conn(0x7f049c0afbe0 msgr2=0x7f049c154840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:27.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 -- 172.21.15.27:0/2631147929 shutdown_connections 2023-08-11T15:52:27.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.743+0000 7f04a3804640 1 -- 172.21.15.27:0/2631147929 wait complete. 2023-08-11T15:52:27.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:52:27.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.993+0000 7f9c52f34640 1 Processor -- start 2023-08-11T15:52:27.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.994+0000 7f9c52f34640 1 -- start start 2023-08-11T15:52:27.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.996+0000 7f9c52f34640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9c4c0c8790 0x7f9c4c06c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:27.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.996+0000 7f9c52f34640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4c15b760 con 0x7f9c4c13d310 2023-08-11T15:52:27.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.996+0000 7f9c52f34640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4c0c8c00 con 0x7f9c4c0c8300 2023-08-11T15:52:27.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.996+0000 7f9c52f34640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c1407b0 con 0x7f9c4c0c8790 2023-08-11T15:52:27.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.996+0000 7f9c51f32640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9c4c13d310 0x7f9c4c15a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47182/0 (socket says 172.21.15.27:47182) 2023-08-11T15:52:27.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.996+0000 7f9c51f32640 1 -- 172.21.15.27:0/976929701 learned_addr learned my addr 172.21.15.27:0/976929701 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:28.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.996+0000 7f9c52733640 1 --2- 172.21.15.27:0/976929701 >> v2:172.21.15.119:3300/0 conn(0x7f9c4c0c8790 0x7f9c4c06c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.996+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248274636 0 0) 0x7f9c4c15b760 con 0x7f9c4c13d310 2023-08-11T15:52:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.996+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c38003520 con 0x7f9c4c13d310 2023-08-11T15:52:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1183044312 0 0) 0x7f9c38003520 con 0x7f9c4c13d310 2023-08-11T15:52:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 >> v2:172.21.15.119:3300/0 conn(0x7f9c4c0c8790 msgr2=0x7f9c4c06c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 --2- 172.21.15.27:0/976929701 >> v2:172.21.15.119:3300/0 conn(0x7f9c4c0c8790 0x7f9c4c06c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 >> v1:172.21.15.119:6789/0 conn(0x7f9c4c0c8300 legacy=0x7f9c4c06bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c15b1a0 con 0x7f9c4c13d310 2023-08-11T15:52:28.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f9c48001670 con 0x7f9c4c13d310 2023-08-11T15:52:28.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 >> v1:172.21.15.27:6789/0 conn(0x7f9c4c13d310 legacy=0x7f9c4c15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c38003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c380047c0 0x7f9c38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9c380052a0 0x7f9c3800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c1407b0 con 0x7f9c380039c0 2023-08-11T15:52:28.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3800c1a0 con 0x7f9c380047c0 2023-08-11T15:52:28.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3800c360 con 0x7f9c380052a0 2023-08-11T15:52:28.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.997+0000 7f9c51f32640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c38003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.998+0000 7f9c51731640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c380047c0 0x7f9c38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.998+0000 7f9c52733640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9c380052a0 0x7f9c3800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.998+0000 7f9c51f32640 1 -- 172.21.15.27:0/976929701 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9c380052a0 msgr2=0x7f9c3800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:28.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.998+0000 7f9c51f32640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9c380052a0 0x7f9c3800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.998+0000 7f9c51f32640 1 -- 172.21.15.27:0/976929701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c380047c0 msgr2=0x7f9c38004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:28.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.998+0000 7f9c51f32640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c380047c0 0x7f9c38004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.998+0000 7f9c51f32640 1 -- 172.21.15.27:0/976929701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c38003520 con 0x7f9c380039c0 2023-08-11T15:52:28.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.998+0000 7f9c52733640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9c380052a0 0x7f9c3800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:28.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.998+0000 7f9c52733640 1 -- 172.21.15.27:0/976929701 reap_dead start 2023-08-11T15:52:28.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:27.999+0000 7f9c51f32640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c38003de0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800c0e0 tx=0x7f9c380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30b6261b4a5ad3d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:28.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.000+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48005ad0 con 0x7f9c380039c0 2023-08-11T15:52:28.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.000+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c48005c50 con 0x7f9c380039c0 2023-08-11T15:52:28.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.000+0000 7f9c50f30640 1 -- 172.21.15.27:0/976929701 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48007070 con 0x7f9c380039c0 2023-08-11T15:52:28.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.000+0000 7f9c52f34640 1 -- 172.21.15.27:0/976929701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 msgr2=0x7f9c38003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.000+0000 7f9c52f34640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c38003de0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800c0e0 tx=0x7f9c380057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.000+0000 7f9c52f34640 1 -- 172.21.15.27:0/976929701 shutdown_connections 2023-08-11T15:52:28.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.000+0000 7f9c52f34640 1 --2- 172.21.15.27:0/976929701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c38003de0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.000+0000 7f9c52f34640 1 -- 172.21.15.27:0/976929701 >> 172.21.15.27:0/976929701 conn(0x7f9c4c069800 msgr2=0x7f9c4c069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:28.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.001+0000 7f9c52f34640 1 -- 172.21.15.27:0/976929701 shutdown_connections 2023-08-11T15:52:28.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.001+0000 7f9c52f34640 1 -- 172.21.15.27:0/976929701 wait complete. 2023-08-11T15:52:28.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.003+0000 7f9c52f34640 1 Processor -- start 2023-08-11T15:52:28.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.003+0000 7f9c52f34640 1 -- start start 2023-08-11T15:52:28.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.004+0000 7f9c52f34640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c4c0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.004+0000 7f9c52f34640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c4c1f4f60 0x7f9c4c0c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.004+0000 7f9c52f34640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9c4c141740 0x7f9c4c141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.004+0000 7f9c52f34640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c13c220 con 0x7f9c380039c0 2023-08-11T15:52:28.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.004+0000 7f9c52f34640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c06f230 con 0x7f9c4c1f4f60 2023-08-11T15:52:28.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.004+0000 7f9c52f34640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c1f56b0 con 0x7f9c4c141740 2023-08-11T15:52:28.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.004+0000 7f9c51f32640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c4c0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.004+0000 7f9c51f32640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c4c0cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60598/0 (socket says 172.21.15.27:60598) 2023-08-11T15:52:28.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.004+0000 7f9c51f32640 1 -- 172.21.15.27:0/3038473020 learned_addr learned my addr 172.21.15.27:0/3038473020 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:28.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c51f32640 1 -- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9c4c141740 msgr2=0x7f9c4c141bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c52733640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9c4c141740 0x7f9c4c141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c51731640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c4c1f4f60 0x7f9c4c0c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c51f32640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9c4c141740 0x7f9c4c141bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c51f32640 1 -- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c4c1f4f60 msgr2=0x7f9c4c0c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c51f32640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c4c1f4f60 0x7f9c4c0c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c51f32640 1 -- 172.21.15.27:0/3038473020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c1f5990 con 0x7f9c380039c0 2023-08-11T15:52:28.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c51731640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c4c1f4f60 0x7f9c4c0c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:28.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c51f32640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c4c0cd290 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9c48000c00 tx=0x7f9c4802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:28.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c427fc640 1 -- 172.21.15.27:0/3038473020 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4803c040 con 0x7f9c380039c0 2023-08-11T15:52:28.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c427fc640 1 -- 172.21.15.27:0/3038473020 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c48007480 con 0x7f9c380039c0 2023-08-11T15:52:28.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c427fc640 1 -- 172.21.15.27:0/3038473020 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4802c770 con 0x7f9c380039c0 2023-08-11T15:52:28.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.005+0000 7f9c52f34640 1 -- 172.21.15.27:0/3038473020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4c1f5c70 con 0x7f9c380039c0 2023-08-11T15:52:28.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.006+0000 7f9c52f34640 1 -- 172.21.15.27:0/3038473020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4c1f53c0 con 0x7f9c380039c0 2023-08-11T15:52:28.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.007+0000 7f9c52f34640 1 -- 172.21.15.27:0/3038473020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4c066e80 con 0x7f9c380039c0 2023-08-11T15:52:28.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.008+0000 7f9c427fc640 1 -- 172.21.15.27:0/3038473020 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9c48032070 con 0x7f9c380039c0 2023-08-11T15:52:28.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.008+0000 7f9c427fc640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9c3006a080 0x7f9c3006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.009+0000 7f9c427fc640 1 -- 172.21.15.27:0/3038473020 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9c480a9d70 con 0x7f9c380039c0 2023-08-11T15:52:28.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.009+0000 7f9c51731640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9c3006a080 0x7f9c3006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.010+0000 7f9c51731640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9c3006a080 0x7f9c3006c540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c007fd0 tx=0x7f9c3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:28.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.013+0000 7f9c427fc640 1 -- 172.21.15.27:0/3038473020 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9c48075130 con 0x7f9c380039c0 2023-08-11T15:52:28.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.222+0000 7f9c52f34640 1 -- 172.21.15.27:0/3038473020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9c4c1f5e00 con 0x7f9c380039c0 2023-08-11T15:52:28.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.223+0000 7f9c427fc640 1 -- 172.21.15.27:0/3038473020 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9c48078750 con 0x7f9c380039c0 2023-08-11T15:52:28.225 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:52:28.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.225+0000 7f9c52f34640 1 -- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9c3006a080 msgr2=0x7f9c3006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.225+0000 7f9c52f34640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9c3006a080 0x7f9c3006c540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c007fd0 tx=0x7f9c3c009040 comp rx=0 tx=0).stop 2023-08-11T15:52:28.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.225+0000 7f9c52f34640 1 -- 172.21.15.27:0/3038473020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 msgr2=0x7f9c4c0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.225+0000 7f9c52f34640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c4c0cd290 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9c48000c00 tx=0x7f9c4802e070 comp rx=0 tx=0).stop 2023-08-11T15:52:28.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.225+0000 7f9c52f34640 1 -- 172.21.15.27:0/3038473020 shutdown_connections 2023-08-11T15:52:28.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.225+0000 7f9c52f34640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9c3006a080 0x7f9c3006c540 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.225+0000 7f9c52f34640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9c4c141740 0x7f9c4c141bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.225+0000 7f9c52f34640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c4c1f4f60 0x7f9c4c0c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.225+0000 7f9c52f34640 1 --2- 172.21.15.27:0/3038473020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c380039c0 0x7f9c4c0cd290 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.225+0000 7f9c52f34640 1 -- 172.21.15.27:0/3038473020 >> 172.21.15.27:0/3038473020 conn(0x7f9c4c069800 msgr2=0x7f9c4c1594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:28.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.226+0000 7f9c52f34640 1 -- 172.21.15.27:0/3038473020 shutdown_connections 2023-08-11T15:52:28.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.226+0000 7f9c52f34640 1 -- 172.21.15.27:0/3038473020 wait complete. 2023-08-11T15:52:28.246 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi119.front.sepia.ceph.com... 2023-08-11T15:52:28.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:52:28.247 DEBUG:teuthology.orchestra.run.smithi119:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-11T15:52:28.354 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:52:28.355 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:52:28.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:52:28.356 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-11T15:52:28.429 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:52:28.429 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:28.494 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:52:28.495 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:28.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:28.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.793+0000 7f86e1be8640 1 Processor -- start 2023-08-11T15:52:28.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.793+0000 7f86e1be8640 1 -- start start 2023-08-11T15:52:28.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.795+0000 7f86e1be8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86dc15d3b0 0x7f86dc1673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.795+0000 7f86e1be8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc15da30 con 0x7f86dc0cc520 2023-08-11T15:52:28.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.795+0000 7f86e1be8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc0c9790 con 0x7f86dc160d20 2023-08-11T15:52:28.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.795+0000 7f86e1be8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc15d820 con 0x7f86dc15d3b0 2023-08-11T15:52:28.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.795+0000 7f86daffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f86dc0cc520 0x7f86dc15cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47194/0 (socket says 172.21.15.27:47194) 2023-08-11T15:52:28.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.795+0000 7f86daffd640 1 -- 172.21.15.27:0/2679847289 learned_addr learned my addr 172.21.15.27:0/2679847289 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:28.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86dbfff640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86dc15d3b0 0x7f86dc1673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425147022 0 0) 0x7f86dc15da30 con 0x7f86dc0cc520 2023-08-11T15:52:28.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c4003520 con 0x7f86dc0cc520 2023-08-11T15:52:28.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4245646987 0 0) 0x7f86c4003520 con 0x7f86dc0cc520 2023-08-11T15:52:28.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86dc15d3b0 msgr2=0x7f86dc1673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:28.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86dc15d3b0 0x7f86dc1673e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 >> v1:172.21.15.119:6789/0 conn(0x7f86dc160d20 legacy=0x7f86dc0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0ca570 con 0x7f86dc0cc520 2023-08-11T15:52:28.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f86d0001690 con 0x7f86dc0cc520 2023-08-11T15:52:28.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 >> v1:172.21.15.27:6789/0 conn(0x7f86dc0cc520 legacy=0x7f86dc15cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c40039a0 0x7f86c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86dc15d3b0 0x7f86c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c4004d00 0x7f86c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc15d820 con 0x7f86c4004d00 2023-08-11T15:52:28.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c400ba90 con 0x7f86c40039a0 2023-08-11T15:52:28.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.796+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c400bc50 con 0x7f86dc15d3b0 2023-08-11T15:52:28.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.797+0000 7f86dbfff640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c4004d00 0x7f86c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.797+0000 7f86db7fe640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c40039a0 0x7f86c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.797+0000 7f86dbfff640 1 -- 172.21.15.27:0/2679847289 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86dc15d3b0 msgr2=0x7f86c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:28.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.797+0000 7f86dbfff640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86dc15d3b0 0x7f86c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.797+0000 7f86dbfff640 1 -- 172.21.15.27:0/2679847289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c40039a0 msgr2=0x7f86c4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:28.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.797+0000 7f86dbfff640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c40039a0 0x7f86c4003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.797+0000 7f86dbfff640 1 -- 172.21.15.27:0/2679847289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4003520 con 0x7f86c4004d00 2023-08-11T15:52:28.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.798+0000 7f86dbfff640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c4004d00 0x7f86c4005180 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009f90 tx=0x7f86c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e65d71586b8021 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.799+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d402cc20 con 0x7f86c4004d00 2023-08-11T15:52:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.799+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d400f040 con 0x7f86c4004d00 2023-08-11T15:52:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.799+0000 7f86da7fc640 1 -- 172.21.15.27:0/2679847289 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4034440 con 0x7f86c4004d00 2023-08-11T15:52:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.799+0000 7f86e1be8640 1 -- 172.21.15.27:0/2679847289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c4004d00 msgr2=0x7f86c4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.799+0000 7f86e1be8640 1 --2- 172.21.15.27:0/2679847289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c4004d00 0x7f86c4005180 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009f90 tx=0x7f86c40057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.799+0000 7f86dbfff640 1 -- 172.21.15.27:0/2679847289 reap_dead start 2023-08-11T15:52:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.799+0000 7f86e1be8640 1 -- 172.21.15.27:0/2679847289 shutdown_connections 2023-08-11T15:52:28.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.799+0000 7f86e1be8640 1 -- 172.21.15.27:0/2679847289 >> 172.21.15.27:0/2679847289 conn(0x7f86dc0afb70 msgr2=0x7f86dc0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:28.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.799+0000 7f86e1be8640 1 -- 172.21.15.27:0/2679847289 shutdown_connections 2023-08-11T15:52:28.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.799+0000 7f86e1be8640 1 -- 172.21.15.27:0/2679847289 wait complete. 2023-08-11T15:52:28.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.801+0000 7f86e1be8640 1 Processor -- start 2023-08-11T15:52:28.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.801+0000 7f86e1be8640 1 -- start start 2023-08-11T15:52:28.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.802+0000 7f86e1be8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86c4004d00 0x7f86dc1ef340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.802+0000 7f86e1be8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc1f5480 0x7f86dc14dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.802+0000 7f86e1be8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86dc14e400 0x7f86dc0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.802+0000 7f86e1be8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc14e880 con 0x7f86dc14e400 2023-08-11T15:52:28.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.802+0000 7f86e1be8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1f1e90 con 0x7f86dc1f5480 2023-08-11T15:52:28.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.802+0000 7f86e1be8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1ef880 con 0x7f86c4004d00 2023-08-11T15:52:28.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.803+0000 7f86dbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86dc14e400 0x7f86dc0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.803+0000 7f86dbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86dc14e400 0x7f86dc0c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60618/0 (socket says 172.21.15.27:60618) 2023-08-11T15:52:28.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.803+0000 7f86dbfff640 1 -- 172.21.15.27:0/1561479773 learned_addr learned my addr 172.21.15.27:0/1561479773 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:28.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.803+0000 7f86db7fe640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86c4004d00 0x7f86dc1ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.803+0000 7f86dbfff640 1 -- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86c4004d00 msgr2=0x7f86dc1ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.803+0000 7f86dbfff640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86c4004d00 0x7f86dc1ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.803+0000 7f86daffd640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc1f5480 0x7f86dc14dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.803+0000 7f86dbfff640 1 -- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc1f5480 msgr2=0x7f86dc14dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:28.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.803+0000 7f86dbfff640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc1f5480 0x7f86dc14dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:28.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.803+0000 7f86dbfff640 1 -- 172.21.15.27:0/1561479773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0c63f0 con 0x7f86dc14e400 2023-08-11T15:52:28.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.804+0000 7f86dbfff640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86dc14e400 0x7f86dc0c8660 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f86d4037040 tx=0x7f86d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:28.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.804+0000 7f86c3fff640 1 -- 172.21.15.27:0/1561479773 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4034cd0 con 0x7f86dc14e400 2023-08-11T15:52:28.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.804+0000 7f86c3fff640 1 -- 172.21.15.27:0/1561479773 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d4034e50 con 0x7f86dc14e400 2023-08-11T15:52:28.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.804+0000 7f86c3fff640 1 -- 172.21.15.27:0/1561479773 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4003bd0 con 0x7f86dc14e400 2023-08-11T15:52:28.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.804+0000 7f86e1be8640 1 -- 172.21.15.27:0/1561479773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc0c66d0 con 0x7f86dc14e400 2023-08-11T15:52:28.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.804+0000 7f86e1be8640 1 -- 172.21.15.27:0/1561479773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc0c6c90 con 0x7f86dc14e400 2023-08-11T15:52:28.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.805+0000 7f86e1be8640 1 -- 172.21.15.27:0/1561479773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc1f1fe0 con 0x7f86dc14e400 2023-08-11T15:52:28.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.806+0000 7f86c3fff640 1 -- 172.21.15.27:0/1561479773 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f86d4003d30 con 0x7f86dc14e400 2023-08-11T15:52:28.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.807+0000 7f86c3fff640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f86c806a050 0x7f86c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:28.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.807+0000 7f86c3fff640 1 -- 172.21.15.27:0/1561479773 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f86d40aaaf0 con 0x7f86dc14e400 2023-08-11T15:52:28.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.808+0000 7f86db7fe640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f86c806a050 0x7f86c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:28.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.809+0000 7f86db7fe640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f86c806a050 0x7f86c806c510 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f86cc001780 tx=0x7f86cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:28.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:28.812+0000 7f86c3fff640 1 -- 172.21.15.27:0/1561479773 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f86dc1f1fe0 con 0x7f86dc14e400 2023-08-11T15:52:29.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.006+0000 7f86e1be8640 1 -- 172.21.15.27:0/1561479773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86dc1f1fe0 con 0x7f86dc14e400 2023-08-11T15:52:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.008+0000 7f86c3fff640 1 -- 172.21.15.27:0/1561479773 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f86dc1f1fe0 con 0x7f86dc14e400 2023-08-11T15:52:29.011 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:29.012 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 -- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f86c806a050 msgr2=0x7f86c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f86c806a050 0x7f86c806c510 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f86cc001780 tx=0x7f86cc009040 comp rx=0 tx=0).stop 2023-08-11T15:52:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 -- 172.21.15.27:0/1561479773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86dc14e400 msgr2=0x7f86dc0c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86dc14e400 0x7f86dc0c8660 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f86d4037040 tx=0x7f86d4009750 comp rx=0 tx=0).stop 2023-08-11T15:52:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 -- 172.21.15.27:0/1561479773 shutdown_connections 2023-08-11T15:52:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f86c806a050 0x7f86c806c510 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86dc14e400 0x7f86dc0c8660 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc1f5480 0x7f86dc14dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 --2- 172.21.15.27:0/1561479773 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86c4004d00 0x7f86dc1ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 -- 172.21.15.27:0/1561479773 >> 172.21.15.27:0/1561479773 conn(0x7f86dc0afb70 msgr2=0x7f86dc071190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.011+0000 7f86e1be8640 1 -- 172.21.15.27:0/1561479773 shutdown_connections 2023-08-11T15:52:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.012+0000 7f86e1be8640 1 -- 172.21.15.27:0/1561479773 wait complete. 2023-08-11T15:52:29.035 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:29.036 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:29.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.272+0000 7fd6b96ec640 1 Processor -- start 2023-08-11T15:52:29.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.272+0000 7fd6b96ec640 1 -- start start 2023-08-11T15:52:29.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.274+0000 7fd6b96ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd6b413ce80 0x7fd6b413f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.274+0000 7fd6b96ec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6b413d260 0x7fd6b413fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.274+0000 7fd6b96ec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b40cd470 con 0x7fd6b40cc720 2023-08-11T15:52:29.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.275+0000 7fd6b96ec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b4140670 con 0x7fd6b413d260 2023-08-11T15:52:29.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.275+0000 7fd6b96ec640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b40cd750 con 0x7fd6b413ce80 2023-08-11T15:52:29.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.275+0000 7fd6b37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6b413d260 0x7fd6b413fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.275+0000 7fd6b37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6b413d260 0x7fd6b413fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44824/0 (socket says 172.21.15.27:44824) 2023-08-11T15:52:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.275+0000 7fd6b37fe640 1 -- 172.21.15.27:0/2878937915 learned_addr learned my addr 172.21.15.27:0/2878937915 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.275+0000 7fd6b8eeb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd6b40cc720 0x7fd6b40ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57850/0 (socket says 172.21.15.27:57850) 2023-08-11T15:52:29.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.275+0000 7fd6b3fff640 1 --2- 172.21.15.27:0/2878937915 >> v2:172.21.15.119:3300/0 conn(0x7fd6b413ce80 0x7fd6b413f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.275+0000 7fd6b37fe640 1 -- 172.21.15.27:0/2878937915 >> v2:172.21.15.119:3300/0 conn(0x7fd6b413ce80 msgr2=0x7fd6b413f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:29.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.275+0000 7fd6b37fe640 1 --2- 172.21.15.27:0/2878937915 >> v2:172.21.15.119:3300/0 conn(0x7fd6b413ce80 0x7fd6b413f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.275+0000 7fd6b37fe640 1 -- 172.21.15.27:0/2878937915 >> v1:172.21.15.27:6789/0 conn(0x7fd6b40cc720 legacy=0x7fd6b40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.276+0000 7fd6b37fe640 1 -- 172.21.15.27:0/2878937915 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40ad7d0 con 0x7fd6b413d260 2023-08-11T15:52:29.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.277+0000 7fd6b37fe640 1 --2- 172.21.15.27:0/2878937915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 0x7fd6b413fd50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd6b406b140 tx=0x7fd6a802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49cb806a77beff4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:29.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.277+0000 7fd6b2ffd640 1 -- 172.21.15.27:0/2878937915 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8030870 con 0x7fd6b413d260 2023-08-11T15:52:29.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.277+0000 7fd6b2ffd640 1 -- 172.21.15.27:0/2878937915 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a80309d0 con 0x7fd6b413d260 2023-08-11T15:52:29.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.277+0000 7fd6b2ffd640 1 -- 172.21.15.27:0/2878937915 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8030d60 con 0x7fd6b413d260 2023-08-11T15:52:29.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.278+0000 7fd6b96ec640 1 -- 172.21.15.27:0/2878937915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 msgr2=0x7fd6b413fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.278+0000 7fd6b96ec640 1 --2- 172.21.15.27:0/2878937915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 0x7fd6b413fd50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd6b406b140 tx=0x7fd6a802b9b0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.278+0000 7fd6b96ec640 1 -- 172.21.15.27:0/2878937915 shutdown_connections 2023-08-11T15:52:29.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.278+0000 7fd6b96ec640 1 --2- 172.21.15.27:0/2878937915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 0x7fd6b413fd50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.278+0000 7fd6b96ec640 1 --2- 172.21.15.27:0/2878937915 >> v2:172.21.15.119:3300/0 conn(0x7fd6b413ce80 0x7fd6b413f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.278+0000 7fd6b96ec640 1 -- 172.21.15.27:0/2878937915 >> 172.21.15.27:0/2878937915 conn(0x7fd6b4069800 msgr2=0x7fd6b4069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:29.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.278+0000 7fd6b96ec640 1 -- 172.21.15.27:0/2878937915 shutdown_connections 2023-08-11T15:52:29.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.278+0000 7fd6b96ec640 1 -- 172.21.15.27:0/2878937915 wait complete. 2023-08-11T15:52:29.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.280+0000 7fd6b96ec640 1 Processor -- start 2023-08-11T15:52:29.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.280+0000 7fd6b96ec640 1 -- start start 2023-08-11T15:52:29.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.281+0000 7fd6b96ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6b40cc720 0x7fd6b4070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.281+0000 7fd6b96ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b413ce80 0x7fd6b413f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.281+0000 7fd6b96ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 0x7fd6b4152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.281+0000 7fd6b96ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b413f910 con 0x7fd6b413d260 2023-08-11T15:52:29.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.281+0000 7fd6b96ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4071310 con 0x7fd6b413ce80 2023-08-11T15:52:29.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.281+0000 7fd6b96ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b406f3f0 con 0x7fd6b40cc720 2023-08-11T15:52:29.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.281+0000 7fd6b8eeb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 0x7fd6b4152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.281+0000 7fd6b8eeb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 0x7fd6b4152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44828/0 (socket says 172.21.15.27:44828) 2023-08-11T15:52:29.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.281+0000 7fd6b8eeb640 1 -- 172.21.15.27:0/3384984450 learned_addr learned my addr 172.21.15.27:0/3384984450 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:29.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b3fff640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6b40cc720 0x7fd6b4070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b37fe640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b413ce80 0x7fd6b413f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b8eeb640 1 -- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6b40cc720 msgr2=0x7fd6b4070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b8eeb640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6b40cc720 0x7fd6b4070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b8eeb640 1 -- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b413ce80 msgr2=0x7fd6b413f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b8eeb640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b413ce80 0x7fd6b413f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b8eeb640 1 -- 172.21.15.27:0/3384984450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a80094d0 con 0x7fd6b413d260 2023-08-11T15:52:29.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b3fff640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6b40cc720 0x7fd6b4070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:29.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b8eeb640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 0x7fd6b4152ed0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00cbc0 tx=0x7fd6ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:29.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b0ff9640 1 -- 172.21.15.27:0/3384984450 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac004db0 con 0x7fd6b413d260 2023-08-11T15:52:29.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b0ff9640 1 -- 172.21.15.27:0/3384984450 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ac00ab40 con 0x7fd6b413d260 2023-08-11T15:52:29.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b96ec640 1 -- 172.21.15.27:0/3384984450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b406f6d0 con 0x7fd6b413d260 2023-08-11T15:52:29.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.282+0000 7fd6b0ff9640 1 -- 172.21.15.27:0/3384984450 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac007070 con 0x7fd6b413d260 2023-08-11T15:52:29.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.283+0000 7fd6b96ec640 1 -- 172.21.15.27:0/3384984450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b406fc90 con 0x7fd6b413d260 2023-08-11T15:52:29.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.284+0000 7fd6b96ec640 1 -- 172.21.15.27:0/3384984450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b406fe80 con 0x7fd6b413d260 2023-08-11T15:52:29.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.285+0000 7fd6b0ff9640 1 -- 172.21.15.27:0/3384984450 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd6ac0071d0 con 0x7fd6b413d260 2023-08-11T15:52:29.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.286+0000 7fd6b0ff9640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6a006a050 0x7fd6a006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.286+0000 7fd6b0ff9640 1 -- 172.21.15.27:0/3384984450 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd6ac0880f0 con 0x7fd6b413d260 2023-08-11T15:52:29.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.286+0000 7fd6b3fff640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6a006a050 0x7fd6a006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.288+0000 7fd6b3fff640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6a006a050 0x7fd6a006c510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4006fd0 tx=0x7fd6a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:29.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.290+0000 7fd6b0ff9640 1 -- 172.21.15.27:0/3384984450 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fd6ac053540 con 0x7fd6b413d260 2023-08-11T15:52:29.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.483+0000 7fd6b96ec640 1 -- 172.21.15.27:0/3384984450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6b4066e80 con 0x7fd6b413d260 2023-08-11T15:52:29.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.485+0000 7fd6b0ff9640 1 -- 172.21.15.27:0/3384984450 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6ac0180a0 con 0x7fd6b413d260 2023-08-11T15:52:29.487 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:29.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.487+0000 7fd6b96ec640 1 -- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6a006a050 msgr2=0x7fd6a006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.487+0000 7fd6b96ec640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6a006a050 0x7fd6a006c510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4006fd0 tx=0x7fd6a4008040 comp rx=0 tx=0).stop 2023-08-11T15:52:29.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.487+0000 7fd6b96ec640 1 -- 172.21.15.27:0/3384984450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 msgr2=0x7fd6b4152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.487+0000 7fd6b96ec640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 0x7fd6b4152ed0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00cbc0 tx=0x7fd6ac00a430 comp rx=0 tx=0).stop 2023-08-11T15:52:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.487+0000 7fd6b96ec640 1 -- 172.21.15.27:0/3384984450 shutdown_connections 2023-08-11T15:52:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.487+0000 7fd6b96ec640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6a006a050 0x7fd6a006c510 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.487+0000 7fd6b96ec640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6b413d260 0x7fd6b4152ed0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.487+0000 7fd6b96ec640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b413ce80 0x7fd6b413f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.487+0000 7fd6b96ec640 1 --2- 172.21.15.27:0/3384984450 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6b40cc720 0x7fd6b4070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.487+0000 7fd6b96ec640 1 -- 172.21.15.27:0/3384984450 >> 172.21.15.27:0/3384984450 conn(0x7fd6b4069800 msgr2=0x7fd6b406c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:29.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.488+0000 7fd6b96ec640 1 -- 172.21.15.27:0/3384984450 shutdown_connections 2023-08-11T15:52:29.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.488+0000 7fd6b96ec640 1 -- 172.21.15.27:0/3384984450 wait complete. 2023-08-11T15:52:29.511 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:29.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.744+0000 7fb524d0f640 1 Processor -- start 2023-08-11T15:52:29.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.745+0000 7fb524d0f640 1 -- start start 2023-08-11T15:52:29.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.746+0000 7fb524d0f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52015c4c0 0x7fb5200ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.747+0000 7fb524d0f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb52006a910 0x7fb5200ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.747+0000 7fb524d0f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52006b830 con 0x7fb5200cb4d0 2023-08-11T15:52:29.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.747+0000 7fb524d0f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52006ba80 con 0x7fb52006a910 2023-08-11T15:52:29.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.747+0000 7fb524d0f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200ca9f0 con 0x7fb52015c4c0 2023-08-11T15:52:29.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.747+0000 7fb515d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb52006a910 0x7fb5200ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.748+0000 7fb515d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb52006a910 0x7fb5200ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38708/0 (socket says 172.21.15.27:38708) 2023-08-11T15:52:29.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.748+0000 7fb515d9b640 1 -- 172.21.15.27:0/1970523792 learned_addr learned my addr 172.21.15.27:0/1970523792 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:29.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.748+0000 7fb51e59c640 1 --2- 172.21.15.27:0/1970523792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52015c4c0 0x7fb5200ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.749+0000 7fb51e59c640 1 -- 172.21.15.27:0/1970523792 >> v2:172.21.15.119:3300/0 conn(0x7fb52006a910 msgr2=0x7fb5200ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:29.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.749+0000 7fb51e59c640 1 --2- 172.21.15.27:0/1970523792 >> v2:172.21.15.119:3300/0 conn(0x7fb52006a910 0x7fb5200ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.749+0000 7fb51e59c640 1 -- 172.21.15.27:0/1970523792 >> v1:172.21.15.119:6789/0 conn(0x7fb5200cb4d0 legacy=0x7fb5200cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.749+0000 7fb51e59c640 1 -- 172.21.15.27:0/1970523792 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5200cacd0 con 0x7fb52015c4c0 2023-08-11T15:52:29.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.751+0000 7fb51e59c640 1 --2- 172.21.15.27:0/1970523792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52015c4c0 0x7fb5200ccaa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb510009d90 tx=0x7fb51002be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9b44f99d2dd4a22 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:29.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.751+0000 7fb51dd9b640 1 -- 172.21.15.27:0/1970523792 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51002cbd0 con 0x7fb52015c4c0 2023-08-11T15:52:29.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.751+0000 7fb51dd9b640 1 -- 172.21.15.27:0/1970523792 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51002cd30 con 0x7fb52015c4c0 2023-08-11T15:52:29.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.751+0000 7fb51dd9b640 1 -- 172.21.15.27:0/1970523792 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51002cea0 con 0x7fb52015c4c0 2023-08-11T15:52:29.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.752+0000 7fb524d0f640 1 -- 172.21.15.27:0/1970523792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52015c4c0 msgr2=0x7fb5200ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.752+0000 7fb524d0f640 1 --2- 172.21.15.27:0/1970523792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52015c4c0 0x7fb5200ccaa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb510009d90 tx=0x7fb51002be20 comp rx=0 tx=0).stop 2023-08-11T15:52:29.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.752+0000 7fb524d0f640 1 -- 172.21.15.27:0/1970523792 shutdown_connections 2023-08-11T15:52:29.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.752+0000 7fb524d0f640 1 --2- 172.21.15.27:0/1970523792 >> v2:172.21.15.119:3300/0 conn(0x7fb52006a910 0x7fb5200ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.752+0000 7fb524d0f640 1 --2- 172.21.15.27:0/1970523792 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52015c4c0 0x7fb5200ccaa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.752+0000 7fb524d0f640 1 -- 172.21.15.27:0/1970523792 >> 172.21.15.27:0/1970523792 conn(0x7fb5200d0220 msgr2=0x7fb5200d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:29.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.752+0000 7fb524d0f640 1 -- 172.21.15.27:0/1970523792 shutdown_connections 2023-08-11T15:52:29.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.752+0000 7fb524d0f640 1 -- 172.21.15.27:0/1970523792 wait complete. 2023-08-11T15:52:29.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.754+0000 7fb524d0f640 1 Processor -- start 2023-08-11T15:52:29.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.754+0000 7fb524d0f640 1 -- start start 2023-08-11T15:52:29.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.755+0000 7fb524d0f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52006a910 0x7fb52006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.756+0000 7fb524d0f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5200cb4d0 0x7fb5200b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.756+0000 7fb524d0f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb52015c4c0 0x7fb5200c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.756+0000 7fb524d0f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200c6710 con 0x7fb5200cb4d0 2023-08-11T15:52:29.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.756+0000 7fb524d0f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200c79e0 con 0x7fb52015c4c0 2023-08-11T15:52:29.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.756+0000 7fb524d0f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200c9a10 con 0x7fb52006a910 2023-08-11T15:52:29.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.756+0000 7fb515d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5200cb4d0 0x7fb5200b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.756+0000 7fb515d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5200cb4d0 0x7fb5200b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44844/0 (socket says 172.21.15.27:44844) 2023-08-11T15:52:29.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.756+0000 7fb515d9b640 1 -- 172.21.15.27:0/3034520692 learned_addr learned my addr 172.21.15.27:0/3034520692 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:29.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.756+0000 7fb51e59c640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52006a910 0x7fb52006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.756+0000 7fb51ed9d640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb52015c4c0 0x7fb5200c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.757+0000 7fb515d9b640 1 -- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52006a910 msgr2=0x7fb52006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.757+0000 7fb515d9b640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52006a910 0x7fb52006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.757+0000 7fb515d9b640 1 -- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb52015c4c0 msgr2=0x7fb5200c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.757+0000 7fb515d9b640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb52015c4c0 0x7fb5200c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.757+0000 7fb515d9b640 1 -- 172.21.15.27:0/3034520692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb510009a70 con 0x7fb5200cb4d0 2023-08-11T15:52:29.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.757+0000 7fb515d9b640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5200cb4d0 0x7fb5200b4e90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb508009fd0 tx=0x7fb50800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:29.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.757+0000 7fb5177fe640 1 -- 172.21.15.27:0/3034520692 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800c940 con 0x7fb5200cb4d0 2023-08-11T15:52:29.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.757+0000 7fb5177fe640 1 -- 172.21.15.27:0/3034520692 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50800caa0 con 0x7fb5200cb4d0 2023-08-11T15:52:29.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.757+0000 7fb5177fe640 1 -- 172.21.15.27:0/3034520692 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800ce30 con 0x7fb5200cb4d0 2023-08-11T15:52:29.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.757+0000 7fb524d0f640 1 -- 172.21.15.27:0/3034520692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52015ca20 con 0x7fb5200cb4d0 2023-08-11T15:52:29.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.758+0000 7fb524d0f640 1 -- 172.21.15.27:0/3034520692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5200c7670 con 0x7fb5200cb4d0 2023-08-11T15:52:29.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.759+0000 7fb524d0f640 1 -- 172.21.15.27:0/3034520692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5200c7b90 con 0x7fb5200cb4d0 2023-08-11T15:52:29.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.760+0000 7fb5177fe640 1 -- 172.21.15.27:0/3034520692 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb508013470 con 0x7fb5200cb4d0 2023-08-11T15:52:29.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.761+0000 7fb5177fe640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb50006a050 0x7fb50006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:29.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.761+0000 7fb5177fe640 1 -- 172.21.15.27:0/3034520692 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb508088e40 con 0x7fb5200cb4d0 2023-08-11T15:52:29.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.761+0000 7fb51e59c640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb50006a050 0x7fb50006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:29.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.763+0000 7fb51e59c640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb50006a050 0x7fb50006c510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb510036000 tx=0x7fb510030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:29.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.765+0000 7fb5177fe640 1 -- 172.21.15.27:0/3034520692 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fb5200c7b90 con 0x7fb5200cb4d0 2023-08-11T15:52:29.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.959+0000 7fb524d0f640 1 -- 172.21.15.27:0/3034520692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb5200c7b50 con 0x7fb5200cb4d0 2023-08-11T15:52:29.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.960+0000 7fb5177fe640 1 -- 172.21.15.27:0/3034520692 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb508054160 con 0x7fb5200cb4d0 2023-08-11T15:52:29.963 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:29.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 -- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb50006a050 msgr2=0x7fb50006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb50006a050 0x7fb50006c510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb510036000 tx=0x7fb510030000 comp rx=0 tx=0).stop 2023-08-11T15:52:29.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 -- 172.21.15.27:0/3034520692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5200cb4d0 msgr2=0x7fb5200b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:29.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5200cb4d0 0x7fb5200b4e90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb508009fd0 tx=0x7fb50800ed40 comp rx=0 tx=0).stop 2023-08-11T15:52:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 -- 172.21.15.27:0/3034520692 shutdown_connections 2023-08-11T15:52:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb50006a050 0x7fb50006c510 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb52015c4c0 0x7fb5200c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5200cb4d0 0x7fb5200b4e90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 --2- 172.21.15.27:0/3034520692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb52006a910 0x7fb52006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 -- 172.21.15.27:0/3034520692 >> 172.21.15.27:0/3034520692 conn(0x7fb5200d0220 msgr2=0x7fb5200acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 -- 172.21.15.27:0/3034520692 shutdown_connections 2023-08-11T15:52:29.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:29.963+0000 7fb524d0f640 1 -- 172.21.15.27:0/3034520692 wait complete. 2023-08-11T15:52:29.984 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.224+0000 7efe789e8640 1 Processor -- start 2023-08-11T15:52:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.225+0000 7efe789e8640 1 -- start start 2023-08-11T15:52:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.226+0000 7efe789e8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 0x7efe740caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.226+0000 7efe789e8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe740cc340 0x7efe7415cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.226+0000 7efe789e8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe741600d0 con 0x7efe740caf40 2023-08-11T15:52:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.226+0000 7efe789e8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe740cb380 con 0x7efe740cc340 2023-08-11T15:52:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.226+0000 7efe789e8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe74160320 con 0x7efe74159f80 2023-08-11T15:52:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.226+0000 7efe72d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe740cc340 0x7efe7415cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.227+0000 7efe72d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe740cc340 0x7efe7415cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44860/0 (socket says 172.21.15.27:44860) 2023-08-11T15:52:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.227+0000 7efe72d9d640 1 -- 172.21.15.27:0/2911006673 learned_addr learned my addr 172.21.15.27:0/2911006673 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.227+0000 7efe6bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efe740caf40 0x7efe740cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57854/0 (socket says 172.21.15.27:57854) 2023-08-11T15:52:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.227+0000 7efe7259c640 1 --2- 172.21.15.27:0/2911006673 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 0x7efe740caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.227+0000 7efe72d9d640 1 -- 172.21.15.27:0/2911006673 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 msgr2=0x7efe740caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:30.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.227+0000 7efe72d9d640 1 --2- 172.21.15.27:0/2911006673 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 0x7efe740caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.227+0000 7efe72d9d640 1 -- 172.21.15.27:0/2911006673 >> v1:172.21.15.27:6789/0 conn(0x7efe740caf40 legacy=0x7efe740cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.227+0000 7efe72d9d640 1 -- 172.21.15.27:0/2911006673 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe74160600 con 0x7efe740cc340 2023-08-11T15:52:30.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.227+0000 7efe7259c640 1 --2- 172.21.15.27:0/2911006673 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 0x7efe740caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:30.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.228+0000 7efe72d9d640 1 --2- 172.21.15.27:0/2911006673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 0x7efe7415cae0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009e50 tx=0x7efe6c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd77c1708c9d8d34 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:30.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.229+0000 7efe71d9b640 1 -- 172.21.15.27:0/2911006673 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c02cb00 con 0x7efe740cc340 2023-08-11T15:52:30.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.229+0000 7efe71d9b640 1 -- 172.21.15.27:0/2911006673 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c02cc60 con 0x7efe740cc340 2023-08-11T15:52:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.229+0000 7efe71d9b640 1 -- 172.21.15.27:0/2911006673 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0343f0 con 0x7efe740cc340 2023-08-11T15:52:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.229+0000 7efe789e8640 1 -- 172.21.15.27:0/2911006673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 msgr2=0x7efe7415cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.229+0000 7efe789e8640 1 --2- 172.21.15.27:0/2911006673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 0x7efe7415cae0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009e50 tx=0x7efe6c02bf60 comp rx=0 tx=0).stop 2023-08-11T15:52:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.229+0000 7efe789e8640 1 -- 172.21.15.27:0/2911006673 shutdown_connections 2023-08-11T15:52:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.229+0000 7efe789e8640 1 --2- 172.21.15.27:0/2911006673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 0x7efe7415cae0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.229+0000 7efe789e8640 1 --2- 172.21.15.27:0/2911006673 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 0x7efe740caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.230+0000 7efe789e8640 1 -- 172.21.15.27:0/2911006673 >> 172.21.15.27:0/2911006673 conn(0x7efe740afbe0 msgr2=0x7efe740affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.230+0000 7efe789e8640 1 -- 172.21.15.27:0/2911006673 shutdown_connections 2023-08-11T15:52:30.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.230+0000 7efe789e8640 1 -- 172.21.15.27:0/2911006673 wait complete. 2023-08-11T15:52:30.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.232+0000 7efe789e8640 1 Processor -- start 2023-08-11T15:52:30.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.233+0000 7efe789e8640 1 -- start start 2023-08-11T15:52:30.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.234+0000 7efe789e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe740caf40 0x7efe740c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.234+0000 7efe789e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 0x7efe741528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.234+0000 7efe789e8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 0x7efe740cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.234+0000 7efe789e8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe74150280 con 0x7efe740cc340 2023-08-11T15:52:30.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.234+0000 7efe789e8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe74150080 con 0x7efe740caf40 2023-08-11T15:52:30.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.234+0000 7efe789e8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe740c40d0 con 0x7efe74159f80 2023-08-11T15:52:30.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.234+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 0x7efe741528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.234+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 0x7efe741528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44876/0 (socket says 172.21.15.27:44876) 2023-08-11T15:52:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.234+0000 7efe6bfff640 1 -- 172.21.15.27:0/3715097103 learned_addr learned my addr 172.21.15.27:0/3715097103 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:30.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.235+0000 7efe7259c640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe740caf40 0x7efe740c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.235+0000 7efe72d9d640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 0x7efe740cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.235+0000 7efe6bfff640 1 -- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 msgr2=0x7efe740cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.235+0000 7efe6bfff640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 0x7efe740cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.235+0000 7efe6bfff640 1 -- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe740caf40 msgr2=0x7efe740c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.235+0000 7efe6bfff640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe740caf40 0x7efe740c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.235+0000 7efe6bfff640 1 -- 172.21.15.27:0/3715097103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c009b30 con 0x7efe740cc340 2023-08-11T15:52:30.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.235+0000 7efe72d9d640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 0x7efe740cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:30.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.235+0000 7efe6bfff640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 0x7efe741528e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00c4d0 tx=0x7efe5c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:30.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.236+0000 7efe6affd640 1 -- 172.21.15.27:0/3715097103 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c007480 con 0x7efe740cc340 2023-08-11T15:52:30.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.236+0000 7efe6affd640 1 -- 172.21.15.27:0/3715097103 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c0075e0 con 0x7efe740cc340 2023-08-11T15:52:30.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.236+0000 7efe6affd640 1 -- 172.21.15.27:0/3715097103 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0049e0 con 0x7efe740cc340 2023-08-11T15:52:30.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.236+0000 7efe789e8640 1 -- 172.21.15.27:0/3715097103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe741532c0 con 0x7efe740cc340 2023-08-11T15:52:30.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.236+0000 7efe789e8640 1 -- 172.21.15.27:0/3715097103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe74153880 con 0x7efe740cc340 2023-08-11T15:52:30.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.237+0000 7efe789e8640 1 -- 172.21.15.27:0/3715097103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe741503c0 con 0x7efe740cc340 2023-08-11T15:52:30.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.238+0000 7efe6affd640 1 -- 172.21.15.27:0/3715097103 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efe5c004b40 con 0x7efe740cc340 2023-08-11T15:52:30.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.239+0000 7efe6affd640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe5406a080 0x7efe5406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.239+0000 7efe6affd640 1 -- 172.21.15.27:0/3715097103 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efe5c087a40 con 0x7efe740cc340 2023-08-11T15:52:30.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.240+0000 7efe7259c640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe5406a080 0x7efe5406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.241+0000 7efe7259c640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe5406a080 0x7efe5406c540 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe6400a950 tx=0x7efe64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:30.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.244+0000 7efe6affd640 1 -- 172.21.15.27:0/3715097103 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7efe741503c0 con 0x7efe740cc340 2023-08-11T15:52:30.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.436+0000 7efe789e8640 1 -- 172.21.15.27:0/3715097103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe741503c0 con 0x7efe740cc340 2023-08-11T15:52:30.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.437+0000 7efe6affd640 1 -- 172.21.15.27:0/3715097103 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe741503c0 con 0x7efe740cc340 2023-08-11T15:52:30.440 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:30.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 -- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe5406a080 msgr2=0x7efe5406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe5406a080 0x7efe5406c540 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe6400a950 tx=0x7efe64008040 comp rx=0 tx=0).stop 2023-08-11T15:52:30.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 -- 172.21.15.27:0/3715097103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 msgr2=0x7efe741528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 0x7efe741528e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00c4d0 tx=0x7efe5c00c990 comp rx=0 tx=0).stop 2023-08-11T15:52:30.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 -- 172.21.15.27:0/3715097103 shutdown_connections 2023-08-11T15:52:30.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe5406a080 0x7efe5406c540 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe74159f80 0x7efe740cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe740cc340 0x7efe741528e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 --2- 172.21.15.27:0/3715097103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe740caf40 0x7efe740c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 -- 172.21.15.27:0/3715097103 >> 172.21.15.27:0/3715097103 conn(0x7efe740afbe0 msgr2=0x7efe740ada50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:30.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.440+0000 7efe789e8640 1 -- 172.21.15.27:0/3715097103 shutdown_connections 2023-08-11T15:52:30.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.441+0000 7efe789e8640 1 -- 172.21.15.27:0/3715097103 wait complete. 2023-08-11T15:52:30.463 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:30.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.701+0000 7f22e4ab3640 1 Processor -- start 2023-08-11T15:52:30.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.701+0000 7f22e4ab3640 1 -- start start 2023-08-11T15:52:30.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.702+0000 7f22e4ab3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 0x7f22e00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.702+0000 7f22e4ab3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22e00cada0 0x7f22e015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.702+0000 7f22e4ab3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22e00cf1f0 0x7f22e00cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.702+0000 7f22e4ab3640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e015a8b0 con 0x7f22e00cada0 2023-08-11T15:52:30.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.702+0000 7f22e4ab3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e00b2630 con 0x7f22e00cf1f0 2023-08-11T15:52:30.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.702+0000 7f22e4ab3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00cfb20 con 0x7f22e00b1d10 2023-08-11T15:52:30.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22d5d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22e00cada0 0x7f22e015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22d5d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22e00cada0 0x7f22e015b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44884/0 (socket says 172.21.15.27:44884) 2023-08-11T15:52:30.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22d5d9b640 1 -- 172.21.15.27:0/1103759702 learned_addr learned my addr 172.21.15.27:0/1103759702 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:30.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22ded9d640 1 --2- 172.21.15.27:0/1103759702 >> v2:172.21.15.119:3300/0 conn(0x7f22e00cf1f0 0x7f22e00cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22d5d9b640 1 -- 172.21.15.27:0/1103759702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 msgr2=0x7f22e00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:30.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22de59c640 1 --2- 172.21.15.27:0/1103759702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 0x7f22e00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22d5d9b640 1 --2- 172.21.15.27:0/1103759702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 0x7f22e00b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22d5d9b640 1 -- 172.21.15.27:0/1103759702 >> v2:172.21.15.119:3300/0 conn(0x7f22e00cf1f0 msgr2=0x7f22e00cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:30.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22d5d9b640 1 --2- 172.21.15.27:0/1103759702 >> v2:172.21.15.119:3300/0 conn(0x7f22e00cf1f0 0x7f22e00cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22d5d9b640 1 -- 172.21.15.27:0/1103759702 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00cfe00 con 0x7f22e00cada0 2023-08-11T15:52:30.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22ded9d640 1 --2- 172.21.15.27:0/1103759702 >> v2:172.21.15.119:3300/0 conn(0x7f22e00cf1f0 0x7f22e00cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:30.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.703+0000 7f22de59c640 1 --2- 172.21.15.27:0/1103759702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 0x7f22e00b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.704+0000 7f22d5d9b640 1 --2- 172.21.15.27:0/1103759702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 0x7f22e015b2c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f22e00cddf0 tx=0x7f22c802b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1296660e55dfdf4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.705+0000 7f22ddd9b640 1 -- 172.21.15.27:0/1103759702 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c80317f0 con 0x7f22e00cada0 2023-08-11T15:52:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.705+0000 7f22ddd9b640 1 -- 172.21.15.27:0/1103759702 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8031950 con 0x7f22e00cada0 2023-08-11T15:52:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.705+0000 7f22ddd9b640 1 -- 172.21.15.27:0/1103759702 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8031ce0 con 0x7f22e00cada0 2023-08-11T15:52:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.705+0000 7f22e4ab3640 1 -- 172.21.15.27:0/1103759702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 msgr2=0x7f22e015b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.705+0000 7f22e4ab3640 1 --2- 172.21.15.27:0/1103759702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 0x7f22e015b2c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f22e00cddf0 tx=0x7f22c802b890 comp rx=0 tx=0).stop 2023-08-11T15:52:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.706+0000 7f22e4ab3640 1 -- 172.21.15.27:0/1103759702 shutdown_connections 2023-08-11T15:52:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.706+0000 7f22e4ab3640 1 --2- 172.21.15.27:0/1103759702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 0x7f22e015b2c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.706+0000 7f22e4ab3640 1 --2- 172.21.15.27:0/1103759702 >> v2:172.21.15.119:3300/0 conn(0x7f22e00cf1f0 0x7f22e00cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.706+0000 7f22e4ab3640 1 --2- 172.21.15.27:0/1103759702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 0x7f22e00b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.706+0000 7f22e4ab3640 1 -- 172.21.15.27:0/1103759702 >> 172.21.15.27:0/1103759702 conn(0x7f22e00ccd10 msgr2=0x7f22e00cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:30.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.706+0000 7f22e4ab3640 1 -- 172.21.15.27:0/1103759702 shutdown_connections 2023-08-11T15:52:30.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.706+0000 7f22e4ab3640 1 -- 172.21.15.27:0/1103759702 wait complete. 2023-08-11T15:52:30.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.707+0000 7f22e4ab3640 1 Processor -- start 2023-08-11T15:52:30.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.708+0000 7f22e4ab3640 1 -- start start 2023-08-11T15:52:30.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22e4ab3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 0x7f22e01f0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22e4ab3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 0x7f22e00c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22e4ab3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e00cf1f0 0x7f22e00c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22e4ab3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e0154f70 con 0x7f22e00cada0 2023-08-11T15:52:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22e4ab3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e0155af0 con 0x7f22e00cf1f0 2023-08-11T15:52:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22e4ab3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e01551c0 con 0x7f22e00b1d10 2023-08-11T15:52:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22d5d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 0x7f22e00c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22d5d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 0x7f22e00c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44896/0 (socket says 172.21.15.27:44896) 2023-08-11T15:52:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22d5d9b640 1 -- 172.21.15.27:0/3601607868 learned_addr learned my addr 172.21.15.27:0/3601607868 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22de59c640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 0x7f22e01f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22ded9d640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e00cf1f0 0x7f22e00c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22d5d9b640 1 -- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 msgr2=0x7f22e01f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22d5d9b640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 0x7f22e01f0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22d5d9b640 1 -- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e00cf1f0 msgr2=0x7f22e00c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.709+0000 7f22d5d9b640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e00cf1f0 0x7f22e00c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.710+0000 7f22d5d9b640 1 -- 172.21.15.27:0/3601607868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c80094d0 con 0x7f22e00cada0 2023-08-11T15:52:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.710+0000 7f22ded9d640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e00cf1f0 0x7f22e00c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.710+0000 7f22de59c640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 0x7f22e01f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.710+0000 7f22d5d9b640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 0x7f22e00c7d70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000c00 tx=0x7f22c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.710+0000 7f22d77fe640 1 -- 172.21.15.27:0/3601607868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c80329d0 con 0x7f22e00cada0 2023-08-11T15:52:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.710+0000 7f22d77fe640 1 -- 172.21.15.27:0/3601607868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8032b50 con 0x7f22e00cada0 2023-08-11T15:52:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.710+0000 7f22d77fe640 1 -- 172.21.15.27:0/3601607868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c80453f0 con 0x7f22e00cada0 2023-08-11T15:52:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.710+0000 7f22e4ab3640 1 -- 172.21.15.27:0/3601607868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e01554a0 con 0x7f22e00cada0 2023-08-11T15:52:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.710+0000 7f22e4ab3640 1 -- 172.21.15.27:0/3601607868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e0156d90 con 0x7f22e00cada0 2023-08-11T15:52:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.712+0000 7f22e4ab3640 1 -- 172.21.15.27:0/3601607868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e0155c80 con 0x7f22e00cada0 2023-08-11T15:52:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.713+0000 7f22d77fe640 1 -- 172.21.15.27:0/3601607868 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f22c8045550 con 0x7f22e00cada0 2023-08-11T15:52:30.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.714+0000 7f22d77fe640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f22c006a050 0x7f22c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:30.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.714+0000 7f22d77fe640 1 -- 172.21.15.27:0/3601607868 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f22c80aae30 con 0x7f22e00cada0 2023-08-11T15:52:30.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.715+0000 7f22de59c640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f22c006a050 0x7f22c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:30.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.716+0000 7f22de59c640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f22c006a050 0x7f22c006c510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f22d0006fd0 tx=0x7f22d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:30.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.718+0000 7f22d77fe640 1 -- 172.21.15.27:0/3601607868 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f22c80761e0 con 0x7f22e00cada0 2023-08-11T15:52:30.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.912+0000 7f22e4ab3640 1 -- 172.21.15.27:0/3601607868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f22e004e420 con 0x7f22e00cada0 2023-08-11T15:52:30.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.913+0000 7f22d77fe640 1 -- 172.21.15.27:0/3601607868 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f22c8079800 con 0x7f22e00cada0 2023-08-11T15:52:30.916 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 -- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f22c006a050 msgr2=0x7f22c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f22c006a050 0x7f22c006c510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f22d0006fd0 tx=0x7f22d0008040 comp rx=0 tx=0).stop 2023-08-11T15:52:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 -- 172.21.15.27:0/3601607868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 msgr2=0x7f22e00c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 0x7f22e00c7d70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000c00 tx=0x7f22c80046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 -- 172.21.15.27:0/3601607868 shutdown_connections 2023-08-11T15:52:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f22c006a050 0x7f22c006c510 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e00cf1f0 0x7f22e00c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00cada0 0x7f22e00c7d70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 --2- 172.21.15.27:0/3601607868 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22e00b1d10 0x7f22e01f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:30.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 -- 172.21.15.27:0/3601607868 >> 172.21.15.27:0/3601607868 conn(0x7f22e00ccd10 msgr2=0x7f22e0072420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:30.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 -- 172.21.15.27:0/3601607868 shutdown_connections 2023-08-11T15:52:30.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:30.916+0000 7f22e4ab3640 1 -- 172.21.15.27:0/3601607868 wait complete. 2023-08-11T15:52:30.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:31.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.170+0000 7f099dc14640 1 Processor -- start 2023-08-11T15:52:31.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.170+0000 7f099dc14640 1 -- start start 2023-08-11T15:52:31.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.172+0000 7f099dc14640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0998151600 0x7f099815df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.172+0000 7f099dc14640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f099815e4b0 0x7f09980ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.172+0000 7f099dc14640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099806c850 con 0x7f0998151220 2023-08-11T15:52:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.172+0000 7f099dc14640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099815e940 con 0x7f0998151600 2023-08-11T15:52:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.172+0000 7f099dc14640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099806bfc0 con 0x7f099815e4b0 2023-08-11T15:52:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.172+0000 7f0996ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0998151600 0x7f099815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.172+0000 7f09977fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0998151220 0x7f099806b8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57860/0 (socket says 172.21.15.27:57860) 2023-08-11T15:52:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.172+0000 7f0996ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0998151600 0x7f099815df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44910/0 (socket says 172.21.15.27:44910) 2023-08-11T15:52:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.172+0000 7f09977fe640 1 -- 172.21.15.27:0/768625467 learned_addr learned my addr 172.21.15.27:0/768625467 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:31.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.173+0000 7f0996ffd640 1 -- 172.21.15.27:0/768625467 >> v2:172.21.15.119:3300/0 conn(0x7f099815e4b0 msgr2=0x7f09980ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.173+0000 7f0996ffd640 1 --2- 172.21.15.27:0/768625467 >> v2:172.21.15.119:3300/0 conn(0x7f099815e4b0 0x7f09980ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.173+0000 7f0996ffd640 1 -- 172.21.15.27:0/768625467 >> v1:172.21.15.27:6789/0 conn(0x7f0998151220 legacy=0x7f099806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.173+0000 7f09967fc640 1 -- 172.21.15.27:0/768625467 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294688935 0 0) 0x7f099806c850 con 0x7f0998151220 2023-08-11T15:52:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.173+0000 7f0996ffd640 1 -- 172.21.15.27:0/768625467 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099806c290 con 0x7f0998151600 2023-08-11T15:52:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.174+0000 7f0996ffd640 1 --2- 172.21.15.27:0/768625467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 0x7f099815df70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f099815b990 tx=0x7f098c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2b74b1ed48869a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.174+0000 7f09967fc640 1 -- 172.21.15.27:0/768625467 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c031710 con 0x7f0998151600 2023-08-11T15:52:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.174+0000 7f09967fc640 1 -- 172.21.15.27:0/768625467 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c02f050 con 0x7f0998151600 2023-08-11T15:52:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.174+0000 7f09967fc640 1 -- 172.21.15.27:0/768625467 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c031a90 con 0x7f0998151600 2023-08-11T15:52:31.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.175+0000 7f099dc14640 1 -- 172.21.15.27:0/768625467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 msgr2=0x7f099815df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.175+0000 7f099dc14640 1 --2- 172.21.15.27:0/768625467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 0x7f099815df70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f099815b990 tx=0x7f098c02b7b0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.175+0000 7f099dc14640 1 -- 172.21.15.27:0/768625467 shutdown_connections 2023-08-11T15:52:31.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.175+0000 7f099dc14640 1 --2- 172.21.15.27:0/768625467 >> v2:172.21.15.119:3300/0 conn(0x7f099815e4b0 0x7f09980ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.175+0000 7f099dc14640 1 --2- 172.21.15.27:0/768625467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 0x7f099815df70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.175+0000 7f099dc14640 1 -- 172.21.15.27:0/768625467 >> 172.21.15.27:0/768625467 conn(0x7f09980afbe0 msgr2=0x7f09980affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:31.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.175+0000 7f099dc14640 1 -- 172.21.15.27:0/768625467 shutdown_connections 2023-08-11T15:52:31.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.175+0000 7f099dc14640 1 -- 172.21.15.27:0/768625467 wait complete. 2023-08-11T15:52:31.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.177+0000 7f099dc14640 1 Processor -- start 2023-08-11T15:52:31.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.177+0000 7f099dc14640 1 -- start start 2023-08-11T15:52:31.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.178+0000 7f099dc14640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0998151220 0x7f099814f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.178+0000 7f099dc14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 0x7f0998154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.178+0000 7f099dc14640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f099815e4b0 0x7f09980ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.178+0000 7f099dc14640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0998156b00 con 0x7f0998151600 2023-08-11T15:52:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.178+0000 7f099dc14640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099814fc90 con 0x7f0998151220 2023-08-11T15:52:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.178+0000 7f099dc14640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0998156d50 con 0x7f099815e4b0 2023-08-11T15:52:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.179+0000 7f0996ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 0x7f0998154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.179+0000 7f0996ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 0x7f0998154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44918/0 (socket says 172.21.15.27:44918) 2023-08-11T15:52:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.179+0000 7f0996ffd640 1 -- 172.21.15.27:0/2948552858 learned_addr learned my addr 172.21.15.27:0/2948552858 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.179+0000 7f09977fe640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0998151220 0x7f099814f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.179+0000 7f0997fff640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f099815e4b0 0x7f09980ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.179+0000 7f0996ffd640 1 -- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f099815e4b0 msgr2=0x7f09980ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.179+0000 7f0996ffd640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f099815e4b0 0x7f09980ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.179+0000 7f0996ffd640 1 -- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0998151220 msgr2=0x7f099814f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.179+0000 7f0996ffd640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0998151220 0x7f099814f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.179+0000 7f0996ffd640 1 -- 172.21.15.27:0/2948552858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c009400 con 0x7f0998151600 2023-08-11T15:52:31.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.180+0000 7f0996ffd640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 0x7f0998154430 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f098c009890 tx=0x7f098c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.180+0000 7f097ffff640 1 -- 172.21.15.27:0/2948552858 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c004230 con 0x7f0998151600 2023-08-11T15:52:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.180+0000 7f097ffff640 1 -- 172.21.15.27:0/2948552858 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c004390 con 0x7f0998151600 2023-08-11T15:52:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.180+0000 7f097ffff640 1 -- 172.21.15.27:0/2948552858 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c033950 con 0x7f0998151600 2023-08-11T15:52:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.180+0000 7f099dc14640 1 -- 172.21.15.27:0/2948552858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0998157030 con 0x7f0998151600 2023-08-11T15:52:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.180+0000 7f099dc14640 1 -- 172.21.15.27:0/2948552858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09981575f0 con 0x7f0998151600 2023-08-11T15:52:31.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.181+0000 7f099dc14640 1 -- 172.21.15.27:0/2948552858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09980ca030 con 0x7f0998151600 2023-08-11T15:52:31.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.182+0000 7f097ffff640 1 -- 172.21.15.27:0/2948552858 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f098c03a040 con 0x7f0998151600 2023-08-11T15:52:31.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.183+0000 7f097ffff640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f098406a050 0x7f098406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.183+0000 7f097ffff640 1 -- 172.21.15.27:0/2948552858 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f098c0aaa60 con 0x7f0998151600 2023-08-11T15:52:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.184+0000 7f09977fe640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f098406a050 0x7f098406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.185+0000 7f09977fe640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f098406a050 0x7f098406c510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0988000f60 tx=0x7f0988007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.188+0000 7f097ffff640 1 -- 172.21.15.27:0/2948552858 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f09980ca030 con 0x7f0998151600 2023-08-11T15:52:31.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.383+0000 7f099dc14640 1 -- 172.21.15.27:0/2948552858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f09980ca030 con 0x7f0998151600 2023-08-11T15:52:31.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.384+0000 7f097ffff640 1 -- 172.21.15.27:0/2948552858 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f09980ca030 con 0x7f0998151600 2023-08-11T15:52:31.387 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:31.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.387+0000 7f099dc14640 1 -- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f098406a050 msgr2=0x7f098406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.387+0000 7f099dc14640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f098406a050 0x7f098406c510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0988000f60 tx=0x7f0988007b50 comp rx=0 tx=0).stop 2023-08-11T15:52:31.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.387+0000 7f099dc14640 1 -- 172.21.15.27:0/2948552858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 msgr2=0x7f0998154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.387+0000 7f099dc14640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 0x7f0998154430 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f098c009890 tx=0x7f098c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.388+0000 7f099dc14640 1 -- 172.21.15.27:0/2948552858 shutdown_connections 2023-08-11T15:52:31.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.388+0000 7f099dc14640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f098406a050 0x7f098406c510 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.388+0000 7f099dc14640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f099815e4b0 0x7f09980ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.388+0000 7f099dc14640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0998151600 0x7f0998154430 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.388+0000 7f099dc14640 1 --2- 172.21.15.27:0/2948552858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0998151220 0x7f099814f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.388+0000 7f099dc14640 1 -- 172.21.15.27:0/2948552858 >> 172.21.15.27:0/2948552858 conn(0x7f09980afbe0 msgr2=0x7f099806b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:31.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.388+0000 7f099dc14640 1 -- 172.21.15.27:0/2948552858 shutdown_connections 2023-08-11T15:52:31.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.388+0000 7f099dc14640 1 -- 172.21.15.27:0/2948552858 wait complete. 2023-08-11T15:52:31.410 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:52:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.643+0000 7f8dc261f640 1 Processor -- start 2023-08-11T15:52:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.643+0000 7f8dc261f640 1 -- start start 2023-08-11T15:52:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.644+0000 7f8dc261f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0d0e30 0x7f8dbc15c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.644+0000 7f8dc261f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8dbc0cd920 0x7f8dbc0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.644+0000 7f8dc261f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc06aa30 con 0x7f8dbc15c570 2023-08-11T15:52:31.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.644+0000 7f8dc261f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dbc15ce90 con 0x7f8dbc0cd920 2023-08-11T15:52:31.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.644+0000 7f8dc261f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc06b2d0 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.645+0000 7f8dc0bbc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8dbc0cd920 0x7f8dbc0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.645+0000 7f8dc0bbc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8dbc0cd920 0x7f8dbc0d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44930/0 (socket says 172.21.15.27:44930) 2023-08-11T15:52:31.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.645+0000 7f8dc0bbc640 1 -- 172.21.15.27:0/3643566374 learned_addr learned my addr 172.21.15.27:0/3643566374 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:31.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.645+0000 7f8dc0bbc640 1 -- 172.21.15.27:0/3643566374 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0d0e30 msgr2=0x7f8dbc15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:31.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.645+0000 7f8dbb7fe640 1 --2- 172.21.15.27:0/3643566374 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0d0e30 0x7f8dbc15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.645+0000 7f8dc0bbc640 1 --2- 172.21.15.27:0/3643566374 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0d0e30 0x7f8dbc15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.645+0000 7f8dc0bbc640 1 -- 172.21.15.27:0/3643566374 >> v1:172.21.15.119:6789/0 conn(0x7f8dbc15c570 legacy=0x7f8dbc06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:31.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.645+0000 7f8dc0bbc640 1 -- 172.21.15.27:0/3643566374 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0d1270 con 0x7f8dbc0cd920 2023-08-11T15:52:31.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.645+0000 7f8dbb7fe640 1 --2- 172.21.15.27:0/3643566374 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0d0e30 0x7f8dbc15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:31.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.646+0000 7f8dc0bbc640 1 --2- 172.21.15.27:0/3643566374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0cd920 0x7f8dbc0d26a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009870 tx=0x7f8db402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1daf11512aee0d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:31.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.647+0000 7f8dbaffd640 1 -- 172.21.15.27:0/3643566374 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db40317c0 con 0x7f8dbc0cd920 2023-08-11T15:52:31.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.647+0000 7f8dbaffd640 1 -- 172.21.15.27:0/3643566374 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db4031920 con 0x7f8dbc0cd920 2023-08-11T15:52:31.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.647+0000 7f8dbaffd640 1 -- 172.21.15.27:0/3643566374 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db4031cb0 con 0x7f8dbc0cd920 2023-08-11T15:52:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.647+0000 7f8dc261f640 1 -- 172.21.15.27:0/3643566374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0cd920 msgr2=0x7f8dbc0d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.647+0000 7f8dc261f640 1 --2- 172.21.15.27:0/3643566374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0cd920 0x7f8dbc0d26a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009870 tx=0x7f8db402b860 comp rx=0 tx=0).stop 2023-08-11T15:52:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.648+0000 7f8dc261f640 1 -- 172.21.15.27:0/3643566374 shutdown_connections 2023-08-11T15:52:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.648+0000 7f8dc261f640 1 --2- 172.21.15.27:0/3643566374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0cd920 0x7f8dbc0d26a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.648+0000 7f8dc261f640 1 --2- 172.21.15.27:0/3643566374 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0d0e30 0x7f8dbc15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.648+0000 7f8dc261f640 1 -- 172.21.15.27:0/3643566374 >> 172.21.15.27:0/3643566374 conn(0x7f8dbc089ff0 msgr2=0x7f8dbc09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.648+0000 7f8dc261f640 1 -- 172.21.15.27:0/3643566374 shutdown_connections 2023-08-11T15:52:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.648+0000 7f8dc261f640 1 -- 172.21.15.27:0/3643566374 wait complete. 2023-08-11T15:52:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.649+0000 7f8dc261f640 1 Processor -- start 2023-08-11T15:52:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.650+0000 7f8dc261f640 1 -- start start 2023-08-11T15:52:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dc261f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0cd920 0x7f8dbc159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dc261f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0d0e30 0x7f8dbc0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dc261f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dbc15c570 0x7f8dbc0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dc261f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0b58d0 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dc261f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0b52a0 con 0x7f8dbc15c570 2023-08-11T15:52:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dc261f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0cc3c0 con 0x7f8dbc0cd920 2023-08-11T15:52:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dbb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0d0e30 0x7f8dbc0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dbb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0d0e30 0x7f8dbc0cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44942/0 (socket says 172.21.15.27:44942) 2023-08-11T15:52:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dbb7fe640 1 -- 172.21.15.27:0/3595926771 learned_addr learned my addr 172.21.15.27:0/3595926771 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dc0bbc640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dbc15c570 0x7f8dbc0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dbbfff640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0cd920 0x7f8dbc159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dbb7fe640 1 -- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0cd920 msgr2=0x7f8dbc159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dbb7fe640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0cd920 0x7f8dbc159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dbb7fe640 1 -- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dbc15c570 msgr2=0x7f8dbc0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dbb7fe640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dbc15c570 0x7f8dbc0b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dbb7fe640 1 -- 172.21.15.27:0/3595926771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db40094d0 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.651+0000 7f8dbbfff640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0cd920 0x7f8dbc159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:31.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.652+0000 7f8dbb7fe640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0d0e30 0x7f8dbc0cf0e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c7f0 tx=0x7f8db000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:31.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.652+0000 7f8db8ff9640 1 -- 172.21.15.27:0/3595926771 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0007820 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.652+0000 7f8db8ff9640 1 -- 172.21.15.27:0/3595926771 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db0007980 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.652+0000 7f8dc261f640 1 -- 172.21.15.27:0/3595926771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dbc0cc6a0 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.652+0000 7f8db8ff9640 1 -- 172.21.15.27:0/3595926771 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0007cf0 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.652+0000 7f8dc261f640 1 -- 172.21.15.27:0/3595926771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc0cf620 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.653+0000 7f8dc261f640 1 -- 172.21.15.27:0/3595926771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc073990 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.654+0000 7f8db8ff9640 1 -- 172.21.15.27:0/3595926771 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8db00113f0 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.656+0000 7f8db8ff9640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8da806a050 0x7f8da806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.656+0000 7f8db8ff9640 1 -- 172.21.15.27:0/3595926771 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8db0088bc0 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.657+0000 7f8dbbfff640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8da806a050 0x7f8da806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.658+0000 7f8dbbfff640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8da806a050 0x7f8da806c510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8dac007fd0 tx=0x7f8dac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.661+0000 7f8db8ff9640 1 -- 172.21.15.27:0/3595926771 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f8db0053f80 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.867+0000 7f8dc261f640 1 -- 172.21.15.27:0/3595926771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8dbc0d3480 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.868+0000 7f8db8ff9640 1 -- 172.21.15.27:0/3595926771 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8db00575a0 con 0x7f8dbc0d0e30 2023-08-11T15:52:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:52:31.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.870+0000 7f8dc261f640 1 -- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8da806a050 msgr2=0x7f8da806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.870+0000 7f8dc261f640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8da806a050 0x7f8da806c510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8dac007fd0 tx=0x7f8dac009040 comp rx=0 tx=0).stop 2023-08-11T15:52:31.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.871+0000 7f8dc261f640 1 -- 172.21.15.27:0/3595926771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0d0e30 msgr2=0x7f8dbc0cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:31.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.871+0000 7f8dc261f640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0d0e30 0x7f8dbc0cf0e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c7f0 tx=0x7f8db000ccb0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.871+0000 7f8dc261f640 1 -- 172.21.15.27:0/3595926771 shutdown_connections 2023-08-11T15:52:31.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.871+0000 7f8dc261f640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8da806a050 0x7f8da806c510 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.871+0000 7f8dc261f640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dbc15c570 0x7f8dbc0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.871+0000 7f8dc261f640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dbc0d0e30 0x7f8dbc0cf0e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.871+0000 7f8dc261f640 1 --2- 172.21.15.27:0/3595926771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dbc0cd920 0x7f8dbc159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.871+0000 7f8dc261f640 1 -- 172.21.15.27:0/3595926771 >> 172.21.15.27:0/3595926771 conn(0x7f8dbc089ff0 msgr2=0x7f8dbc15bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:31.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.871+0000 7f8dc261f640 1 -- 172.21.15.27:0/3595926771 shutdown_connections 2023-08-11T15:52:31.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:31.871+0000 7f8dc261f640 1 -- 172.21.15.27:0/3595926771 wait complete. 2023-08-11T15:52:31.892 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:52:31.893 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:31.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:32.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.185+0000 7fee55339640 1 Processor -- start 2023-08-11T15:52:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.185+0000 7fee55339640 1 -- start start 2023-08-11T15:52:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.187+0000 7fee55339640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee5015c620 0x7fee500b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.187+0000 7fee55339640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee500cfb00 0x7fee500d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.187+0000 7fee55339640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee500cda30 con 0x7fee500b82f0 2023-08-11T15:52:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.187+0000 7fee55339640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee500ccc10 con 0x7fee500cfb00 2023-08-11T15:52:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.187+0000 7fee55339640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee50161110 con 0x7fee5015c620 2023-08-11T15:52:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.187+0000 7fee4f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee500cfb00 0x7fee500d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.187+0000 7fee4f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee500cfb00 0x7fee500d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44946/0 (socket says 172.21.15.27:44946) 2023-08-11T15:52:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.187+0000 7fee4f7fe640 1 -- 172.21.15.27:0/3117410627 learned_addr learned my addr 172.21.15.27:0/3117410627 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.187+0000 7fee4effd640 1 --2- 172.21.15.27:0/3117410627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee5015c620 0x7fee500b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.188+0000 7fee4f7fe640 1 -- 172.21.15.27:0/3117410627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee5015c620 msgr2=0x7fee500b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.188+0000 7fee4f7fe640 1 --2- 172.21.15.27:0/3117410627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee5015c620 0x7fee500b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.188+0000 7fee4f7fe640 1 -- 172.21.15.27:0/3117410627 >> v1:172.21.15.119:6789/0 conn(0x7fee500b82f0 legacy=0x7fee500cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.188+0000 7fee4f7fe640 1 -- 172.21.15.27:0/3117410627 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500cce20 con 0x7fee500cfb00 2023-08-11T15:52:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.189+0000 7fee4f7fe640 1 --2- 172.21.15.27:0/3117410627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee500cfb00 0x7fee500d0b00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fee48009e50 tx=0x7fee4802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d342060e5a3294e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.189+0000 7fee4dffb640 1 -- 172.21.15.27:0/3117410627 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4802ca70 con 0x7fee500cfb00 2023-08-11T15:52:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.189+0000 7fee4dffb640 1 -- 172.21.15.27:0/3117410627 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4802cbd0 con 0x7fee500cfb00 2023-08-11T15:52:32.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.189+0000 7fee4dffb640 1 -- 172.21.15.27:0/3117410627 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee480343f0 con 0x7fee500cfb00 2023-08-11T15:52:32.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.190+0000 7fee55339640 1 -- 172.21.15.27:0/3117410627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee500cfb00 msgr2=0x7fee500d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:32.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.190+0000 7fee55339640 1 --2- 172.21.15.27:0/3117410627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee500cfb00 0x7fee500d0b00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fee48009e50 tx=0x7fee4802bed0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.190+0000 7fee55339640 1 -- 172.21.15.27:0/3117410627 shutdown_connections 2023-08-11T15:52:32.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.190+0000 7fee55339640 1 --2- 172.21.15.27:0/3117410627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee500cfb00 0x7fee500d0b00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.190+0000 7fee55339640 1 --2- 172.21.15.27:0/3117410627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee5015c620 0x7fee500b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.190+0000 7fee55339640 1 -- 172.21.15.27:0/3117410627 >> 172.21.15.27:0/3117410627 conn(0x7fee50071110 msgr2=0x7fee500714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.190+0000 7fee55339640 1 -- 172.21.15.27:0/3117410627 shutdown_connections 2023-08-11T15:52:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.190+0000 7fee55339640 1 -- 172.21.15.27:0/3117410627 wait complete. 2023-08-11T15:52:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.192+0000 7fee55339640 1 Processor -- start 2023-08-11T15:52:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.192+0000 7fee55339640 1 -- start start 2023-08-11T15:52:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.193+0000 7fee55339640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee500b82f0 0x7fee5015da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.193+0000 7fee55339640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee500cfb00 0x7fee500cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.193+0000 7fee55339640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee5015c620 0x7fee5015f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.193+0000 7fee55339640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5015e280 con 0x7fee5015c620 2023-08-11T15:52:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.193+0000 7fee55339640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5015e560 con 0x7fee500cfb00 2023-08-11T15:52:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.193+0000 7fee55339640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5015e840 con 0x7fee500b82f0 2023-08-11T15:52:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.193+0000 7fee4f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee5015c620 0x7fee5015f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.193+0000 7fee4f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee5015c620 0x7fee5015f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44950/0 (socket says 172.21.15.27:44950) 2023-08-11T15:52:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.193+0000 7fee4f7fe640 1 -- 172.21.15.27:0/1540486462 learned_addr learned my addr 172.21.15.27:0/1540486462 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.194+0000 7fee4effd640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee500b82f0 0x7fee5015da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.194+0000 7fee4e7fc640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee500cfb00 0x7fee500cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:32.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.194+0000 7fee4f7fe640 1 -- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee500b82f0 msgr2=0x7fee5015da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:32.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.194+0000 7fee4f7fe640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee500b82f0 0x7fee5015da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.194+0000 7fee4f7fe640 1 -- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee500cfb00 msgr2=0x7fee500cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:32.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.194+0000 7fee4f7fe640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee500cfb00 0x7fee500cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.194+0000 7fee4f7fe640 1 -- 172.21.15.27:0/1540486462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48009b30 con 0x7fee5015c620 2023-08-11T15:52:32.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.194+0000 7fee4f7fe640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee5015c620 0x7fee5015f320 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fee48036040 tx=0x7fee48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:32.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.195+0000 7fee377fe640 1 -- 172.21.15.27:0/1540486462 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48009510 con 0x7fee5015c620 2023-08-11T15:52:32.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.195+0000 7fee377fe640 1 -- 172.21.15.27:0/1540486462 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4803e450 con 0x7fee5015c620 2023-08-11T15:52:32.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.195+0000 7fee377fe640 1 -- 172.21.15.27:0/1540486462 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4803e7e0 con 0x7fee5015c620 2023-08-11T15:52:32.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.195+0000 7fee55339640 1 -- 172.21.15.27:0/1540486462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5015eb20 con 0x7fee5015c620 2023-08-11T15:52:32.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.195+0000 7fee55339640 1 -- 172.21.15.27:0/1540486462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5015df90 con 0x7fee5015c620 2023-08-11T15:52:32.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.197+0000 7fee55339640 1 -- 172.21.15.27:0/1540486462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5015f040 con 0x7fee5015c620 2023-08-11T15:52:32.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.198+0000 7fee377fe640 1 -- 172.21.15.27:0/1540486462 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fee4803e940 con 0x7fee5015c620 2023-08-11T15:52:32.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.199+0000 7fee377fe640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3c06a050 0x7fee3c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.199+0000 7fee377fe640 1 -- 172.21.15.27:0/1540486462 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fee480ab740 con 0x7fee5015c620 2023-08-11T15:52:32.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.200+0000 7fee4effd640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3c06a050 0x7fee3c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:32.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.201+0000 7fee4effd640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3c06a050 0x7fee3c06c510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fee40006fd0 tx=0x7fee40008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:32.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.205+0000 7fee377fe640 1 -- 172.21.15.27:0/1540486462 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fee5015f040 con 0x7fee5015c620 2023-08-11T15:52:32.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.408+0000 7fee55339640 1 -- 172.21.15.27:0/1540486462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee5015f040 con 0x7fee5015c620 2023-08-11T15:52:32.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.411+0000 7fee377fe640 1 -- 172.21.15.27:0/1540486462 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7fee5015f040 con 0x7fee5015c620 2023-08-11T15:52:32.414 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:32.414 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.414+0000 7fee55339640 1 -- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3c06a050 msgr2=0x7fee3c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.414+0000 7fee55339640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3c06a050 0x7fee3c06c510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fee40006fd0 tx=0x7fee40008040 comp rx=0 tx=0).stop 2023-08-11T15:52:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.414+0000 7fee55339640 1 -- 172.21.15.27:0/1540486462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee5015c620 msgr2=0x7fee5015f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.414+0000 7fee55339640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee5015c620 0x7fee5015f320 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fee48036040 tx=0x7fee48004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.414+0000 7fee55339640 1 -- 172.21.15.27:0/1540486462 shutdown_connections 2023-08-11T15:52:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.414+0000 7fee55339640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee3c06a050 0x7fee3c06c510 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.414+0000 7fee55339640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee5015c620 0x7fee5015f320 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.414+0000 7fee55339640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee500cfb00 0x7fee500cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.414+0000 7fee55339640 1 --2- 172.21.15.27:0/1540486462 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee500b82f0 0x7fee5015da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.415+0000 7fee55339640 1 -- 172.21.15.27:0/1540486462 >> 172.21.15.27:0/1540486462 conn(0x7fee50071110 msgr2=0x7fee500b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.415+0000 7fee55339640 1 -- 172.21.15.27:0/1540486462 shutdown_connections 2023-08-11T15:52:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.415+0000 7fee55339640 1 -- 172.21.15.27:0/1540486462 wait complete. 2023-08-11T15:52:32.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:32.438 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:32.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.665+0000 7f0813083640 1 Processor -- start 2023-08-11T15:52:32.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.665+0000 7f0813083640 1 -- start start 2023-08-11T15:52:32.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.667+0000 7f0813083640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f080c15ceb0 0x7f080c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.667+0000 7f0813083640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f080c0cb530 0x7f080c070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.667+0000 7f0813083640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f080c0cd040 con 0x7f080c0ca080 2023-08-11T15:52:32.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.667+0000 7f0813083640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f080c0cb9e0 con 0x7f080c0cb530 2023-08-11T15:52:32.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.667+0000 7f0813083640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080c070600 con 0x7f080c15ceb0 2023-08-11T15:52:32.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.667+0000 7f0811620640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f080c0cb530 0x7f080c070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:32.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.667+0000 7f0811620640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f080c0cb530 0x7f080c070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44966/0 (socket says 172.21.15.27:44966) 2023-08-11T15:52:32.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.667+0000 7f0811620640 1 -- 172.21.15.27:0/4273080912 learned_addr learned my addr 172.21.15.27:0/4273080912 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:32.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.668+0000 7f080bfff640 1 --2- 172.21.15.27:0/4273080912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f080c15ceb0 0x7f080c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:32.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.668+0000 7f0811620640 1 -- 172.21.15.27:0/4273080912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f080c15ceb0 msgr2=0x7f080c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:32.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.668+0000 7f0811620640 1 --2- 172.21.15.27:0/4273080912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f080c15ceb0 0x7f080c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.668+0000 7f0811620640 1 -- 172.21.15.27:0/4273080912 >> v1:172.21.15.119:6789/0 conn(0x7f080c0ca080 legacy=0x7f080c15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:32.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.668+0000 7f0811620640 1 -- 172.21.15.27:0/4273080912 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c15d370 con 0x7f080c0cb530 2023-08-11T15:52:32.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.669+0000 7f0811620640 1 --2- 172.21.15.27:0/4273080912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c0cb530 0x7f080c070ac0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0804009e50 tx=0x7f080402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c48df80176fcbb9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:32.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.669+0000 7f080b7fe640 1 -- 172.21.15.27:0/4273080912 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080402c9e0 con 0x7f080c0cb530 2023-08-11T15:52:32.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.669+0000 7f080b7fe640 1 -- 172.21.15.27:0/4273080912 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080402cb40 con 0x7f080c0cb530 2023-08-11T15:52:32.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.669+0000 7f080b7fe640 1 -- 172.21.15.27:0/4273080912 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08040343f0 con 0x7f080c0cb530 2023-08-11T15:52:32.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.670+0000 7f0813083640 1 -- 172.21.15.27:0/4273080912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c0cb530 msgr2=0x7f080c070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:32.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.670+0000 7f0813083640 1 --2- 172.21.15.27:0/4273080912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c0cb530 0x7f080c070ac0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0804009e50 tx=0x7f080402be40 comp rx=0 tx=0).stop 2023-08-11T15:52:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.670+0000 7f0813083640 1 -- 172.21.15.27:0/4273080912 shutdown_connections 2023-08-11T15:52:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.670+0000 7f0813083640 1 --2- 172.21.15.27:0/4273080912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c0cb530 0x7f080c070ac0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.670+0000 7f0813083640 1 --2- 172.21.15.27:0/4273080912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f080c15ceb0 0x7f080c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.670+0000 7f0813083640 1 -- 172.21.15.27:0/4273080912 >> 172.21.15.27:0/4273080912 conn(0x7f080c069bf0 msgr2=0x7f080c06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.670+0000 7f0813083640 1 -- 172.21.15.27:0/4273080912 shutdown_connections 2023-08-11T15:52:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.670+0000 7f0813083640 1 -- 172.21.15.27:0/4273080912 wait complete. 2023-08-11T15:52:32.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.672+0000 7f0813083640 1 Processor -- start 2023-08-11T15:52:32.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.672+0000 7f0813083640 1 -- start start 2023-08-11T15:52:32.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.673+0000 7f0813083640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080c0ca080 0x7f080c14dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.673+0000 7f0813083640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f080c0cb530 0x7f080c151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.673+0000 7f0813083640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c15ceb0 0x7f080c152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.673+0000 7f0813083640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c1515d0 con 0x7f080c15ceb0 2023-08-11T15:52:32.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.673+0000 7f0813083640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c153bd0 con 0x7f080c0ca080 2023-08-11T15:52:32.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.673+0000 7f0813083640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080c153610 con 0x7f080c0cb530 2023-08-11T15:52:32.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.673+0000 7f0811620640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c15ceb0 0x7f080c152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:32.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.673+0000 7f0811620640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c15ceb0 0x7f080c152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44982/0 (socket says 172.21.15.27:44982) 2023-08-11T15:52:32.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.673+0000 7f0811620640 1 -- 172.21.15.27:0/2025348943 learned_addr learned my addr 172.21.15.27:0/2025348943 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:32.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f0811620640 1 -- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f080c0cb530 msgr2=0x7f080c151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:32.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f0811620640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f080c0cb530 0x7f080c151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f0811620640 1 -- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080c0ca080 msgr2=0x7f080c14dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:32.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f0811620640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080c0ca080 0x7f080c14dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f0811620640 1 -- 172.21.15.27:0/2025348943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0804009b30 con 0x7f080c15ceb0 2023-08-11T15:52:32.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f0811620640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c15ceb0 0x7f080c152b70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0804036040 tx=0x7f0804004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:32.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f08097fa640 1 -- 172.21.15.27:0/2025348943 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804009510 con 0x7f080c15ceb0 2023-08-11T15:52:32.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f08097fa640 1 -- 172.21.15.27:0/2025348943 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080403e450 con 0x7f080c15ceb0 2023-08-11T15:52:32.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f0813083640 1 -- 172.21.15.27:0/2025348943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080c1538f0 con 0x7f080c15ceb0 2023-08-11T15:52:32.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f08097fa640 1 -- 172.21.15.27:0/2025348943 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080403e7e0 con 0x7f080c15ceb0 2023-08-11T15:52:32.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.674+0000 7f0813083640 1 -- 172.21.15.27:0/2025348943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c14c210 con 0x7f080c15ceb0 2023-08-11T15:52:32.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.675+0000 7f0813083640 1 -- 172.21.15.27:0/2025348943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080c04e420 con 0x7f080c15ceb0 2023-08-11T15:52:32.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.676+0000 7f08097fa640 1 -- 172.21.15.27:0/2025348943 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f080403e940 con 0x7f080c15ceb0 2023-08-11T15:52:32.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.677+0000 7f08097fa640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f07f006a050 0x7f07f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:32.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.677+0000 7f08097fa640 1 -- 172.21.15.27:0/2025348943 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f08040ab6c0 con 0x7f080c15ceb0 2023-08-11T15:52:32.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.678+0000 7f0810e1f640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f07f006a050 0x7f07f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:32.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.679+0000 7f0810e1f640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f07f006a050 0x7f07f006c510 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f080c150e30 tx=0x7f08000079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:32.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.681+0000 7f08097fa640 1 -- 172.21.15.27:0/2025348943 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0804076a70 con 0x7f080c15ceb0 2023-08-11T15:52:32.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.875+0000 7f0813083640 1 -- 172.21.15.27:0/2025348943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f080c0ccad0 con 0x7f080c15ceb0 2023-08-11T15:52:32.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.875+0000 7f08097fa640 1 -- 172.21.15.27:0/2025348943 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f080407a090 con 0x7f080c15ceb0 2023-08-11T15:52:32.878 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:32.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.878+0000 7f0813083640 1 -- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f07f006a050 msgr2=0x7f07f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:32.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.878+0000 7f0813083640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f07f006a050 0x7f07f006c510 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f080c150e30 tx=0x7f08000079b0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.878+0000 7f0813083640 1 -- 172.21.15.27:0/2025348943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c15ceb0 msgr2=0x7f080c152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:32.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.878+0000 7f0813083640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c15ceb0 0x7f080c152b70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0804036040 tx=0x7f0804004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:32.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.878+0000 7f0813083640 1 -- 172.21.15.27:0/2025348943 shutdown_connections 2023-08-11T15:52:32.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.878+0000 7f0813083640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f07f006a050 0x7f07f006c510 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.878+0000 7f0813083640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f080c0cb530 0x7f080c151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.878+0000 7f0813083640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080c15ceb0 0x7f080c152b70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.878+0000 7f0813083640 1 --2- 172.21.15.27:0/2025348943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080c0ca080 0x7f080c14dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:32.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.878+0000 7f0813083640 1 -- 172.21.15.27:0/2025348943 >> 172.21.15.27:0/2025348943 conn(0x7f080c069bf0 msgr2=0x7f080c0adbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:32.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.879+0000 7f0813083640 1 -- 172.21.15.27:0/2025348943 shutdown_connections 2023-08-11T15:52:32.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:32.879+0000 7f0813083640 1 -- 172.21.15.27:0/2025348943 wait complete. 2023-08-11T15:52:32.901 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:33.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.131+0000 7f13e0cec640 1 Processor -- start 2023-08-11T15:52:33.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.132+0000 7f13e0cec640 1 -- start start 2023-08-11T15:52:33.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.133+0000 7f13e0cec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13dc0cb700 0x7f13dc13fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.133+0000 7f13e0cec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13dc15b6b0 0x7f13dc1401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.133+0000 7f13e0cec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13dc140ba0 con 0x7f13dc154ac0 2023-08-11T15:52:33.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.133+0000 7f13e0cec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13dc071b10 con 0x7f13dc15b6b0 2023-08-11T15:52:33.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.133+0000 7f13e0cec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc15b220 con 0x7f13dc0cb700 2023-08-11T15:52:33.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.134+0000 7f13da59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f13dc154ac0 0x7f13dc0caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57874/0 (socket says 172.21.15.27:57874) 2023-08-11T15:52:33.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.134+0000 7f13da59c640 1 -- 172.21.15.27:0/1297457019 learned_addr learned my addr 172.21.15.27:0/1297457019 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:33.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.134+0000 7f13dad9d640 1 --2- 172.21.15.27:0/1297457019 >> v2:172.21.15.27:3300/0 conn(0x7f13dc15b6b0 0x7f13dc1401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.134+0000 7f13d3fff640 1 --2- 172.21.15.27:0/1297457019 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13dc0cb700 0x7f13dc13fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.134+0000 7f13dad9d640 1 -- 172.21.15.27:0/1297457019 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13dc0cb700 msgr2=0x7f13dc13fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:33.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.134+0000 7f13dad9d640 1 --2- 172.21.15.27:0/1297457019 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13dc0cb700 0x7f13dc13fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.134+0000 7f13dad9d640 1 -- 172.21.15.27:0/1297457019 >> v1:172.21.15.27:6789/0 conn(0x7f13dc154ac0 legacy=0x7f13dc0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.134+0000 7f13dad9d640 1 -- 172.21.15.27:0/1297457019 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc1408c0 con 0x7f13dc15b6b0 2023-08-11T15:52:33.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.134+0000 7f13d9d9b640 1 -- 172.21.15.27:0/1297457019 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799451298 0 0) 0x7f13dc140ba0 con 0x7f13dc154ac0 2023-08-11T15:52:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.135+0000 7f13dad9d640 1 --2- 172.21.15.27:0/1297457019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc15b6b0 0x7f13dc1401d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009d90 tx=0x7f13d402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ffdb7611469445 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.136+0000 7f13d9d9b640 1 -- 172.21.15.27:0/1297457019 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d402cbc0 con 0x7f13dc15b6b0 2023-08-11T15:52:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.136+0000 7f13d9d9b640 1 -- 172.21.15.27:0/1297457019 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d402cd20 con 0x7f13dc15b6b0 2023-08-11T15:52:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.136+0000 7f13d9d9b640 1 -- 172.21.15.27:0/1297457019 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d4034660 con 0x7f13dc15b6b0 2023-08-11T15:52:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.136+0000 7f13e0cec640 1 -- 172.21.15.27:0/1297457019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc15b6b0 msgr2=0x7f13dc1401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.136+0000 7f13e0cec640 1 --2- 172.21.15.27:0/1297457019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc15b6b0 0x7f13dc1401d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009d90 tx=0x7f13d402be10 comp rx=0 tx=0).stop 2023-08-11T15:52:33.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.136+0000 7f13e0cec640 1 -- 172.21.15.27:0/1297457019 shutdown_connections 2023-08-11T15:52:33.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.136+0000 7f13e0cec640 1 --2- 172.21.15.27:0/1297457019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc15b6b0 0x7f13dc1401d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.136+0000 7f13e0cec640 1 --2- 172.21.15.27:0/1297457019 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13dc0cb700 0x7f13dc13fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.137+0000 7f13e0cec640 1 -- 172.21.15.27:0/1297457019 >> 172.21.15.27:0/1297457019 conn(0x7f13dc069bf0 msgr2=0x7f13dc06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:33.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.137+0000 7f13e0cec640 1 -- 172.21.15.27:0/1297457019 shutdown_connections 2023-08-11T15:52:33.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.137+0000 7f13e0cec640 1 -- 172.21.15.27:0/1297457019 wait complete. 2023-08-11T15:52:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.138+0000 7f13e0cec640 1 Processor -- start 2023-08-11T15:52:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.139+0000 7f13e0cec640 1 -- start start 2023-08-11T15:52:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.139+0000 7f13e0cec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc0cb700 0x7f13dc150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.139+0000 7f13e0cec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13dc154ac0 0x7f13dc1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.139+0000 7f13e0cec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13dc15b6b0 0x7f13dc1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.139+0000 7f13e0cec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc156ad0 con 0x7f13dc0cb700 2023-08-11T15:52:33.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.139+0000 7f13e0cec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc0b08f0 con 0x7f13dc154ac0 2023-08-11T15:52:33.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.139+0000 7f13e0cec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc0b02b0 con 0x7f13dc15b6b0 2023-08-11T15:52:33.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.140+0000 7f13da59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc0cb700 0x7f13dc150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.140+0000 7f13da59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc0cb700 0x7f13dc150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45004/0 (socket says 172.21.15.27:45004) 2023-08-11T15:52:33.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.140+0000 7f13da59c640 1 -- 172.21.15.27:0/3149502997 learned_addr learned my addr 172.21.15.27:0/3149502997 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:33.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.140+0000 7f13dad9d640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13dc15b6b0 0x7f13dc1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.140+0000 7f13da59c640 1 -- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13dc15b6b0 msgr2=0x7f13dc1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.140+0000 7f13da59c640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13dc15b6b0 0x7f13dc1555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.140+0000 7f13da59c640 1 -- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13dc154ac0 msgr2=0x7f13dc1550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.140+0000 7f13da59c640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13dc154ac0 0x7f13dc1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.140+0000 7f13da59c640 1 -- 172.21.15.27:0/3149502997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4009a70 con 0x7f13dc0cb700 2023-08-11T15:52:33.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.141+0000 7f13da59c640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc0cb700 0x7f13dc150f50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00c4d0 tx=0x7f13cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.141+0000 7f13d2ffd640 1 -- 172.21.15.27:0/3149502997 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0074e0 con 0x7f13dc0cb700 2023-08-11T15:52:33.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.141+0000 7f13d2ffd640 1 -- 172.21.15.27:0/3149502997 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc007640 con 0x7f13dc0cb700 2023-08-11T15:52:33.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.141+0000 7f13d2ffd640 1 -- 172.21.15.27:0/3149502997 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc004a20 con 0x7f13dc0cb700 2023-08-11T15:52:33.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.141+0000 7f13e0cec640 1 -- 172.21.15.27:0/3149502997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13dc156d20 con 0x7f13dc0cb700 2023-08-11T15:52:33.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.141+0000 7f13e0cec640 1 -- 172.21.15.27:0/3149502997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13dc1572e0 con 0x7f13dc0cb700 2023-08-11T15:52:33.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.142+0000 7f13e0cec640 1 -- 172.21.15.27:0/3149502997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13dc157500 con 0x7f13dc0cb700 2023-08-11T15:52:33.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.143+0000 7f13d2ffd640 1 -- 172.21.15.27:0/3149502997 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f13cc004b80 con 0x7f13dc0cb700 2023-08-11T15:52:33.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.144+0000 7f13d2ffd640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13c006a080 0x7f13c006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.144+0000 7f13d2ffd640 1 -- 172.21.15.27:0/3149502997 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f13cc087af0 con 0x7f13dc0cb700 2023-08-11T15:52:33.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.145+0000 7f13d3fff640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13c006a080 0x7f13c006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.146+0000 7f13d3fff640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13c006a080 0x7f13c006c540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f13c4006fd0 tx=0x7f13c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.149+0000 7f13d2ffd640 1 -- 172.21.15.27:0/3149502997 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f13cc052e10 con 0x7f13dc0cb700 2023-08-11T15:52:33.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.342+0000 7f13e0cec640 1 -- 172.21.15.27:0/3149502997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f13dc0c59c0 con 0x7f13dc0cb700 2023-08-11T15:52:33.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.342+0000 7f13d2ffd640 1 -- 172.21.15.27:0/3149502997 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f13cc018020 con 0x7f13dc0cb700 2023-08-11T15:52:33.345 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:33.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.345+0000 7f13e0cec640 1 -- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13c006a080 msgr2=0x7f13c006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.345+0000 7f13e0cec640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13c006a080 0x7f13c006c540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f13c4006fd0 tx=0x7f13c4008040 comp rx=0 tx=0).stop 2023-08-11T15:52:33.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.345+0000 7f13e0cec640 1 -- 172.21.15.27:0/3149502997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc0cb700 msgr2=0x7f13dc150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.345+0000 7f13e0cec640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc0cb700 0x7f13dc150f50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00c4d0 tx=0x7f13cc00c990 comp rx=0 tx=0).stop 2023-08-11T15:52:33.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.345+0000 7f13e0cec640 1 -- 172.21.15.27:0/3149502997 shutdown_connections 2023-08-11T15:52:33.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.345+0000 7f13e0cec640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13c006a080 0x7f13c006c540 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.345+0000 7f13e0cec640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13dc15b6b0 0x7f13dc1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.345+0000 7f13e0cec640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13dc154ac0 0x7f13dc1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.345+0000 7f13e0cec640 1 --2- 172.21.15.27:0/3149502997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13dc0cb700 0x7f13dc150f50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.345+0000 7f13e0cec640 1 -- 172.21.15.27:0/3149502997 >> 172.21.15.27:0/3149502997 conn(0x7f13dc069bf0 msgr2=0x7f13dc15e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:33.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.346+0000 7f13e0cec640 1 -- 172.21.15.27:0/3149502997 shutdown_connections 2023-08-11T15:52:33.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.346+0000 7f13e0cec640 1 -- 172.21.15.27:0/3149502997 wait complete. 2023-08-11T15:52:33.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:33.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.603+0000 7f8bfb90f640 1 Processor -- start 2023-08-11T15:52:33.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.604+0000 7f8bfb90f640 1 -- start start 2023-08-11T15:52:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.605+0000 7f8bfb90f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bf4151220 0x7f8bf406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.605+0000 7f8bfb90f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8bf406bdf0 0x7f8bf415df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.605+0000 7f8bfb90f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf406c850 con 0x7f8bf4151600 2023-08-11T15:52:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.606+0000 7f8bfb90f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf406c290 con 0x7f8bf406bdf0 2023-08-11T15:52:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.606+0000 7f8bfb90f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf406c570 con 0x7f8bf4151220 2023-08-11T15:52:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.606+0000 7f8bf8eaa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8bf406bdf0 0x7f8bf415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.606+0000 7f8bf8eaa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8bf406bdf0 0x7f8bf415df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45016/0 (socket says 172.21.15.27:45016) 2023-08-11T15:52:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.606+0000 7f8bf8eaa640 1 -- 172.21.15.27:0/765699048 learned_addr learned my addr 172.21.15.27:0/765699048 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.606+0000 7f8bf9eac640 1 --1- 172.21.15.27:0/765699048 >> v1:172.21.15.27:6789/0 conn(0x7f8bf4151600 0x7f8bf40ca850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57878/0 (socket says 172.21.15.27:57878) 2023-08-11T15:52:33.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.606+0000 7f8bf96ab640 1 --2- 172.21.15.27:0/765699048 >> v2:172.21.15.119:3300/0 conn(0x7f8bf4151220 0x7f8bf406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.607+0000 7f8bf8eaa640 1 -- 172.21.15.27:0/765699048 >> v2:172.21.15.119:3300/0 conn(0x7f8bf4151220 msgr2=0x7f8bf406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:33.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.607+0000 7f8bf8eaa640 1 --2- 172.21.15.27:0/765699048 >> v2:172.21.15.119:3300/0 conn(0x7f8bf4151220 0x7f8bf406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.607+0000 7f8bf8eaa640 1 -- 172.21.15.27:0/765699048 >> v1:172.21.15.27:6789/0 conn(0x7f8bf4151600 legacy=0x7f8bf40ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.607+0000 7f8bf8eaa640 1 -- 172.21.15.27:0/765699048 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf40ca050 con 0x7f8bf406bdf0 2023-08-11T15:52:33.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.607+0000 7f8bebfff640 1 -- 172.21.15.27:0/765699048 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910707225 0 0) 0x7f8bf406c850 con 0x7f8bf4151600 2023-08-11T15:52:33.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.608+0000 7f8bf8eaa640 1 --2- 172.21.15.27:0/765699048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf406bdf0 0x7f8bf415df70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40b0ae0 tx=0x7f8be402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269dc429e4fc9b67 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.608+0000 7f8bebfff640 1 -- 172.21.15.27:0/765699048 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be40317e0 con 0x7f8bf406bdf0 2023-08-11T15:52:33.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.608+0000 7f8bebfff640 1 -- 172.21.15.27:0/765699048 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be4031940 con 0x7f8bf406bdf0 2023-08-11T15:52:33.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.608+0000 7f8bebfff640 1 -- 172.21.15.27:0/765699048 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4031cd0 con 0x7f8bf406bdf0 2023-08-11T15:52:33.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.609+0000 7f8bfb90f640 1 -- 172.21.15.27:0/765699048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf406bdf0 msgr2=0x7f8bf415df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.609+0000 7f8bfb90f640 1 --2- 172.21.15.27:0/765699048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf406bdf0 0x7f8bf415df70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40b0ae0 tx=0x7f8be402b880 comp rx=0 tx=0).stop 2023-08-11T15:52:33.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.609+0000 7f8bfb90f640 1 -- 172.21.15.27:0/765699048 shutdown_connections 2023-08-11T15:52:33.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.609+0000 7f8bfb90f640 1 --2- 172.21.15.27:0/765699048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf406bdf0 0x7f8bf415df70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.609+0000 7f8bfb90f640 1 --2- 172.21.15.27:0/765699048 >> v2:172.21.15.119:3300/0 conn(0x7f8bf4151220 0x7f8bf406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.609+0000 7f8bfb90f640 1 -- 172.21.15.27:0/765699048 >> 172.21.15.27:0/765699048 conn(0x7f8bf40afbe0 msgr2=0x7f8bf40affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:33.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.609+0000 7f8bfb90f640 1 -- 172.21.15.27:0/765699048 shutdown_connections 2023-08-11T15:52:33.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.609+0000 7f8bfb90f640 1 -- 172.21.15.27:0/765699048 wait complete. 2023-08-11T15:52:33.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.612+0000 7f8bfb90f640 1 Processor -- start 2023-08-11T15:52:33.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.612+0000 7f8bfb90f640 1 -- start start 2023-08-11T15:52:33.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.613+0000 7f8bfb90f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8bf406bdf0 0x7f8bf414fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.613+0000 7f8bfb90f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf4151220 0x7f8bf40cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.613+0000 7f8bfb90f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf4151600 0x7f8bf40cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.613+0000 7f8bfb90f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4156d70 con 0x7f8bf4151220 2023-08-11T15:52:33.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.613+0000 7f8bfb90f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4157050 con 0x7f8bf4151600 2023-08-11T15:52:33.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.613+0000 7f8bfb90f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4157330 con 0x7f8bf406bdf0 2023-08-11T15:52:33.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf8eaa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf4151220 0x7f8bf40cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf8eaa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf4151220 0x7f8bf40cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45018/0 (socket says 172.21.15.27:45018) 2023-08-11T15:52:33.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf8eaa640 1 -- 172.21.15.27:0/3391376953 learned_addr learned my addr 172.21.15.27:0/3391376953 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:33.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf9eac640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf4151600 0x7f8bf40cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf96ab640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8bf406bdf0 0x7f8bf414fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf8eaa640 1 -- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8bf406bdf0 msgr2=0x7f8bf414fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf8eaa640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8bf406bdf0 0x7f8bf414fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf8eaa640 1 -- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf4151600 msgr2=0x7f8bf40cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf8eaa640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf4151600 0x7f8bf40cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf8eaa640 1 -- 172.21.15.27:0/3391376953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be40094d0 con 0x7f8bf4151220 2023-08-11T15:52:33.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf9eac640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf4151600 0x7f8bf40cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:33.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.614+0000 7f8bf8eaa640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf4151220 0x7f8bf40cbff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8be402b8c0 tx=0x7f8be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.615+0000 7f8be9ffb640 1 -- 172.21.15.27:0/3391376953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4032880 con 0x7f8bf4151220 2023-08-11T15:52:33.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.615+0000 7f8be9ffb640 1 -- 172.21.15.27:0/3391376953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be40329e0 con 0x7f8bf4151220 2023-08-11T15:52:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.615+0000 7f8be9ffb640 1 -- 172.21.15.27:0/3391376953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4032d90 con 0x7f8bf4151220 2023-08-11T15:52:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.615+0000 7f8bfb90f640 1 -- 172.21.15.27:0/3391376953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf4157610 con 0x7f8bf4151220 2023-08-11T15:52:33.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.615+0000 7f8bfb90f640 1 -- 172.21.15.27:0/3391376953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf4150100 con 0x7f8bf4151220 2023-08-11T15:52:33.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.616+0000 7f8bfb90f640 1 -- 172.21.15.27:0/3391376953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf404e420 con 0x7f8bf4151220 2023-08-11T15:52:33.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.617+0000 7f8be9ffb640 1 -- 172.21.15.27:0/3391376953 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8be404e450 con 0x7f8bf4151220 2023-08-11T15:52:33.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.619+0000 7f8be9ffb640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8bd806a050 0x7f8bd806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.619+0000 7f8be9ffb640 1 -- 172.21.15.27:0/3391376953 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8be40aab90 con 0x7f8bf4151220 2023-08-11T15:52:33.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.620+0000 7f8bf96ab640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8bd806a050 0x7f8bd806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.621+0000 7f8bf96ab640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8bd806a050 0x7f8bd806c510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0006fd0 tx=0x7f8bf0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.624+0000 7f8be9ffb640 1 -- 172.21.15.27:0/3391376953 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f8be4075fa0 con 0x7f8bf4151220 2023-08-11T15:52:33.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.818+0000 7f8bfb90f640 1 -- 172.21.15.27:0/3391376953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8bf406d380 con 0x7f8bf4151220 2023-08-11T15:52:33.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.819+0000 7f8be9ffb640 1 -- 172.21.15.27:0/3391376953 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8be40795c0 con 0x7f8bf4151220 2023-08-11T15:52:33.822 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:33.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.821+0000 7f8bfb90f640 1 -- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8bd806a050 msgr2=0x7f8bd806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8bd806a050 0x7f8bd806c510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0006fd0 tx=0x7f8bf0008040 comp rx=0 tx=0).stop 2023-08-11T15:52:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 -- 172.21.15.27:0/3391376953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf4151220 msgr2=0x7f8bf40cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf4151220 0x7f8bf40cbff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8be402b8c0 tx=0x7f8be40046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 -- 172.21.15.27:0/3391376953 shutdown_connections 2023-08-11T15:52:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8bd806a050 0x7f8bd806c510 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf4151600 0x7f8bf40cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf4151220 0x7f8bf40cbff0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 --2- 172.21.15.27:0/3391376953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8bf406bdf0 0x7f8bf414fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 -- 172.21.15.27:0/3391376953 >> 172.21.15.27:0/3391376953 conn(0x7f8bf40afbe0 msgr2=0x7f8bf40cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 -- 172.21.15.27:0/3391376953 shutdown_connections 2023-08-11T15:52:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:33.822+0000 7f8bfb90f640 1 -- 172.21.15.27:0/3391376953 wait complete. 2023-08-11T15:52:33.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.063+0000 7f7933fff640 1 Processor -- start 2023-08-11T15:52:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.063+0000 7f7933fff640 1 -- start start 2023-08-11T15:52:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.065+0000 7f7933fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f793415c430 0x7f79340b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.065+0000 7f7933fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79340d1d30 0x7f79340d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.065+0000 7f7933fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79340cec30 0x7f79340cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.065+0000 7f7933fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79340d1910 con 0x7f79340d1d30 2023-08-11T15:52:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.065+0000 7f7933fff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79340b8280 con 0x7f793415c430 2023-08-11T15:52:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.065+0000 7f7933fff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79340d0aa0 con 0x7f79340cec30 2023-08-11T15:52:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.065+0000 7f79327fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79340d1d30 0x7f79340d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f79327fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79340d1d30 0x7f79340d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45032/0 (socket says 172.21.15.27:45032) 2023-08-11T15:52:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f79327fc640 1 -- 172.21.15.27:0/2703546362 learned_addr learned my addr 172.21.15.27:0/2703546362 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:34.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f7932ffd640 1 --2- 172.21.15.27:0/2703546362 >> v2:172.21.15.119:3300/0 conn(0x7f793415c430 0x7f79340b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f79337fe640 1 --2- 172.21.15.27:0/2703546362 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79340cec30 0x7f79340cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f79327fc640 1 -- 172.21.15.27:0/2703546362 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79340cec30 msgr2=0x7f79340cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:34.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f79327fc640 1 --2- 172.21.15.27:0/2703546362 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79340cec30 0x7f79340cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f79327fc640 1 -- 172.21.15.27:0/2703546362 >> v2:172.21.15.119:3300/0 conn(0x7f793415c430 msgr2=0x7f79340b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:34.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f79327fc640 1 --2- 172.21.15.27:0/2703546362 >> v2:172.21.15.119:3300/0 conn(0x7f793415c430 0x7f79340b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f79327fc640 1 -- 172.21.15.27:0/2703546362 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340d0c90 con 0x7f79340d1d30 2023-08-11T15:52:34.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f7932ffd640 1 --2- 172.21.15.27:0/2703546362 >> v2:172.21.15.119:3300/0 conn(0x7f793415c430 0x7f79340b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:34.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.066+0000 7f79337fe640 1 --2- 172.21.15.27:0/2703546362 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79340cec30 0x7f79340cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:34.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.067+0000 7f79327fc640 1 --2- 172.21.15.27:0/2703546362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 0x7f79340d0560 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7934071f60 tx=0x7f792802b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be8b332a4f9d49 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:34.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.068+0000 7f7931ffb640 1 -- 172.21.15.27:0/2703546362 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79280317f0 con 0x7f79340d1d30 2023-08-11T15:52:34.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.068+0000 7f7931ffb640 1 -- 172.21.15.27:0/2703546362 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7928031950 con 0x7f79340d1d30 2023-08-11T15:52:34.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.068+0000 7f7931ffb640 1 -- 172.21.15.27:0/2703546362 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928031ce0 con 0x7f79340d1d30 2023-08-11T15:52:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.068+0000 7f7933fff640 1 -- 172.21.15.27:0/2703546362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 msgr2=0x7f79340d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.068+0000 7f7933fff640 1 --2- 172.21.15.27:0/2703546362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 0x7f79340d0560 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7934071f60 tx=0x7f792802b890 comp rx=0 tx=0).stop 2023-08-11T15:52:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.068+0000 7f7933fff640 1 -- 172.21.15.27:0/2703546362 shutdown_connections 2023-08-11T15:52:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.068+0000 7f7933fff640 1 --2- 172.21.15.27:0/2703546362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 0x7f79340d0560 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.068+0000 7f7933fff640 1 --2- 172.21.15.27:0/2703546362 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79340cec30 0x7f79340cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.068+0000 7f7933fff640 1 --2- 172.21.15.27:0/2703546362 >> v2:172.21.15.119:3300/0 conn(0x7f793415c430 0x7f79340b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.068+0000 7f7933fff640 1 -- 172.21.15.27:0/2703546362 >> 172.21.15.27:0/2703546362 conn(0x7f793406a2b0 msgr2=0x7f793406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.069+0000 7f7933fff640 1 -- 172.21.15.27:0/2703546362 shutdown_connections 2023-08-11T15:52:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.069+0000 7f7933fff640 1 -- 172.21.15.27:0/2703546362 wait complete. 2023-08-11T15:52:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.070+0000 7f7933fff640 1 Processor -- start 2023-08-11T15:52:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.070+0000 7f7933fff640 1 -- start start 2023-08-11T15:52:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.071+0000 7f7933fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79340cec30 0x7f79340cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.071+0000 7f7933fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 0x7f793414a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.071+0000 7f7933fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f793415c430 0x7f793414c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.071+0000 7f7933fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793414b420 con 0x7f79340d1d30 2023-08-11T15:52:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.071+0000 7f7933fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793414b700 con 0x7f79340cec30 2023-08-11T15:52:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.071+0000 7f7933fff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793414b9e0 con 0x7f793415c430 2023-08-11T15:52:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.072+0000 7f79327fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 0x7f793414a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.072+0000 7f79327fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 0x7f793414a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45036/0 (socket says 172.21.15.27:45036) 2023-08-11T15:52:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.072+0000 7f79327fc640 1 -- 172.21.15.27:0/3423389814 learned_addr learned my addr 172.21.15.27:0/3423389814 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.072+0000 7f7932ffd640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79340cec30 0x7f79340cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.072+0000 7f79337fe640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f793415c430 0x7f793414c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.072+0000 7f79327fc640 1 -- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f793415c430 msgr2=0x7f793414c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.072+0000 7f79327fc640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f793415c430 0x7f793414c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.072+0000 7f79327fc640 1 -- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79340cec30 msgr2=0x7f79340cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.072+0000 7f79327fc640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79340cec30 0x7f79340cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.072+0000 7f79327fc640 1 -- 172.21.15.27:0/3423389814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79280094d0 con 0x7f79340d1d30 2023-08-11T15:52:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.073+0000 7f79327fc640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 0x7f793414a510 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7928000c00 tx=0x7f79280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.073+0000 7f791bfff640 1 -- 172.21.15.27:0/3423389814 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928032910 con 0x7f79340d1d30 2023-08-11T15:52:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.073+0000 7f791bfff640 1 -- 172.21.15.27:0/3423389814 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7928032a70 con 0x7f79340d1d30 2023-08-11T15:52:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.073+0000 7f791bfff640 1 -- 172.21.15.27:0/3423389814 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928032e20 con 0x7f79340d1d30 2023-08-11T15:52:34.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.073+0000 7f7933fff640 1 -- 172.21.15.27:0/3423389814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793414bcc0 con 0x7f79340d1d30 2023-08-11T15:52:34.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.073+0000 7f7933fff640 1 -- 172.21.15.27:0/3423389814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79340cbec0 con 0x7f79340d1d30 2023-08-11T15:52:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.075+0000 7f7933fff640 1 -- 172.21.15.27:0/3423389814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793414c1e0 con 0x7f79340d1d30 2023-08-11T15:52:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.076+0000 7f791bfff640 1 -- 172.21.15.27:0/3423389814 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f792804e450 con 0x7f79340d1d30 2023-08-11T15:52:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.076+0000 7f791bfff640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792006a050 0x7f792006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.077+0000 7f791bfff640 1 -- 172.21.15.27:0/3423389814 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f79280aada0 con 0x7f79340d1d30 2023-08-11T15:52:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.077+0000 7f7932ffd640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792006a050 0x7f792006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.078+0000 7f7932ffd640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792006a050 0x7f792006c510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f792400a990 tx=0x7f7924008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.081+0000 7f791bfff640 1 -- 172.21.15.27:0/3423389814 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f793414c1e0 con 0x7f79340d1d30 2023-08-11T15:52:34.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.273+0000 7f7933fff640 1 -- 172.21.15.27:0/3423389814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f793414c1e0 con 0x7f79340d1d30 2023-08-11T15:52:34.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.274+0000 7f791bfff640 1 -- 172.21.15.27:0/3423389814 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f793414c1e0 con 0x7f79340d1d30 2023-08-11T15:52:34.277 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:34.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.276+0000 7f7933fff640 1 -- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792006a050 msgr2=0x7f792006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.276+0000 7f7933fff640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792006a050 0x7f792006c510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f792400a990 tx=0x7f7924008040 comp rx=0 tx=0).stop 2023-08-11T15:52:34.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.276+0000 7f7933fff640 1 -- 172.21.15.27:0/3423389814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 msgr2=0x7f793414a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.276+0000 7f7933fff640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 0x7f793414a510 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7928000c00 tx=0x7f79280046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.277+0000 7f7933fff640 1 -- 172.21.15.27:0/3423389814 shutdown_connections 2023-08-11T15:52:34.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.277+0000 7f7933fff640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792006a050 0x7f792006c510 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.277+0000 7f7933fff640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f793415c430 0x7f793414c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.277+0000 7f7933fff640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79340d1d30 0x7f793414a510 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.277+0000 7f7933fff640 1 --2- 172.21.15.27:0/3423389814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79340cec30 0x7f79340cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.277+0000 7f7933fff640 1 -- 172.21.15.27:0/3423389814 >> 172.21.15.27:0/3423389814 conn(0x7f793406a2b0 msgr2=0x7f79340ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:34.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.277+0000 7f7933fff640 1 -- 172.21.15.27:0/3423389814 shutdown_connections 2023-08-11T15:52:34.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.277+0000 7f7933fff640 1 -- 172.21.15.27:0/3423389814 wait complete. 2023-08-11T15:52:34.298 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:34.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.533+0000 7ff2b359e640 1 Processor -- start 2023-08-11T15:52:34.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.533+0000 7ff2b359e640 1 -- start start 2023-08-11T15:52:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.534+0000 7ff2b359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff2b415c430 0x7ff2b40b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.535+0000 7ff2b359e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 0x7ff2b40d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.535+0000 7ff2b359e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b40d0c90 con 0x7ff2b40d1d30 2023-08-11T15:52:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.535+0000 7ff2b359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b40d1910 con 0x7ff2b415c430 2023-08-11T15:52:34.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.535+0000 7ff2b359e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40b8280 con 0x7ff2b40cf340 2023-08-11T15:52:34.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.535+0000 7ff2b259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff2b415c430 0x7ff2b40b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.535+0000 7ff2b259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff2b415c430 0x7ff2b40b7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45040/0 (socket says 172.21.15.27:45040) 2023-08-11T15:52:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.535+0000 7ff2b259c640 1 -- 172.21.15.27:0/1283841964 learned_addr learned my addr 172.21.15.27:0/1283841964 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.535+0000 7ff2b2d9d640 1 --2- 172.21.15.27:0/1283841964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 0x7ff2b40d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.536+0000 7ff2b259c640 1 -- 172.21.15.27:0/1283841964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 msgr2=0x7ff2b40d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.536+0000 7ff2b259c640 1 --2- 172.21.15.27:0/1283841964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 0x7ff2b40d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.536+0000 7ff2b259c640 1 -- 172.21.15.27:0/1283841964 >> v1:172.21.15.119:6789/0 conn(0x7ff2b40d1d30 legacy=0x7ff2b40cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.536+0000 7ff2b259c640 1 -- 172.21.15.27:0/1283841964 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b40d0f70 con 0x7ff2b415c430 2023-08-11T15:52:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.537+0000 7ff2b259c640 1 --2- 172.21.15.27:0/1283841964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b415c430 0x7ff2b40b7d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009870 tx=0x7ff2a402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba0514f25f5ab12 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.537+0000 7ff2b1d9b640 1 -- 172.21.15.27:0/1283841964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40317c0 con 0x7ff2b415c430 2023-08-11T15:52:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.537+0000 7ff2b1d9b640 1 -- 172.21.15.27:0/1283841964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a4031920 con 0x7ff2b415c430 2023-08-11T15:52:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.537+0000 7ff2b1d9b640 1 -- 172.21.15.27:0/1283841964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4031cb0 con 0x7ff2b415c430 2023-08-11T15:52:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.538+0000 7ff2b359e640 1 -- 172.21.15.27:0/1283841964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b415c430 msgr2=0x7ff2b40b7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.538+0000 7ff2b359e640 1 --2- 172.21.15.27:0/1283841964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b415c430 0x7ff2b40b7d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009870 tx=0x7ff2a402b860 comp rx=0 tx=0).stop 2023-08-11T15:52:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.538+0000 7ff2b359e640 1 -- 172.21.15.27:0/1283841964 shutdown_connections 2023-08-11T15:52:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.538+0000 7ff2b359e640 1 --2- 172.21.15.27:0/1283841964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 0x7ff2b40d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.538+0000 7ff2b359e640 1 --2- 172.21.15.27:0/1283841964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b415c430 0x7ff2b40b7d40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.538+0000 7ff2b359e640 1 -- 172.21.15.27:0/1283841964 >> 172.21.15.27:0/1283841964 conn(0x7ff2b406a2b0 msgr2=0x7ff2b406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.538+0000 7ff2b359e640 1 -- 172.21.15.27:0/1283841964 shutdown_connections 2023-08-11T15:52:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.538+0000 7ff2b359e640 1 -- 172.21.15.27:0/1283841964 wait complete. 2023-08-11T15:52:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.540+0000 7ff2b359e640 1 Processor -- start 2023-08-11T15:52:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.540+0000 7ff2b359e640 1 -- start start 2023-08-11T15:52:34.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.541+0000 7ff2b359e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 0x7ff2b40b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.541+0000 7ff2b359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b40d1d30 0x7ff2b41f1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.541+0000 7ff2b359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2b415c430 0x7ff2b40b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.541+0000 7ff2b359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b41f1990 con 0x7ff2b40d1d30 2023-08-11T15:52:34.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.541+0000 7ff2b359e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40c9390 con 0x7ff2b415c430 2023-08-11T15:52:34.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.541+0000 7ff2b359e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40cbf50 con 0x7ff2b40cf340 2023-08-11T15:52:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.541+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b40d1d30 0x7ff2b41f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.541+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b40d1d30 0x7ff2b41f1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45044/0 (socket says 172.21.15.27:45044) 2023-08-11T15:52:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.541+0000 7ff2abfff640 1 -- 172.21.15.27:0/2839171639 learned_addr learned my addr 172.21.15.27:0/2839171639 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2b259c640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 0x7ff2b40b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2abfff640 1 -- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 msgr2=0x7ff2b40b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2abfff640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 0x7ff2b40b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2abfff640 1 -- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2b415c430 msgr2=0x7ff2b40b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2abfff640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2b415c430 0x7ff2b40b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2abfff640 1 -- 172.21.15.27:0/2839171639 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40094d0 con 0x7ff2b40d1d30 2023-08-11T15:52:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2b259c640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 0x7ff2b40b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2abfff640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b40d1d30 0x7ff2b41f1450 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00c430 tx=0x7ff29c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2aaffd640 1 -- 172.21.15.27:0/2839171639 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c011790 con 0x7ff2b40d1d30 2023-08-11T15:52:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2aaffd640 1 -- 172.21.15.27:0/2839171639 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c0118f0 con 0x7ff2b40d1d30 2023-08-11T15:52:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2aaffd640 1 -- 172.21.15.27:0/2839171639 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c011ce0 con 0x7ff2b40d1d30 2023-08-11T15:52:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2b359e640 1 -- 172.21.15.27:0/2839171639 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b40c9630 con 0x7ff2b40d1d30 2023-08-11T15:52:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.542+0000 7ff2b359e640 1 -- 172.21.15.27:0/2839171639 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b40cc230 con 0x7ff2b40d1d30 2023-08-11T15:52:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.544+0000 7ff2b359e640 1 -- 172.21.15.27:0/2839171639 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b404e420 con 0x7ff2b40d1d30 2023-08-11T15:52:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.545+0000 7ff2aaffd640 1 -- 172.21.15.27:0/2839171639 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff29c01c460 con 0x7ff2b40d1d30 2023-08-11T15:52:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.545+0000 7ff2aaffd640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff29406a080 0x7ff29406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.546+0000 7ff2aaffd640 1 -- 172.21.15.27:0/2839171639 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff29c0887f0 con 0x7ff2b40d1d30 2023-08-11T15:52:34.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.546+0000 7ff2b259c640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff29406a080 0x7ff29406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.547+0000 7ff2b259c640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff29406a080 0x7ff29406c540 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4036000 tx=0x7ff2a403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:34.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.550+0000 7ff2aaffd640 1 -- 172.21.15.27:0/2839171639 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ff29c053ba0 con 0x7ff2b40d1d30 2023-08-11T15:52:34.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.740+0000 7ff2b359e640 1 -- 172.21.15.27:0/2839171639 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff2b40b5e20 con 0x7ff2b40d1d30 2023-08-11T15:52:34.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.741+0000 7ff2aaffd640 1 -- 172.21.15.27:0/2839171639 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff29c0571c0 con 0x7ff2b40d1d30 2023-08-11T15:52:34.744 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 -- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff29406a080 msgr2=0x7ff29406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff29406a080 0x7ff29406c540 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4036000 tx=0x7ff2a403d040 comp rx=0 tx=0).stop 2023-08-11T15:52:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 -- 172.21.15.27:0/2839171639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b40d1d30 msgr2=0x7ff2b41f1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b40d1d30 0x7ff2b41f1450 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00c430 tx=0x7ff29c00c8f0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 -- 172.21.15.27:0/2839171639 shutdown_connections 2023-08-11T15:52:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff29406a080 0x7ff29406c540 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2b415c430 0x7ff2b40b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2b40d1d30 0x7ff2b41f1450 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 --2- 172.21.15.27:0/2839171639 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2b40cf340 0x7ff2b40b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 -- 172.21.15.27:0/2839171639 >> 172.21.15.27:0/2839171639 conn(0x7ff2b406a2b0 msgr2=0x7ff2b40c6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:34.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 -- 172.21.15.27:0/2839171639 shutdown_connections 2023-08-11T15:52:34.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.744+0000 7ff2b359e640 1 -- 172.21.15.27:0/2839171639 wait complete. 2023-08-11T15:52:34.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:52:34.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.992+0000 7f09cd56d640 1 Processor -- start 2023-08-11T15:52:34.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.993+0000 7f09cd56d640 1 -- start start 2023-08-11T15:52:34.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09cd56d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09c806ca00 0x7f09c80aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09cd56d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f09c80c3250 0x7f09c80acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:34.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09cd56d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c80ab970 con 0x7f09c80c2e70 2023-08-11T15:52:34.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09cd56d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c80abc50 con 0x7f09c80c3250 2023-08-11T15:52:34.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09cd56d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c80abf30 con 0x7f09c806ca00 2023-08-11T15:52:34.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09ccd6c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f09c80c3250 0x7f09c80acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09ccd6c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f09c80c3250 0x7f09c80acf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45050/0 (socket says 172.21.15.27:45050) 2023-08-11T15:52:34.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09ccd6c640 1 -- 172.21.15.27:0/2730754762 learned_addr learned my addr 172.21.15.27:0/2730754762 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:34.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09bffff640 1 --2- 172.21.15.27:0/2730754762 >> v2:172.21.15.119:3300/0 conn(0x7f09c806ca00 0x7f09c80aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:34.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09ccd6c640 1 -- 172.21.15.27:0/2730754762 >> v2:172.21.15.119:3300/0 conn(0x7f09c806ca00 msgr2=0x7f09c80aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.995+0000 7f09ccd6c640 1 --2- 172.21.15.27:0/2730754762 >> v2:172.21.15.119:3300/0 conn(0x7f09c806ca00 0x7f09c80aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.996+0000 7f09ccd6c640 1 -- 172.21.15.27:0/2730754762 >> v1:172.21.15.27:6789/0 conn(0x7f09c80c2e70 legacy=0x7f09c8160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.996+0000 7f09ccd6c640 1 -- 172.21.15.27:0/2730754762 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c80ac210 con 0x7f09c80c3250 2023-08-11T15:52:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.997+0000 7f09ccd6c640 1 --2- 172.21.15.27:0/2730754762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c80c3250 0x7f09c80acf50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f09c80ad510 tx=0x7f09c002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989def4a6f1cb320 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.997+0000 7f09c77fe640 1 -- 172.21.15.27:0/2730754762 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c002cb10 con 0x7f09c80c3250 2023-08-11T15:52:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.997+0000 7f09c77fe640 1 -- 172.21.15.27:0/2730754762 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c002cc70 con 0x7f09c80c3250 2023-08-11T15:52:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.997+0000 7f09c77fe640 1 -- 172.21.15.27:0/2730754762 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00343f0 con 0x7f09c80c3250 2023-08-11T15:52:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.998+0000 7f09cd56d640 1 -- 172.21.15.27:0/2730754762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c80c3250 msgr2=0x7f09c80acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.998+0000 7f09cd56d640 1 --2- 172.21.15.27:0/2730754762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c80c3250 0x7f09c80acf50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f09c80ad510 tx=0x7f09c002bf70 comp rx=0 tx=0).stop 2023-08-11T15:52:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.998+0000 7f09cd56d640 1 -- 172.21.15.27:0/2730754762 shutdown_connections 2023-08-11T15:52:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.998+0000 7f09cd56d640 1 --2- 172.21.15.27:0/2730754762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c80c3250 0x7f09c80acf50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.998+0000 7f09cd56d640 1 --2- 172.21.15.27:0/2730754762 >> v2:172.21.15.119:3300/0 conn(0x7f09c806ca00 0x7f09c80aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.998+0000 7f09cd56d640 1 -- 172.21.15.27:0/2730754762 >> 172.21.15.27:0/2730754762 conn(0x7f09c8069800 msgr2=0x7f09c8069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.998+0000 7f09cd56d640 1 -- 172.21.15.27:0/2730754762 shutdown_connections 2023-08-11T15:52:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:34.998+0000 7f09cd56d640 1 -- 172.21.15.27:0/2730754762 wait complete. 2023-08-11T15:52:35.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.000+0000 7f09cd56d640 1 Processor -- start 2023-08-11T15:52:35.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.000+0000 7f09cd56d640 1 -- start start 2023-08-11T15:52:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.001+0000 7f09cd56d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c806ca00 0x7f09c814f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.001+0000 7f09cd56d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09c80c2e70 0x7f09c80b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.001+0000 7f09cd56d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c80c3250 0x7f09c80b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.001+0000 7f09cd56d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c80b2270 con 0x7f09c806ca00 2023-08-11T15:52:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.001+0000 7f09cd56d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c80b2550 con 0x7f09c80c3250 2023-08-11T15:52:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.001+0000 7f09cd56d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c80b2830 con 0x7f09c80c2e70 2023-08-11T15:52:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.001+0000 7f09c7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c806ca00 0x7f09c814f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.001+0000 7f09c7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c806ca00 0x7f09c814f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45060/0 (socket says 172.21.15.27:45060) 2023-08-11T15:52:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.001+0000 7f09c7fff640 1 -- 172.21.15.27:0/5506033 learned_addr learned my addr 172.21.15.27:0/5506033 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.002+0000 7f09ccd6c640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c80c3250 0x7f09c80b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.002+0000 7f09bffff640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09c80c2e70 0x7f09c80b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.002+0000 7f09c7fff640 1 -- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09c80c2e70 msgr2=0x7f09c80b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.002+0000 7f09c7fff640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09c80c2e70 0x7f09c80b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.002+0000 7f09c7fff640 1 -- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c80c3250 msgr2=0x7f09c80b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.002+0000 7f09c7fff640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c80c3250 0x7f09c80b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.002+0000 7f09c7fff640 1 -- 172.21.15.27:0/5506033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0009b30 con 0x7f09c806ca00 2023-08-11T15:52:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.002+0000 7f09ccd6c640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c80c3250 0x7f09c80b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.002+0000 7f09c7fff640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c806ca00 0x7f09c814f310 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f09b800c450 tx=0x7f09b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.003+0000 7f09c57fa640 1 -- 172.21.15.27:0/5506033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b80052d0 con 0x7f09c806ca00 2023-08-11T15:52:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.003+0000 7f09cd56d640 1 -- 172.21.15.27:0/5506033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c80b2b10 con 0x7f09c806ca00 2023-08-11T15:52:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.003+0000 7f09c57fa640 1 -- 172.21.15.27:0/5506033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b8005430 con 0x7f09c806ca00 2023-08-11T15:52:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.003+0000 7f09c57fa640 1 -- 172.21.15.27:0/5506033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8012870 con 0x7f09c806ca00 2023-08-11T15:52:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.003+0000 7f09cd56d640 1 -- 172.21.15.27:0/5506033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c80b1f90 con 0x7f09c806ca00 2023-08-11T15:52:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.004+0000 7f09cd56d640 1 -- 172.21.15.27:0/5506033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c8066b30 con 0x7f09c806ca00 2023-08-11T15:52:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.005+0000 7f09c57fa640 1 -- 172.21.15.27:0/5506033 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f09b80129d0 con 0x7f09c806ca00 2023-08-11T15:52:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.006+0000 7f09c57fa640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09a806a080 0x7f09a806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.006+0000 7f09c57fa640 1 -- 172.21.15.27:0/5506033 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f09b80883e0 con 0x7f09c806ca00 2023-08-11T15:52:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.007+0000 7f09bffff640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09a806a080 0x7f09a806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.008+0000 7f09bffff640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09a806a080 0x7f09a806c540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09c815a6e0 tx=0x7f09b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.011+0000 7f09c57fa640 1 -- 172.21.15.27:0/5506033 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f09b8053790 con 0x7f09c806ca00 2023-08-11T15:52:35.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.218+0000 7f09cd56d640 1 -- 172.21.15.27:0/5506033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f09c80b3030 con 0x7f09c806ca00 2023-08-11T15:52:35.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.218+0000 7f09c57fa640 1 -- 172.21.15.27:0/5506033 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f09c80b3030 con 0x7f09c806ca00 2023-08-11T15:52:35.222 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:52:35.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 -- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09a806a080 msgr2=0x7f09a806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09a806a080 0x7f09a806c540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09c815a6e0 tx=0x7f09b0008040 comp rx=0 tx=0).stop 2023-08-11T15:52:35.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 -- 172.21.15.27:0/5506033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c806ca00 msgr2=0x7f09c814f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c806ca00 0x7f09c814f310 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f09b800c450 tx=0x7f09b800c910 comp rx=0 tx=0).stop 2023-08-11T15:52:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 -- 172.21.15.27:0/5506033 shutdown_connections 2023-08-11T15:52:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09a806a080 0x7f09a806c540 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09c80c2e70 0x7f09c80b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c80c3250 0x7f09c80b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 --2- 172.21.15.27:0/5506033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c806ca00 0x7f09c814f310 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 -- 172.21.15.27:0/5506033 >> 172.21.15.27:0/5506033 conn(0x7f09c8069800 msgr2=0x7f09c8159610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:35.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.221+0000 7f09cd56d640 1 -- 172.21.15.27:0/5506033 shutdown_connections 2023-08-11T15:52:35.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.222+0000 7f09cd56d640 1 -- 172.21.15.27:0/5506033 wait complete. 2023-08-11T15:52:35.241 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-08-11T15:52:35.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:52:35.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-08-11T15:52:35.344 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:52:35.344 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:52:35.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:52:35.345 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-11T15:52:35.400 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:52:35.400 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:35.457 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:52:35.457 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:35.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:35.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.749+0000 7f40a2c71640 1 Processor -- start 2023-08-11T15:52:35.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.749+0000 7f40a2c71640 1 -- start start 2023-08-11T15:52:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.751+0000 7f40a2c71640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c15a020 0x7f409c15e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:35.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.751+0000 7f40a2c71640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f409c0c9600 0x7f409c071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:35.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.751+0000 7f40a2c71640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c0c85f0 con 0x7f409c06f5d0 2023-08-11T15:52:35.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.751+0000 7f40a2c71640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c15eae0 con 0x7f409c0c9600 2023-08-11T15:52:35.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.751+0000 7f40a2c71640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c071980 con 0x7f409c15a020 2023-08-11T15:52:35.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.752+0000 7f40a146e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f409c0c9600 0x7f409c071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:35.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.752+0000 7f40a1c6f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c15a020 0x7f409c15e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:35.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.752+0000 7f40a1c6f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c15a020 0x7f409c15e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:46560/0 (socket says 172.21.15.27:46560) 2023-08-11T15:52:35.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.752+0000 7f40a1c6f640 1 -- 172.21.15.27:0/2974304364 learned_addr learned my addr 172.21.15.27:0/2974304364 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:35.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.753+0000 7f40a1c6f640 1 -- 172.21.15.27:0/2974304364 >> v2:172.21.15.119:3300/0 conn(0x7f409c0c9600 msgr2=0x7f409c071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:35.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.753+0000 7f40a1c6f640 1 --2- 172.21.15.27:0/2974304364 >> v2:172.21.15.119:3300/0 conn(0x7f409c0c9600 0x7f409c071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.753+0000 7f40a1c6f640 1 -- 172.21.15.27:0/2974304364 >> v1:172.21.15.119:6789/0 conn(0x7f409c06f5d0 legacy=0x7f409c06b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.753+0000 7f40a1c6f640 1 -- 172.21.15.27:0/2974304364 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c0c8840 con 0x7f409c15a020 2023-08-11T15:52:35.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.753+0000 7f40a146e640 1 --2- 172.21.15.27:0/2974304364 >> v2:172.21.15.119:3300/0 conn(0x7f409c0c9600 0x7f409c071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:35.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.754+0000 7f40a1c6f640 1 --2- 172.21.15.27:0/2974304364 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c15a020 0x7f409c15e5a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4098009870 tx=0x7f409802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9161717ee3412c61 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:35.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.754+0000 7f40a0c6d640 1 -- 172.21.15.27:0/2974304364 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098031640 con 0x7f409c15a020 2023-08-11T15:52:35.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.754+0000 7f40a0c6d640 1 -- 172.21.15.27:0/2974304364 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40980317a0 con 0x7f409c15a020 2023-08-11T15:52:35.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.754+0000 7f40a0c6d640 1 -- 172.21.15.27:0/2974304364 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098031b30 con 0x7f409c15a020 2023-08-11T15:52:35.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.755+0000 7f40a2c71640 1 -- 172.21.15.27:0/2974304364 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c15a020 msgr2=0x7f409c15e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.755+0000 7f40a2c71640 1 --2- 172.21.15.27:0/2974304364 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c15a020 0x7f409c15e5a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4098009870 tx=0x7f409802b8f0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.755+0000 7f40a2c71640 1 -- 172.21.15.27:0/2974304364 shutdown_connections 2023-08-11T15:52:35.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.755+0000 7f40a2c71640 1 --2- 172.21.15.27:0/2974304364 >> v2:172.21.15.119:3300/0 conn(0x7f409c0c9600 0x7f409c071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.755+0000 7f40a2c71640 1 --2- 172.21.15.27:0/2974304364 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c15a020 0x7f409c15e5a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.755+0000 7f40a2c71640 1 -- 172.21.15.27:0/2974304364 >> 172.21.15.27:0/2974304364 conn(0x7f409c0afb70 msgr2=0x7f409c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:35.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.755+0000 7f40a2c71640 1 -- 172.21.15.27:0/2974304364 shutdown_connections 2023-08-11T15:52:35.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.755+0000 7f40a2c71640 1 -- 172.21.15.27:0/2974304364 wait complete. 2023-08-11T15:52:35.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.758+0000 7f40a2c71640 1 Processor -- start 2023-08-11T15:52:35.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.758+0000 7f40a2c71640 1 -- start start 2023-08-11T15:52:35.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.759+0000 7f40a2c71640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c06f5d0 0x7f409c1f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:35.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.759+0000 7f40a2c71640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f409c0c9600 0x7f409c0cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:35.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.759+0000 7f40a2c71640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c15a020 0x7f409c14e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:35.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.759+0000 7f40a2c71640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0ac6c0 con 0x7f409c0c9600 2023-08-11T15:52:35.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.759+0000 7f40a2c71640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c14dd70 con 0x7f409c15a020 2023-08-11T15:52:35.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.759+0000 7f40a2c71640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0aba80 con 0x7f409c06f5d0 2023-08-11T15:52:35.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.759+0000 7f40a146e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f409c0c9600 0x7f409c0cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:35.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.759+0000 7f40a146e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f409c0c9600 0x7f409c0cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45066/0 (socket says 172.21.15.27:45066) 2023-08-11T15:52:35.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.760+0000 7f40a146e640 1 -- 172.21.15.27:0/796285585 learned_addr learned my addr 172.21.15.27:0/796285585 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:35.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.760+0000 7f40a1c6f640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c06f5d0 0x7f409c1f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:35.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.760+0000 7f40a2470640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c15a020 0x7f409c14e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:35.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.760+0000 7f40a146e640 1 -- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c06f5d0 msgr2=0x7f409c1f1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.760+0000 7f40a146e640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c06f5d0 0x7f409c1f1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.760+0000 7f40a146e640 1 -- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c15a020 msgr2=0x7f409c14e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.760+0000 7f40a146e640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c15a020 0x7f409c14e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.760+0000 7f40a146e640 1 -- 172.21.15.27:0/796285585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c009400 con 0x7f409c0c9600 2023-08-11T15:52:35.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.760+0000 7f40a2470640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c15a020 0x7f409c14e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:35.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.760+0000 7f40a146e640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f409c0c9600 0x7f409c0cd5b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f408c00ebb0 tx=0x7f408c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:35.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.761+0000 7f40927fc640 1 -- 172.21.15.27:0/796285585 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c00eeb0 con 0x7f409c0c9600 2023-08-11T15:52:35.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.761+0000 7f40927fc640 1 -- 172.21.15.27:0/796285585 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c00cc30 con 0x7f409c0c9600 2023-08-11T15:52:35.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.761+0000 7f40927fc640 1 -- 172.21.15.27:0/796285585 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c013470 con 0x7f409c0c9600 2023-08-11T15:52:35.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.761+0000 7f40a2c71640 1 -- 172.21.15.27:0/796285585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40980094d0 con 0x7f409c0c9600 2023-08-11T15:52:35.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.761+0000 7f40a2c71640 1 -- 172.21.15.27:0/796285585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c0ac000 con 0x7f409c0c9600 2023-08-11T15:52:35.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.762+0000 7f40a2c71640 1 -- 172.21.15.27:0/796285585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c0ac930 con 0x7f409c0c9600 2023-08-11T15:52:35.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.763+0000 7f40927fc640 1 -- 172.21.15.27:0/796285585 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f408c0135d0 con 0x7f409c0c9600 2023-08-11T15:52:35.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.764+0000 7f40927fc640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f408006a050 0x7f408006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:35.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.764+0000 7f40927fc640 1 -- 172.21.15.27:0/796285585 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f408c088c40 con 0x7f409c0c9600 2023-08-11T15:52:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.765+0000 7f40a1c6f640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f408006a050 0x7f408006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.766+0000 7f40a1c6f640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f408006a050 0x7f408006c510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4098036000 tx=0x7f409803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.769+0000 7f40927fc640 1 -- 172.21.15.27:0/796285585 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f408c053ff0 con 0x7f409c0c9600 2023-08-11T15:52:35.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.962+0000 7f40a2c71640 1 -- 172.21.15.27:0/796285585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f409c04e420 con 0x7f409c0c9600 2023-08-11T15:52:35.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.964+0000 7f40927fc640 1 -- 172.21.15.27:0/796285585 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f408c057610 con 0x7f409c0c9600 2023-08-11T15:52:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:35.967 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:35.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 -- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f408006a050 msgr2=0x7f408006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f408006a050 0x7f408006c510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4098036000 tx=0x7f409803c040 comp rx=0 tx=0).stop 2023-08-11T15:52:35.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 -- 172.21.15.27:0/796285585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f409c0c9600 msgr2=0x7f409c0cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:35.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f409c0c9600 0x7f409c0cd5b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f408c00ebb0 tx=0x7f408c00c520 comp rx=0 tx=0).stop 2023-08-11T15:52:35.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 -- 172.21.15.27:0/796285585 shutdown_connections 2023-08-11T15:52:35.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f408006a050 0x7f408006c510 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c15a020 0x7f409c14e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f409c0c9600 0x7f409c0cd5b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 --2- 172.21.15.27:0/796285585 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f409c06f5d0 0x7f409c1f1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:35.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 -- 172.21.15.27:0/796285585 >> 172.21.15.27:0/796285585 conn(0x7f409c0afb70 msgr2=0x7f409c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:35.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.967+0000 7f40a2c71640 1 -- 172.21.15.27:0/796285585 shutdown_connections 2023-08-11T15:52:35.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:35.968+0000 7f40a2c71640 1 -- 172.21.15.27:0/796285585 wait complete. 2023-08-11T15:52:35.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:35.993 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:36.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.227+0000 7fe445cd3640 1 Processor -- start 2023-08-11T15:52:36.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.227+0000 7fe445cd3640 1 -- start start 2023-08-11T15:52:36.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.229+0000 7fe445cd3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe44015c760 0x7fe4400d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.229+0000 7fe445cd3640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44006bc10 con 0x7fe44006b670 2023-08-11T15:52:36.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.229+0000 7fe445cd3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44006bef0 con 0x7fe4400d2a20 2023-08-11T15:52:36.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.229+0000 7fe445cd3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44006c790 con 0x7fe44015c760 2023-08-11T15:52:36.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.229+0000 7fe43effd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe44006b670 0x7fe4400d2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57894/0 (socket says 172.21.15.27:57894) 2023-08-11T15:52:36.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.229+0000 7fe43effd640 1 -- 172.21.15.27:0/1427305912 learned_addr learned my addr 172.21.15.27:0/1427305912 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:36.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.230+0000 7fe43f7fe640 1 --2- 172.21.15.27:0/1427305912 >> v2:172.21.15.119:3300/0 conn(0x7fe44015c760 0x7fe4400d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.230+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771479144 0 0) 0x7fe44006bc10 con 0x7fe44006b670 2023-08-11T15:52:36.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.230+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe428003520 con 0x7fe44006b670 2023-08-11T15:52:36.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.230+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1416409967 0 0) 0x7fe428003520 con 0x7fe44006b670 2023-08-11T15:52:36.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 >> v2:172.21.15.119:3300/0 conn(0x7fe44015c760 msgr2=0x7fe4400d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:36.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 --2- 172.21.15.27:0/1427305912 >> v2:172.21.15.119:3300/0 conn(0x7fe44015c760 0x7fe4400d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 >> v1:172.21.15.119:6789/0 conn(0x7fe4400d2a20 legacy=0x7fe4400af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44006c1d0 con 0x7fe44006b670 2023-08-11T15:52:36.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fe434001670 con 0x7fe44006b670 2023-08-11T15:52:36.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 >> v1:172.21.15.27:6789/0 conn(0x7fe44006b670 legacy=0x7fe4400d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe428003970 0x7fe428003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4280047c0 0x7fe428004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe428005180 0x7fe42800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42800bff0 con 0x7fe4280047c0 2023-08-11T15:52:36.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42800c160 con 0x7fe428005180 2023-08-11T15:52:36.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42800c320 con 0x7fe428003970 2023-08-11T15:52:36.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43effd640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4280047c0 0x7fe428004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43f7fe640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe428003970 0x7fe428003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43ffff640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe428005180 0x7fe42800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43effd640 1 -- 172.21.15.27:0/1427305912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe428003970 msgr2=0x7fe428003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.231+0000 7fe43effd640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe428003970 0x7fe428003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.232+0000 7fe43effd640 1 -- 172.21.15.27:0/1427305912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe428005180 msgr2=0x7fe42800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.232+0000 7fe43effd640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe428005180 0x7fe42800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.232+0000 7fe43effd640 1 -- 172.21.15.27:0/1427305912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe428003520 con 0x7fe4280047c0 2023-08-11T15:52:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.232+0000 7fe43ffff640 1 -- 172.21.15.27:0/1427305912 reap_dead start 2023-08-11T15:52:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.232+0000 7fe43f7fe640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe428003970 0x7fe428003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.233+0000 7fe43effd640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4280047c0 0x7fe428004c40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c240 tx=0x7fe4280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e103cd297d194371 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:36.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.233+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434005ad0 con 0x7fe4280047c0 2023-08-11T15:52:36.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.233+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe434005c30 con 0x7fe4280047c0 2023-08-11T15:52:36.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.233+0000 7fe43e7fc640 1 -- 172.21.15.27:0/1427305912 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43402caf0 con 0x7fe4280047c0 2023-08-11T15:52:36.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.233+0000 7fe445cd3640 1 -- 172.21.15.27:0/1427305912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4280047c0 msgr2=0x7fe428004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.233+0000 7fe445cd3640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4280047c0 0x7fe428004c40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c240 tx=0x7fe4280057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.234+0000 7fe445cd3640 1 -- 172.21.15.27:0/1427305912 shutdown_connections 2023-08-11T15:52:36.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.234+0000 7fe445cd3640 1 --2- 172.21.15.27:0/1427305912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4280047c0 0x7fe428004c40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.234+0000 7fe445cd3640 1 -- 172.21.15.27:0/1427305912 >> 172.21.15.27:0/1427305912 conn(0x7fe440089ff0 msgr2=0x7fe44009b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:36.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.234+0000 7fe445cd3640 1 -- 172.21.15.27:0/1427305912 shutdown_connections 2023-08-11T15:52:36.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.234+0000 7fe445cd3640 1 -- 172.21.15.27:0/1427305912 wait complete. 2023-08-11T15:52:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.236+0000 7fe445cd3640 1 Processor -- start 2023-08-11T15:52:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.236+0000 7fe445cd3640 1 -- start start 2023-08-11T15:52:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.237+0000 7fe445cd3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4280047c0 0x7fe4401579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.237+0000 7fe445cd3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4400b7e30 0x7fe4400c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.237+0000 7fe445cd3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4400b82b0 0x7fe4400c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.237+0000 7fe445cd3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4400c7210 con 0x7fe4400b82b0 2023-08-11T15:52:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.237+0000 7fe445cd3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4400ce490 con 0x7fe4400b7e30 2023-08-11T15:52:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.237+0000 7fe445cd3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4400c9120 con 0x7fe4280047c0 2023-08-11T15:52:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.237+0000 7fe43ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4400b82b0 0x7fe4400c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.237+0000 7fe43ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4400b82b0 0x7fe4400c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45092/0 (socket says 172.21.15.27:45092) 2023-08-11T15:52:36.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.237+0000 7fe43ffff640 1 -- 172.21.15.27:0/1566823689 learned_addr learned my addr 172.21.15.27:0/1566823689 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:36.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.237+0000 7fe43f7fe640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4280047c0 0x7fe4401579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe43ffff640 1 -- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4280047c0 msgr2=0x7fe4401579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe43ffff640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4280047c0 0x7fe4401579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe43ffff640 1 -- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4400b7e30 msgr2=0x7fe4400c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe43effd640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4400b7e30 0x7fe4400c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe43ffff640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4400b7e30 0x7fe4400c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe43ffff640 1 -- 172.21.15.27:0/1566823689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4401f2c10 con 0x7fe4400b82b0 2023-08-11T15:52:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe43f7fe640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4280047c0 0x7fe4401579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe43ffff640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4400b82b0 0x7fe4400c8be0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe44008cba0 tx=0x7fe43800e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe427fff640 1 -- 172.21.15.27:0/1566823689 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438006e20 con 0x7fe4400b82b0 2023-08-11T15:52:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe427fff640 1 -- 172.21.15.27:0/1566823689 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe438003970 con 0x7fe4400b82b0 2023-08-11T15:52:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe445cd3640 1 -- 172.21.15.27:0/1566823689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4400c69b0 con 0x7fe4400b82b0 2023-08-11T15:52:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.238+0000 7fe427fff640 1 -- 172.21.15.27:0/1566823689 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438003d20 con 0x7fe4400b82b0 2023-08-11T15:52:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.239+0000 7fe445cd3640 1 -- 172.21.15.27:0/1566823689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4400c6f30 con 0x7fe4400b82b0 2023-08-11T15:52:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.240+0000 7fe445cd3640 1 -- 172.21.15.27:0/1566823689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4400ce5d0 con 0x7fe4400b82b0 2023-08-11T15:52:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.240+0000 7fe427fff640 1 -- 172.21.15.27:0/1566823689 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe438005110 con 0x7fe4400b82b0 2023-08-11T15:52:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.241+0000 7fe427fff640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42c06a080 0x7fe42c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.242+0000 7fe427fff640 1 -- 172.21.15.27:0/1566823689 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe4380880c0 con 0x7fe4400b82b0 2023-08-11T15:52:36.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.242+0000 7fe43f7fe640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42c06a080 0x7fe42c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.243+0000 7fe43f7fe640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42c06a080 0x7fe42c06c540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe4400b2f30 tx=0x7fe430008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:36.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.246+0000 7fe427fff640 1 -- 172.21.15.27:0/1566823689 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe4400ce5d0 con 0x7fe4400b82b0 2023-08-11T15:52:36.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.448+0000 7fe445cd3640 1 -- 172.21.15.27:0/1566823689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4400ce5d0 con 0x7fe4400b82b0 2023-08-11T15:52:36.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.449+0000 7fe427fff640 1 -- 172.21.15.27:0/1566823689 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe438019020 con 0x7fe4400b82b0 2023-08-11T15:52:36.452 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 -- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42c06a080 msgr2=0x7fe42c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42c06a080 0x7fe42c06c540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe4400b2f30 tx=0x7fe430008040 comp rx=0 tx=0).stop 2023-08-11T15:52:36.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 -- 172.21.15.27:0/1566823689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4400b82b0 msgr2=0x7fe4400c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4400b82b0 0x7fe4400c8be0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe44008cba0 tx=0x7fe43800e930 comp rx=0 tx=0).stop 2023-08-11T15:52:36.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 -- 172.21.15.27:0/1566823689 shutdown_connections 2023-08-11T15:52:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe42c06a080 0x7fe42c06c540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4400b82b0 0x7fe4400c8be0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4400b7e30 0x7fe4400c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 --2- 172.21.15.27:0/1566823689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4280047c0 0x7fe4401579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 -- 172.21.15.27:0/1566823689 >> 172.21.15.27:0/1566823689 conn(0x7fe440089ff0 msgr2=0x7fe44015b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:36.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 -- 172.21.15.27:0/1566823689 shutdown_connections 2023-08-11T15:52:36.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.452+0000 7fe445cd3640 1 -- 172.21.15.27:0/1566823689 wait complete. 2023-08-11T15:52:36.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:36.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.710+0000 7f14e79fd640 1 Processor -- start 2023-08-11T15:52:36.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.710+0000 7f14e79fd640 1 -- start start 2023-08-11T15:52:36.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.711+0000 7f14e79fd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14e015c760 0x7f14e00d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.711+0000 7f14e79fd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14e00d2a20 0x7f14e00af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.711+0000 7f14e79fd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e006bc10 con 0x7f14e006b670 2023-08-11T15:52:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.711+0000 7f14e79fd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e006c790 con 0x7f14e00d2a20 2023-08-11T15:52:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.711+0000 7f14e79fd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e006bef0 con 0x7f14e015c760 2023-08-11T15:52:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.711+0000 7f14e5f9a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14e00d2a20 0x7f14e00af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.712+0000 7f14e5f9a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14e00d2a20 0x7f14e00af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45100/0 (socket says 172.21.15.27:45100) 2023-08-11T15:52:36.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.712+0000 7f14e5f9a640 1 -- 172.21.15.27:0/747519482 learned_addr learned my addr 172.21.15.27:0/747519482 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:36.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.712+0000 7f14e5799640 1 --2- 172.21.15.27:0/747519482 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14e015c760 0x7f14e00d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.712+0000 7f14e5f9a640 1 -- 172.21.15.27:0/747519482 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14e015c760 msgr2=0x7f14e00d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:36.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.712+0000 7f14e5f9a640 1 --2- 172.21.15.27:0/747519482 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14e015c760 0x7f14e00d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.712+0000 7f14e5f9a640 1 -- 172.21.15.27:0/747519482 >> v1:172.21.15.119:6789/0 conn(0x7f14e006b670 legacy=0x7f14e00d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.712+0000 7f14e5f9a640 1 -- 172.21.15.27:0/747519482 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e006c1d0 con 0x7f14e00d2a20 2023-08-11T15:52:36.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.713+0000 7f14e5f9a640 1 --2- 172.21.15.27:0/747519482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00d2a20 0x7f14e00af610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009e50 tx=0x7f14d802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d277df36307ccb server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:36.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.714+0000 7f14d7fff640 1 -- 172.21.15.27:0/747519482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d802ca60 con 0x7f14e00d2a20 2023-08-11T15:52:36.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.714+0000 7f14d7fff640 1 -- 172.21.15.27:0/747519482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d802cbc0 con 0x7f14e00d2a20 2023-08-11T15:52:36.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.714+0000 7f14d7fff640 1 -- 172.21.15.27:0/747519482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d80343f0 con 0x7f14e00d2a20 2023-08-11T15:52:36.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.714+0000 7f14e79fd640 1 -- 172.21.15.27:0/747519482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00d2a20 msgr2=0x7f14e00af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.714+0000 7f14e79fd640 1 --2- 172.21.15.27:0/747519482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00d2a20 0x7f14e00af610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009e50 tx=0x7f14d802bec0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.714+0000 7f14e79fd640 1 -- 172.21.15.27:0/747519482 shutdown_connections 2023-08-11T15:52:36.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.714+0000 7f14e79fd640 1 --2- 172.21.15.27:0/747519482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00d2a20 0x7f14e00af610 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.714+0000 7f14e79fd640 1 --2- 172.21.15.27:0/747519482 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14e015c760 0x7f14e00d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.714+0000 7f14e79fd640 1 -- 172.21.15.27:0/747519482 >> 172.21.15.27:0/747519482 conn(0x7f14e0089ff0 msgr2=0x7f14e009b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:36.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.715+0000 7f14e79fd640 1 -- 172.21.15.27:0/747519482 shutdown_connections 2023-08-11T15:52:36.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.715+0000 7f14e79fd640 1 -- 172.21.15.27:0/747519482 wait complete. 2023-08-11T15:52:36.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.716+0000 7f14e79fd640 1 Processor -- start 2023-08-11T15:52:36.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.716+0000 7f14e79fd640 1 -- start start 2023-08-11T15:52:36.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.717+0000 7f14e79fd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14e006b670 0x7f14e015ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.717+0000 7f14e79fd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00d2a20 0x7f14e01580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.717+0000 7f14e79fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015c760 0x7f14e015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.717+0000 7f14e79fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00cecf0 con 0x7f14e015c760 2023-08-11T15:52:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.717+0000 7f14e79fd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00ce120 con 0x7f14e00d2a20 2023-08-11T15:52:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.717+0000 7f14e79fd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00ce3c0 con 0x7f14e006b670 2023-08-11T15:52:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e5f9a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015c760 0x7f14e015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e5f9a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015c760 0x7f14e015a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45114/0 (socket says 172.21.15.27:45114) 2023-08-11T15:52:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e5f9a640 1 -- 172.21.15.27:0/2876914427 learned_addr learned my addr 172.21.15.27:0/2876914427 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e5799640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14e006b670 0x7f14e015ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e4f98640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00d2a20 0x7f14e01580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e5f9a640 1 -- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14e006b670 msgr2=0x7f14e015ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e5f9a640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14e006b670 0x7f14e015ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e5f9a640 1 -- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00d2a20 msgr2=0x7f14e01580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e5f9a640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00d2a20 0x7f14e01580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e5f9a640 1 -- 172.21.15.27:0/2876914427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d8009b30 con 0x7f14e015c760 2023-08-11T15:52:36.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.718+0000 7f14e4f98640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00d2a20 0x7f14e01580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:36.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.719+0000 7f14e5f9a640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015c760 0x7f14e015a060 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009e50 tx=0x7f14d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:36.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.719+0000 7f14d5ffb640 1 -- 172.21.15.27:0/2876914427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d8009510 con 0x7f14e015c760 2023-08-11T15:52:36.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.719+0000 7f14d5ffb640 1 -- 172.21.15.27:0/2876914427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d803e450 con 0x7f14e015c760 2023-08-11T15:52:36.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.719+0000 7f14d5ffb640 1 -- 172.21.15.27:0/2876914427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d803e800 con 0x7f14e015c760 2023-08-11T15:52:36.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.719+0000 7f14e79fd640 1 -- 172.21.15.27:0/2876914427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e00ce6a0 con 0x7f14e015c760 2023-08-11T15:52:36.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.719+0000 7f14e79fd640 1 -- 172.21.15.27:0/2876914427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e01585f0 con 0x7f14e015c760 2023-08-11T15:52:36.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.720+0000 7f14e79fd640 1 -- 172.21.15.27:0/2876914427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e00ceea0 con 0x7f14e015c760 2023-08-11T15:52:36.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.721+0000 7f14d5ffb640 1 -- 172.21.15.27:0/2876914427 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f14d803e960 con 0x7f14e015c760 2023-08-11T15:52:36.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.722+0000 7f14d5ffb640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f14c406a080 0x7f14c406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:36.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.722+0000 7f14d5ffb640 1 -- 172.21.15.27:0/2876914427 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f14d80ab6a0 con 0x7f14e015c760 2023-08-11T15:52:36.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.723+0000 7f14e5799640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f14c406a080 0x7f14c406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:36.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.724+0000 7f14e5799640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f14c406a080 0x7f14c406c540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006fd0 tx=0x7f14dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:36.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.727+0000 7f14d5ffb640 1 -- 172.21.15.27:0/2876914427 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f14e00ceea0 con 0x7f14e015c760 2023-08-11T15:52:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.926+0000 7f14e79fd640 1 -- 172.21.15.27:0/2876914427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f14e004e420 con 0x7f14e015c760 2023-08-11T15:52:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.927+0000 7f14d5ffb640 1 -- 172.21.15.27:0/2876914427 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f14d8076ae0 con 0x7f14e015c760 2023-08-11T15:52:36.929 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:36.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 -- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f14c406a080 msgr2=0x7f14c406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f14c406a080 0x7f14c406c540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006fd0 tx=0x7f14dc008040 comp rx=0 tx=0).stop 2023-08-11T15:52:36.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 -- 172.21.15.27:0/2876914427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015c760 msgr2=0x7f14e015a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015c760 0x7f14e015a060 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009e50 tx=0x7f14d8004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 -- 172.21.15.27:0/2876914427 shutdown_connections 2023-08-11T15:52:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f14c406a080 0x7f14c406c540 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015c760 0x7f14e015a060 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e00d2a20 0x7f14e01580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 --2- 172.21.15.27:0/2876914427 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14e006b670 0x7f14e015ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 -- 172.21.15.27:0/2876914427 >> 172.21.15.27:0/2876914427 conn(0x7f14e0089ff0 msgr2=0x7f14e00af510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 -- 172.21.15.27:0/2876914427 shutdown_connections 2023-08-11T15:52:36.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:36.930+0000 7f14e79fd640 1 -- 172.21.15.27:0/2876914427 wait complete. 2023-08-11T15:52:36.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:37.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.191+0000 7f96a6ac2640 1 Processor -- start 2023-08-11T15:52:37.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.192+0000 7f96a6ac2640 1 -- start start 2023-08-11T15:52:37.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.193+0000 7f96a6ac2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96a00c86c0 0x7f96a00afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.193+0000 7f96a6ac2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a00b0410 con 0x7f96a00afff0 2023-08-11T15:52:37.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.194+0000 7f96a6ac2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a00c8b30 con 0x7f96a015c3f0 2023-08-11T15:52:37.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.194+0000 7f96a6ac2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a00ca810 con 0x7f96a00c86c0 2023-08-11T15:52:37.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.194+0000 7f96a62c1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f96a00afff0 0x7f96a006ba90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57896/0 (socket says 172.21.15.27:57896) 2023-08-11T15:52:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.194+0000 7f96a62c1640 1 -- 172.21.15.27:0/2413529743 learned_addr learned my addr 172.21.15.27:0/2413529743 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.194+0000 7f96a52bf640 1 --2- 172.21.15.27:0/2413529743 >> v2:172.21.15.119:3300/0 conn(0x7f96a00c86c0 0x7f96a00afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.194+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285670415 0 0) 0x7f96a00b0410 con 0x7f96a00afff0 2023-08-11T15:52:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.194+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f968c003520 con 0x7f96a00afff0 2023-08-11T15:52:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2163665570 0 0) 0x7f968c003520 con 0x7f96a00afff0 2023-08-11T15:52:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 >> v2:172.21.15.119:3300/0 conn(0x7f96a00c86c0 msgr2=0x7f96a00afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 --2- 172.21.15.27:0/2413529743 >> v2:172.21.15.119:3300/0 conn(0x7f96a00c86c0 0x7f96a00afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 >> v1:172.21.15.119:6789/0 conn(0x7f96a015c3f0 legacy=0x7f96a00cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a01600f0 con 0x7f96a00afff0 2023-08-11T15:52:37.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f96980019f0 con 0x7f96a00afff0 2023-08-11T15:52:37.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 >> v1:172.21.15.27:6789/0 conn(0x7f96a00afff0 legacy=0x7f96a006ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f968c003970 0x7f968c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f968c0047c0 0x7f968c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f968c005180 0x7f968c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00ca810 con 0x7f968c003970 2023-08-11T15:52:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c00c030 con 0x7f968c005180 2023-08-11T15:52:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c00c1f0 con 0x7f968c0047c0 2023-08-11T15:52:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.195+0000 7f96a5ac0640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f968c003970 0x7f968c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.196+0000 7f96a62c1640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f968c005180 0x7f968c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.196+0000 7f96a52bf640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f968c0047c0 0x7f968c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.196+0000 7f96a5ac0640 1 -- 172.21.15.27:0/2413529743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f968c0047c0 msgr2=0x7f968c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.196+0000 7f96a5ac0640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f968c0047c0 0x7f968c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.196+0000 7f96a5ac0640 1 -- 172.21.15.27:0/2413529743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f968c005180 msgr2=0x7f968c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.196+0000 7f96a5ac0640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f968c005180 0x7f968c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.196+0000 7f96a5ac0640 1 -- 172.21.15.27:0/2413529743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c003520 con 0x7f968c003970 2023-08-11T15:52:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.196+0000 7f96a62c1640 1 -- 172.21.15.27:0/2413529743 reap_dead start 2023-08-11T15:52:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.197+0000 7f96a5ac0640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f968c003970 0x7f968c003d90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f969c001750 tx=0x7f968c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca05c8d5db74ef93 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.197+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0052d0 con 0x7f968c003970 2023-08-11T15:52:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.197+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c02e050 con 0x7f968c003970 2023-08-11T15:52:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.197+0000 7f96a4abe640 1 -- 172.21.15.27:0/2413529743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c003830 con 0x7f968c003970 2023-08-11T15:52:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.198+0000 7f96a6ac2640 1 -- 172.21.15.27:0/2413529743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f968c003970 msgr2=0x7f968c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.198+0000 7f96a6ac2640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f968c003970 0x7f968c003d90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f969c001750 tx=0x7f968c0057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.198+0000 7f96a6ac2640 1 -- 172.21.15.27:0/2413529743 shutdown_connections 2023-08-11T15:52:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.198+0000 7f96a6ac2640 1 --2- 172.21.15.27:0/2413529743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f968c003970 0x7f968c003d90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.198+0000 7f96a6ac2640 1 -- 172.21.15.27:0/2413529743 >> 172.21.15.27:0/2413529743 conn(0x7f96a00caab0 msgr2=0x7f96a00cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.198+0000 7f96a6ac2640 1 -- 172.21.15.27:0/2413529743 shutdown_connections 2023-08-11T15:52:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.198+0000 7f96a6ac2640 1 -- 172.21.15.27:0/2413529743 wait complete. 2023-08-11T15:52:37.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.200+0000 7f96a6ac2640 1 Processor -- start 2023-08-11T15:52:37.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.200+0000 7f96a6ac2640 1 -- start start 2023-08-11T15:52:37.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.201+0000 7f96a6ac2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f968c003970 0x7f96a00c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.201+0000 7f96a6ac2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a00af2f0 0x7f96a00af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.201+0000 7f96a6ac2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96a00c6140 0x7f96a00c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.201+0000 7f96a6ac2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00afcb0 con 0x7f96a00af2f0 2023-08-11T15:52:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.201+0000 7f96a6ac2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00c7f80 con 0x7f968c003970 2023-08-11T15:52:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.201+0000 7f96a6ac2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00c6ee0 con 0x7f96a00c6140 2023-08-11T15:52:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.201+0000 7f96a52bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a00af2f0 0x7f96a00af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.202+0000 7f96a52bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a00af2f0 0x7f96a00af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45134/0 (socket says 172.21.15.27:45134) 2023-08-11T15:52:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.202+0000 7f96a52bf640 1 -- 172.21.15.27:0/1139793102 learned_addr learned my addr 172.21.15.27:0/1139793102 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.202+0000 7f96a5ac0640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f968c003970 0x7f96a00c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.202+0000 7f96a52bf640 1 -- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96a00c6140 msgr2=0x7f96a00c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.202+0000 7f96a62c1640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96a00c6140 0x7f96a00c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.202+0000 7f96a52bf640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96a00c6140 0x7f96a00c65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.202+0000 7f96a52bf640 1 -- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f968c003970 msgr2=0x7f96a00c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.202+0000 7f96a52bf640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f968c003970 0x7f96a00c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.202+0000 7f96a52bf640 1 -- 172.21.15.27:0/1139793102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a00c71c0 con 0x7f96a00af2f0 2023-08-11T15:52:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.202+0000 7f96a52bf640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a00af2f0 0x7f96a00af770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f96900114e0 tx=0x7f96900119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.203+0000 7f96967fc640 1 -- 172.21.15.27:0/1139793102 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690015870 con 0x7f96a00af2f0 2023-08-11T15:52:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.203+0000 7f96967fc640 1 -- 172.21.15.27:0/1139793102 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96900159d0 con 0x7f96a00af2f0 2023-08-11T15:52:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.203+0000 7f96967fc640 1 -- 172.21.15.27:0/1139793102 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690015d40 con 0x7f96a00af2f0 2023-08-11T15:52:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.203+0000 7f96a6ac2640 1 -- 172.21.15.27:0/1139793102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a00c74a0 con 0x7f96a00af2f0 2023-08-11T15:52:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.203+0000 7f96a6ac2640 1 -- 172.21.15.27:0/1139793102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a00c7a60 con 0x7f96a00af2f0 2023-08-11T15:52:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.204+0000 7f96a6ac2640 1 -- 172.21.15.27:0/1139793102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a004e420 con 0x7f96a00af2f0 2023-08-11T15:52:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.205+0000 7f96967fc640 1 -- 172.21.15.27:0/1139793102 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9690013400 con 0x7f96a00af2f0 2023-08-11T15:52:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.206+0000 7f96967fc640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f968406a080 0x7f968406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.206+0000 7f96967fc640 1 -- 172.21.15.27:0/1139793102 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9690088cd0 con 0x7f96a00af2f0 2023-08-11T15:52:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.207+0000 7f96a5ac0640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f968406a080 0x7f968406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.208+0000 7f96a5ac0640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f968406a080 0x7f968406c540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f969c033000 tx=0x7f969c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:37.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.211+0000 7f96967fc640 1 -- 172.21.15.27:0/1139793102 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9690053ff0 con 0x7f96a00af2f0 2023-08-11T15:52:37.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.404+0000 7f96a6ac2640 1 -- 172.21.15.27:0/1139793102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96a006ceb0 con 0x7f96a00af2f0 2023-08-11T15:52:37.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.405+0000 7f96967fc640 1 -- 172.21.15.27:0/1139793102 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9690057610 con 0x7f96a00af2f0 2023-08-11T15:52:37.408 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:37.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 -- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f968406a080 msgr2=0x7f968406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f968406a080 0x7f968406c540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f969c033000 tx=0x7f969c009030 comp rx=0 tx=0).stop 2023-08-11T15:52:37.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 -- 172.21.15.27:0/1139793102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a00af2f0 msgr2=0x7f96a00af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a00af2f0 0x7f96a00af770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f96900114e0 tx=0x7f96900119a0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 -- 172.21.15.27:0/1139793102 shutdown_connections 2023-08-11T15:52:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f968406a080 0x7f968406c540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96a00c6140 0x7f96a00c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a00af2f0 0x7f96a00af770 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 --2- 172.21.15.27:0/1139793102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f968c003970 0x7f96a00c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 -- 172.21.15.27:0/1139793102 >> 172.21.15.27:0/1139793102 conn(0x7f96a00caab0 msgr2=0x7f96a006fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 -- 172.21.15.27:0/1139793102 shutdown_connections 2023-08-11T15:52:37.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.408+0000 7f96a6ac2640 1 -- 172.21.15.27:0/1139793102 wait complete. 2023-08-11T15:52:37.429 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.658+0000 7f34a6e39640 1 Processor -- start 2023-08-11T15:52:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.658+0000 7f34a6e39640 1 -- start start 2023-08-11T15:52:37.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.659+0000 7f34a6e39640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f34a015c430 0x7f34a00b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.659+0000 7f34a6e39640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f34a00cf340 0x7f34a00d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.659+0000 7f34a6e39640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a00d0c90 con 0x7f34a00d1d30 2023-08-11T15:52:37.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.659+0000 7f34a6e39640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a00d1910 con 0x7f34a00cf340 2023-08-11T15:52:37.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.659+0000 7f34a6e39640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a00b8280 con 0x7f34a015c430 2023-08-11T15:52:37.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.659+0000 7f34a5636640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f34a00d1d30 0x7f34a00cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57906/0 (socket says 172.21.15.27:57906) 2023-08-11T15:52:37.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.659+0000 7f34a5636640 1 -- 172.21.15.27:0/2935394943 learned_addr learned my addr 172.21.15.27:0/2935394943 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:37.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.659+0000 7f34a6638640 1 --2- 172.21.15.27:0/2935394943 >> v2:172.21.15.27:3300/0 conn(0x7f34a00cf340 0x7f34a00d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.660+0000 7f34a5e37640 1 --2- 172.21.15.27:0/2935394943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f34a015c430 0x7f34a00b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.660+0000 7f34a4e35640 1 -- 172.21.15.27:0/2935394943 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314111371 0 0) 0x7f34a00d0c90 con 0x7f34a00d1d30 2023-08-11T15:52:37.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.660+0000 7f34a4e35640 1 -- 172.21.15.27:0/2935394943 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f348c003520 con 0x7f34a00d1d30 2023-08-11T15:52:37.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.660+0000 7f34a6638640 1 -- 172.21.15.27:0/2935394943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f34a015c430 msgr2=0x7f34a00b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:37.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.660+0000 7f34a6638640 1 --2- 172.21.15.27:0/2935394943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f34a015c430 0x7f34a00b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.660+0000 7f34a6638640 1 -- 172.21.15.27:0/2935394943 >> v1:172.21.15.27:6789/0 conn(0x7f34a00d1d30 legacy=0x7f34a00cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.660+0000 7f34a6638640 1 -- 172.21.15.27:0/2935394943 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a00d0f70 con 0x7f34a00cf340 2023-08-11T15:52:37.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.661+0000 7f34a6638640 1 --2- 172.21.15.27:0/2935394943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a00cf340 0x7f34a00d0560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3498009d90 tx=0x7f349802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10920f67881abd2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:37.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.662+0000 7f34a4e35640 1 -- 172.21.15.27:0/2935394943 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349802cbc0 con 0x7f34a00cf340 2023-08-11T15:52:37.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.662+0000 7f34a4e35640 1 -- 172.21.15.27:0/2935394943 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349802cd20 con 0x7f34a00cf340 2023-08-11T15:52:37.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.662+0000 7f34a4e35640 1 -- 172.21.15.27:0/2935394943 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498034640 con 0x7f34a00cf340 2023-08-11T15:52:37.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.662+0000 7f34a6e39640 1 -- 172.21.15.27:0/2935394943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a00cf340 msgr2=0x7f34a00d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.662+0000 7f34a6e39640 1 --2- 172.21.15.27:0/2935394943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a00cf340 0x7f34a00d0560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3498009d90 tx=0x7f349802be10 comp rx=0 tx=0).stop 2023-08-11T15:52:37.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.662+0000 7f34a6e39640 1 -- 172.21.15.27:0/2935394943 shutdown_connections 2023-08-11T15:52:37.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.662+0000 7f34a6e39640 1 --2- 172.21.15.27:0/2935394943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a00cf340 0x7f34a00d0560 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.662+0000 7f34a6e39640 1 --2- 172.21.15.27:0/2935394943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f34a015c430 0x7f34a00b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.662+0000 7f34a6e39640 1 -- 172.21.15.27:0/2935394943 >> 172.21.15.27:0/2935394943 conn(0x7f34a006a2b0 msgr2=0x7f34a006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:37.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.663+0000 7f34a6e39640 1 -- 172.21.15.27:0/2935394943 shutdown_connections 2023-08-11T15:52:37.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.663+0000 7f34a6e39640 1 -- 172.21.15.27:0/2935394943 wait complete. 2023-08-11T15:52:37.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.664+0000 7f34a6e39640 1 Processor -- start 2023-08-11T15:52:37.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.664+0000 7f34a6e39640 1 -- start start 2023-08-11T15:52:37.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.665+0000 7f34a6e39640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f34a00cf340 0x7f34a01f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.665+0000 7f34a6e39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a00d1d30 0x7f34a015e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.665+0000 7f34a6e39640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a015c430 0x7f34a00cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.665+0000 7f34a6e39640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a015d560 con 0x7f34a015c430 2023-08-11T15:52:37.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.665+0000 7f34a6e39640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a015d800 con 0x7f34a00d1d30 2023-08-11T15:52:37.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.665+0000 7f34a6e39640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a015dae0 con 0x7f34a00cf340 2023-08-11T15:52:37.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.666+0000 7f34a6638640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a015c430 0x7f34a00cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.666+0000 7f34a6638640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a015c430 0x7f34a00cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45142/0 (socket says 172.21.15.27:45142) 2023-08-11T15:52:37.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.666+0000 7f34a6638640 1 -- 172.21.15.27:0/1353652619 learned_addr learned my addr 172.21.15.27:0/1353652619 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:37.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.666+0000 7f34a5636640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a00d1d30 0x7f34a015e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.666+0000 7f34a5e37640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f34a00cf340 0x7f34a01f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.666+0000 7f34a6638640 1 -- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f34a00cf340 msgr2=0x7f34a01f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.666+0000 7f34a6638640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f34a00cf340 0x7f34a01f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.666+0000 7f34a6638640 1 -- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a00d1d30 msgr2=0x7f34a015e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.666+0000 7f34a6638640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a00d1d30 0x7f34a015e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.666+0000 7f34a6638640 1 -- 172.21.15.27:0/1353652619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3498009a70 con 0x7f34a015c430 2023-08-11T15:52:37.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.667+0000 7f34a6638640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a015c430 0x7f34a00cc590 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3498009eb0 tx=0x7f3498004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.667+0000 7f34967fc640 1 -- 172.21.15.27:0/1353652619 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349803e610 con 0x7f34a015c430 2023-08-11T15:52:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.667+0000 7f34967fc640 1 -- 172.21.15.27:0/1353652619 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349802ce90 con 0x7f34a015c430 2023-08-11T15:52:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.667+0000 7f34967fc640 1 -- 172.21.15.27:0/1353652619 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349803e9f0 con 0x7f34a015c430 2023-08-11T15:52:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.667+0000 7f34a6e39640 1 -- 172.21.15.27:0/1353652619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a015ddc0 con 0x7f34a015c430 2023-08-11T15:52:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.667+0000 7f34a6e39640 1 -- 172.21.15.27:0/1353652619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a015fa40 con 0x7f34a015c430 2023-08-11T15:52:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.668+0000 7f34a6e39640 1 -- 172.21.15.27:0/1353652619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a015e2e0 con 0x7f34a015c430 2023-08-11T15:52:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.669+0000 7f34967fc640 1 -- 172.21.15.27:0/1353652619 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f349803b050 con 0x7f34a015c430 2023-08-11T15:52:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.670+0000 7f34967fc640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f348406a050 0x7f348406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.670+0000 7f34967fc640 1 -- 172.21.15.27:0/1353652619 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f34980aa940 con 0x7f34a015c430 2023-08-11T15:52:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.671+0000 7f34a5e37640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f348406a050 0x7f348406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.672+0000 7f34a5e37640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f348406a050 0x7f348406c510 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f349c009a30 tx=0x7f349c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.675+0000 7f34967fc640 1 -- 172.21.15.27:0/1353652619 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f34a015e2e0 con 0x7f34a015c430 2023-08-11T15:52:37.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.877+0000 7f34a6e39640 1 -- 172.21.15.27:0/1353652619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34a015e2e0 con 0x7f34a015c430 2023-08-11T15:52:37.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.878+0000 7f34967fc640 1 -- 172.21.15.27:0/1353652619 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34a015e2e0 con 0x7f34a015c430 2023-08-11T15:52:37.880 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:37.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.880+0000 7f34a6e39640 1 -- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f348406a050 msgr2=0x7f348406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.880+0000 7f34a6e39640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f348406a050 0x7f348406c510 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f349c009a30 tx=0x7f349c008040 comp rx=0 tx=0).stop 2023-08-11T15:52:37.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.880+0000 7f34a6e39640 1 -- 172.21.15.27:0/1353652619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a015c430 msgr2=0x7f34a00cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:37.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.880+0000 7f34a6e39640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a015c430 0x7f34a00cc590 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3498009eb0 tx=0x7f3498004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.881+0000 7f34a6e39640 1 -- 172.21.15.27:0/1353652619 shutdown_connections 2023-08-11T15:52:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.881+0000 7f34a6e39640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f348406a050 0x7f348406c510 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.881+0000 7f34a6e39640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a015c430 0x7f34a00cc590 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.881+0000 7f34a6e39640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a00d1d30 0x7f34a015e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.881+0000 7f34a6e39640 1 --2- 172.21.15.27:0/1353652619 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f34a00cf340 0x7f34a01f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:37.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.881+0000 7f34a6e39640 1 -- 172.21.15.27:0/1353652619 >> 172.21.15.27:0/1353652619 conn(0x7f34a006a2b0 msgr2=0x7f34a015b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:37.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.881+0000 7f34a6e39640 1 -- 172.21.15.27:0/1353652619 shutdown_connections 2023-08-11T15:52:37.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:37.881+0000 7f34a6e39640 1 -- 172.21.15.27:0/1353652619 wait complete. 2023-08-11T15:52:37.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:38.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.132+0000 7f1046ba4640 1 Processor -- start 2023-08-11T15:52:38.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.132+0000 7f1046ba4640 1 -- start start 2023-08-11T15:52:38.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.134+0000 7f1046ba4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1040151600 0x7f104015df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.134+0000 7f1046ba4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104006c850 con 0x7f104015e4b0 2023-08-11T15:52:38.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.134+0000 7f1046ba4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104006bfc0 con 0x7f1040151220 2023-08-11T15:52:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.134+0000 7f1046ba4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104015e940 con 0x7f1040151600 2023-08-11T15:52:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.134+0000 7f1045141640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f104015e4b0 0x7f10400ca850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57916/0 (socket says 172.21.15.27:57916) 2023-08-11T15:52:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.134+0000 7f1045141640 1 -- 172.21.15.27:0/1323721953 learned_addr learned my addr 172.21.15.27:0/1323721953 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.134+0000 7f103ffff640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1040151600 0x7f104015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438239899 0 0) 0x7f104006c850 con 0x7f104015e4b0 2023-08-11T15:52:38.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1024003520 con 0x7f104015e4b0 2023-08-11T15:52:38.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634295908 0 0) 0x7f104006bfc0 con 0x7f1040151220 2023-08-11T15:52:38.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104006c850 con 0x7f1040151220 2023-08-11T15:52:38.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232466726 0 0) 0x7f1024003520 con 0x7f104015e4b0 2023-08-11T15:52:38.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1040151600 msgr2=0x7f104015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:38.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1040151600 0x7f104015df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 >> v1:172.21.15.119:6789/0 conn(0x7f1040151220 legacy=0x7f104006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104006c290 con 0x7f104015e4b0 2023-08-11T15:52:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f1038001690 con 0x7f104015e4b0 2023-08-11T15:52:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 >> v1:172.21.15.27:6789/0 conn(0x7f104015e4b0 legacy=0x7f10400ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 0x7f1024003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1040151600 0x7f10240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1024004d00 0x7f1024005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104015e940 con 0x7f1024003b30 2023-08-11T15:52:38.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102400bb70 con 0x7f1024004d00 2023-08-11T15:52:38.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.135+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102400bd30 con 0x7f1040151600 2023-08-11T15:52:38.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.136+0000 7f1044940640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 0x7f1024003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.136+0000 7f1045141640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1024004d00 0x7f1024005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.136+0000 7f103ffff640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1040151600 0x7f10240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.136+0000 7f1044940640 1 -- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1040151600 msgr2=0x7f10240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:38.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.136+0000 7f1044940640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1040151600 0x7f10240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.136+0000 7f1044940640 1 -- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1024004d00 msgr2=0x7f1024005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:38.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.136+0000 7f1044940640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1024004d00 0x7f1024005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.136+0000 7f1044940640 1 -- 172.21.15.27:0/1323721953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1024003520 con 0x7f1024003b30 2023-08-11T15:52:38.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.136+0000 7f103ffff640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1040151600 0x7f10240047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:38.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.137+0000 7f1044940640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 0x7f1024003f30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f10300018d0 tx=0x7f10240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58165fdf6988760f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:38.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.138+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10300052d0 con 0x7f1024003b30 2023-08-11T15:52:38.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.138+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103002e070 con 0x7f1024003b30 2023-08-11T15:52:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.138+0000 7f103f7fe640 1 -- 172.21.15.27:0/1323721953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10300035f0 con 0x7f1024003b30 2023-08-11T15:52:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.138+0000 7f1046ba4640 1 -- 172.21.15.27:0/1323721953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 msgr2=0x7f1024003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.138+0000 7f1046ba4640 1 --2- 172.21.15.27:0/1323721953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 0x7f1024003f30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f10300018d0 tx=0x7f10240057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.138+0000 7f1045141640 1 -- 172.21.15.27:0/1323721953 reap_dead start 2023-08-11T15:52:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.138+0000 7f1046ba4640 1 -- 172.21.15.27:0/1323721953 shutdown_connections 2023-08-11T15:52:38.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.138+0000 7f1046ba4640 1 -- 172.21.15.27:0/1323721953 >> 172.21.15.27:0/1323721953 conn(0x7f10400afbe0 msgr2=0x7f10400affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:38.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.139+0000 7f1046ba4640 1 -- 172.21.15.27:0/1323721953 shutdown_connections 2023-08-11T15:52:38.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.139+0000 7f1046ba4640 1 -- 172.21.15.27:0/1323721953 wait complete. 2023-08-11T15:52:38.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.140+0000 7f1046ba4640 1 Processor -- start 2023-08-11T15:52:38.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.141+0000 7f1046ba4640 1 -- start start 2023-08-11T15:52:38.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f1046ba4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 0x7f10400af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f1046ba4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10400ace50 0x7f10400ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f1046ba4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10400c3680 0x7f10400c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f1046ba4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400ac810 con 0x7f1024003b30 2023-08-11T15:52:38.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f1046ba4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400ac250 con 0x7f10400c3680 2023-08-11T15:52:38.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f1046ba4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10400ac530 con 0x7f10400ace50 2023-08-11T15:52:38.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f1044940640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 0x7f10400af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f1044940640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 0x7f10400af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45152/0 (socket says 172.21.15.27:45152) 2023-08-11T15:52:38.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f1044940640 1 -- 172.21.15.27:0/461127274 learned_addr learned my addr 172.21.15.27:0/461127274 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:38.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f103ffff640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10400ace50 0x7f10400ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.142+0000 7f1045141640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10400c3680 0x7f10400c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.143+0000 7f1044940640 1 -- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10400ace50 msgr2=0x7f10400ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.143+0000 7f1044940640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10400ace50 0x7f10400ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.143+0000 7f1044940640 1 -- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10400c3680 msgr2=0x7f10400c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.143+0000 7f1044940640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10400c3680 0x7f10400c3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.143+0000 7f1044940640 1 -- 172.21.15.27:0/461127274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10400b1d80 con 0x7f1024003b30 2023-08-11T15:52:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.143+0000 7f103ffff640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10400ace50 0x7f10400ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.143+0000 7f1044940640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 0x7f10400af090 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1030005040 tx=0x7f10300012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.143+0000 7f103d7fa640 1 -- 172.21.15.27:0/461127274 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030006e10 con 0x7f1024003b30 2023-08-11T15:52:38.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.143+0000 7f103d7fa640 1 -- 172.21.15.27:0/461127274 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1030007040 con 0x7f1024003b30 2023-08-11T15:52:38.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.143+0000 7f1046ba4640 1 -- 172.21.15.27:0/461127274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10400b2060 con 0x7f1024003b30 2023-08-11T15:52:38.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.144+0000 7f103d7fa640 1 -- 172.21.15.27:0/461127274 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10300073d0 con 0x7f1024003b30 2023-08-11T15:52:38.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.144+0000 7f1046ba4640 1 -- 172.21.15.27:0/461127274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10400af5d0 con 0x7f1024003b30 2023-08-11T15:52:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.145+0000 7f1046ba4640 1 -- 172.21.15.27:0/461127274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104004e420 con 0x7f1024003b30 2023-08-11T15:52:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.146+0000 7f103d7fa640 1 -- 172.21.15.27:0/461127274 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1030007530 con 0x7f1024003b30 2023-08-11T15:52:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.147+0000 7f103d7fa640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f102806a080 0x7f102806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.147+0000 7f103d7fa640 1 -- 172.21.15.27:0/461127274 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f10300a9f90 con 0x7f1024003b30 2023-08-11T15:52:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.147+0000 7f103ffff640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f102806a080 0x7f102806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.149+0000 7f103ffff640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f102806a080 0x7f102806c540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f10400c9430 tx=0x7f102c00d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:38.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.151+0000 7f103d7fa640 1 -- 172.21.15.27:0/461127274 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f10300dc7c0 con 0x7f1024003b30 2023-08-11T15:52:38.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.340+0000 7f1046ba4640 1 -- 172.21.15.27:0/461127274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f10400c8f50 con 0x7f1024003b30 2023-08-11T15:52:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.341+0000 7f103d7fa640 1 -- 172.21.15.27:0/461127274 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f103004b0a0 con 0x7f1024003b30 2023-08-11T15:52:38.343 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 -- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f102806a080 msgr2=0x7f102806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f102806a080 0x7f102806c540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f10400c9430 tx=0x7f102c00d040 comp rx=0 tx=0).stop 2023-08-11T15:52:38.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 -- 172.21.15.27:0/461127274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 msgr2=0x7f10400af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 0x7f10400af090 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1030005040 tx=0x7f10300012d0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 -- 172.21.15.27:0/461127274 shutdown_connections 2023-08-11T15:52:38.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f102806a080 0x7f102806c540 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10400ace50 0x7f10400ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10400c3680 0x7f10400c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 --2- 172.21.15.27:0/461127274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024003b30 0x7f10400af090 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 -- 172.21.15.27:0/461127274 >> 172.21.15.27:0/461127274 conn(0x7f10400afbe0 msgr2=0x7f10400adb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 -- 172.21.15.27:0/461127274 shutdown_connections 2023-08-11T15:52:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.344+0000 7f1046ba4640 1 -- 172.21.15.27:0/461127274 wait complete. 2023-08-11T15:52:38.362 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:52:38.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.597+0000 7f1830f29640 1 Processor -- start 2023-08-11T15:52:38.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.597+0000 7f1830f29640 1 -- start start 2023-08-11T15:52:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.598+0000 7f1830f29640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f182c06ca00 0x7f182c0aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.598+0000 7f1830f29640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c3250 0x7f182c0acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.598+0000 7f1830f29640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0ab970 con 0x7f182c0c2e70 2023-08-11T15:52:38.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.598+0000 7f1830f29640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c0abc50 con 0x7f182c06ca00 2023-08-11T15:52:38.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.598+0000 7f1830f29640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0abf30 con 0x7f182c0c3250 2023-08-11T15:52:38.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.599+0000 7f182affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f182c06ca00 0x7f182c0aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.599+0000 7f182affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f182c06ca00 0x7f182c0aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45156/0 (socket says 172.21.15.27:45156) 2023-08-11T15:52:38.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.599+0000 7f182affd640 1 -- 172.21.15.27:0/3401843936 learned_addr learned my addr 172.21.15.27:0/3401843936 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:38.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.599+0000 7f182bfff640 1 --2- 172.21.15.27:0/3401843936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c3250 0x7f182c0acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.599+0000 7f182affd640 1 -- 172.21.15.27:0/3401843936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c3250 msgr2=0x7f182c0acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:38.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.599+0000 7f182affd640 1 --2- 172.21.15.27:0/3401843936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c3250 0x7f182c0acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.599+0000 7f182affd640 1 -- 172.21.15.27:0/3401843936 >> v1:172.21.15.119:6789/0 conn(0x7f182c0c2e70 legacy=0x7f182c160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:38.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.599+0000 7f182affd640 1 -- 172.21.15.27:0/3401843936 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0ac210 con 0x7f182c06ca00 2023-08-11T15:52:38.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.600+0000 7f182affd640 1 --2- 172.21.15.27:0/3401843936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 0x7f182c0aca10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1820009870 tx=0x7f182002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfce686cb142499 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:38.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.601+0000 7f182a7fc640 1 -- 172.21.15.27:0/3401843936 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820031640 con 0x7f182c06ca00 2023-08-11T15:52:38.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.601+0000 7f182a7fc640 1 -- 172.21.15.27:0/3401843936 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18200317a0 con 0x7f182c06ca00 2023-08-11T15:52:38.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.601+0000 7f182a7fc640 1 -- 172.21.15.27:0/3401843936 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820031b30 con 0x7f182c06ca00 2023-08-11T15:52:38.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.601+0000 7f1830f29640 1 -- 172.21.15.27:0/3401843936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 msgr2=0x7f182c0aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.601+0000 7f1830f29640 1 --2- 172.21.15.27:0/3401843936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 0x7f182c0aca10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1820009870 tx=0x7f182002b8f0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.602+0000 7f1830f29640 1 -- 172.21.15.27:0/3401843936 shutdown_connections 2023-08-11T15:52:38.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.602+0000 7f1830f29640 1 --2- 172.21.15.27:0/3401843936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c3250 0x7f182c0acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.602+0000 7f1830f29640 1 --2- 172.21.15.27:0/3401843936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 0x7f182c0aca10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.602+0000 7f1830f29640 1 -- 172.21.15.27:0/3401843936 >> 172.21.15.27:0/3401843936 conn(0x7f182c069800 msgr2=0x7f182c069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:38.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.602+0000 7f1830f29640 1 -- 172.21.15.27:0/3401843936 shutdown_connections 2023-08-11T15:52:38.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.602+0000 7f1830f29640 1 -- 172.21.15.27:0/3401843936 wait complete. 2023-08-11T15:52:38.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.603+0000 7f1830f29640 1 Processor -- start 2023-08-11T15:52:38.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.604+0000 7f1830f29640 1 -- start start 2023-08-11T15:52:38.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f1830f29640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 0x7f182c152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f1830f29640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c2e70 0x7f182c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f1830f29640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c0c3250 0x7f182c15ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f1830f29640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c070680 con 0x7f182c06ca00 2023-08-11T15:52:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f1830f29640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c152d80 con 0x7f182c0c3250 2023-08-11T15:52:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f1830f29640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c152fd0 con 0x7f182c0c2e70 2023-08-11T15:52:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 0x7f182c152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 0x7f182c152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45158/0 (socket says 172.21.15.27:45158) 2023-08-11T15:52:38.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182b7fe640 1 -- 172.21.15.27:0/14691906 learned_addr learned my addr 172.21.15.27:0/14691906 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:38.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182affd640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c2e70 0x7f182c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182bfff640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c0c3250 0x7f182c15ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182b7fe640 1 -- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c2e70 msgr2=0x7f182c06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182b7fe640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c2e70 0x7f182c06f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182b7fe640 1 -- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c0c3250 msgr2=0x7f182c15ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182b7fe640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c0c3250 0x7f182c15ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182b7fe640 1 -- 172.21.15.27:0/14691906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18200094d0 con 0x7f182c06ca00 2023-08-11T15:52:38.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182affd640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c2e70 0x7f182c06f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:38.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.605+0000 7f182bfff640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c0c3250 0x7f182c15ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:38.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.606+0000 7f182b7fe640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 0x7f182c152710 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f181c00c430 tx=0x7f181c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:38.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.606+0000 7f1813fff640 1 -- 172.21.15.27:0/14691906 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c011790 con 0x7f182c06ca00 2023-08-11T15:52:38.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.606+0000 7f1813fff640 1 -- 172.21.15.27:0/14691906 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c0118f0 con 0x7f182c06ca00 2023-08-11T15:52:38.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.606+0000 7f1830f29640 1 -- 172.21.15.27:0/14691906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c1532b0 con 0x7f182c06ca00 2023-08-11T15:52:38.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.606+0000 7f1813fff640 1 -- 172.21.15.27:0/14691906 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c011ce0 con 0x7f182c06ca00 2023-08-11T15:52:38.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.606+0000 7f1830f29640 1 -- 172.21.15.27:0/14691906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c153870 con 0x7f182c06ca00 2023-08-11T15:52:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.607+0000 7f1830f29640 1 -- 172.21.15.27:0/14691906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c066e80 con 0x7f182c06ca00 2023-08-11T15:52:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.608+0000 7f1813fff640 1 -- 172.21.15.27:0/14691906 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f181c01c460 con 0x7f182c06ca00 2023-08-11T15:52:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.609+0000 7f1813fff640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f181806a050 0x7f181806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.609+0000 7f1813fff640 1 -- 172.21.15.27:0/14691906 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f181c088860 con 0x7f182c06ca00 2023-08-11T15:52:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.610+0000 7f182affd640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f181806a050 0x7f181806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.611+0000 7f182affd640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f181806a050 0x7f181806c510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1820036000 tx=0x7f182003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.613+0000 7f1813fff640 1 -- 172.21.15.27:0/14691906 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f181c053c20 con 0x7f182c06ca00 2023-08-11T15:52:38.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.813+0000 7f1830f29640 1 -- 172.21.15.27:0/14691906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f182c156a60 con 0x7f182c06ca00 2023-08-11T15:52:38.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.813+0000 7f1813fff640 1 -- 172.21.15.27:0/14691906 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f181c057240 con 0x7f182c06ca00 2023-08-11T15:52:38.816 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:52:38.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.816+0000 7f1830f29640 1 -- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f181806a050 msgr2=0x7f181806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.816+0000 7f1830f29640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f181806a050 0x7f181806c510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1820036000 tx=0x7f182003c040 comp rx=0 tx=0).stop 2023-08-11T15:52:38.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.816+0000 7f1830f29640 1 -- 172.21.15.27:0/14691906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 msgr2=0x7f182c152710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:38.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.816+0000 7f1830f29640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 0x7f182c152710 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f181c00c430 tx=0x7f181c00c8f0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.816+0000 7f1830f29640 1 -- 172.21.15.27:0/14691906 shutdown_connections 2023-08-11T15:52:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.816+0000 7f1830f29640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f181806a050 0x7f181806c510 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.816+0000 7f1830f29640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f182c0c2e70 0x7f182c06f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.816+0000 7f1830f29640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c0c3250 0x7f182c15ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.816+0000 7f1830f29640 1 --2- 172.21.15.27:0/14691906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f182c06ca00 0x7f182c152710 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.816+0000 7f1830f29640 1 -- 172.21.15.27:0/14691906 >> 172.21.15.27:0/14691906 conn(0x7f182c069800 msgr2=0x7f182c1557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.817+0000 7f1830f29640 1 -- 172.21.15.27:0/14691906 shutdown_connections 2023-08-11T15:52:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:38.817+0000 7f1830f29640 1 -- 172.21.15.27:0/14691906 wait complete. 2023-08-11T15:52:38.837 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:52:38.837 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:52:38.897 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:39.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.122+0000 7f75e667c640 1 Processor -- start 2023-08-11T15:52:39.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.122+0000 7f75e667c640 1 -- start start 2023-08-11T15:52:39.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.124+0000 7f75e667c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f75e015a140 0x7f75e006b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.124+0000 7f75e667c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75e00cb430 0x7f75e00b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.124+0000 7f75e667c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75e006f5d0 0x7f75e006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.124+0000 7f75e667c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e006ffa0 con 0x7f75e006f5d0 2023-08-11T15:52:39.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.124+0000 7f75e667c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e00c8a30 con 0x7f75e015a140 2023-08-11T15:52:39.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.124+0000 7f75e667c640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e006c0c0 con 0x7f75e00cb430 2023-08-11T15:52:39.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.124+0000 7f75e4c19640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75e006f5d0 0x7f75e006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.124+0000 7f75e4c19640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75e006f5d0 0x7f75e006fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43008/0 (socket says 172.21.15.27:43008) 2023-08-11T15:52:39.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.124+0000 7f75e4c19640 1 -- 172.21.15.27:0/2638899628 learned_addr learned my addr 172.21.15.27:0/2638899628 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:39.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.125+0000 7f75df7fe640 1 --2- 172.21.15.27:0/2638899628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75e00cb430 0x7f75e00b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.125+0000 7f75dffff640 1 --2- 172.21.15.27:0/2638899628 >> v2:172.21.15.119:3300/0 conn(0x7f75e015a140 0x7f75e006b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.125+0000 7f75e4c19640 1 -- 172.21.15.27:0/2638899628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75e00cb430 msgr2=0x7f75e00b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:39.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.125+0000 7f75e4c19640 1 --2- 172.21.15.27:0/2638899628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75e00cb430 0x7f75e00b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.125+0000 7f75e4c19640 1 -- 172.21.15.27:0/2638899628 >> v2:172.21.15.119:3300/0 conn(0x7f75e015a140 msgr2=0x7f75e006b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:39.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.125+0000 7f75e4c19640 1 --2- 172.21.15.27:0/2638899628 >> v2:172.21.15.119:3300/0 conn(0x7f75e015a140 0x7f75e006b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.125+0000 7f75e4c19640 1 -- 172.21.15.27:0/2638899628 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e006c3a0 con 0x7f75e006f5d0 2023-08-11T15:52:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.126+0000 7f75e4c19640 1 --2- 172.21.15.27:0/2638899628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 0x7f75e006fa60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f75e006dae0 tx=0x7f75d802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cd39bac5df7640 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75deffd640 1 -- 172.21.15.27:0/2638899628 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d802ca80 con 0x7f75e006f5d0 2023-08-11T15:52:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75deffd640 1 -- 172.21.15.27:0/2638899628 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d802cbe0 con 0x7f75e006f5d0 2023-08-11T15:52:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75deffd640 1 -- 172.21.15.27:0/2638899628 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d80343f0 con 0x7f75e006f5d0 2023-08-11T15:52:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75e667c640 1 -- 172.21.15.27:0/2638899628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 msgr2=0x7f75e006fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75e667c640 1 --2- 172.21.15.27:0/2638899628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 0x7f75e006fa60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f75e006dae0 tx=0x7f75d802bee0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75e667c640 1 -- 172.21.15.27:0/2638899628 shutdown_connections 2023-08-11T15:52:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75e667c640 1 --2- 172.21.15.27:0/2638899628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 0x7f75e006fa60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75e667c640 1 --2- 172.21.15.27:0/2638899628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75e00cb430 0x7f75e00b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75e667c640 1 --2- 172.21.15.27:0/2638899628 >> v2:172.21.15.119:3300/0 conn(0x7f75e015a140 0x7f75e006b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75e667c640 1 -- 172.21.15.27:0/2638899628 >> 172.21.15.27:0/2638899628 conn(0x7f75e00afb70 msgr2=0x7f75e00aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75e667c640 1 -- 172.21.15.27:0/2638899628 shutdown_connections 2023-08-11T15:52:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.127+0000 7f75e667c640 1 -- 172.21.15.27:0/2638899628 wait complete. 2023-08-11T15:52:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.129+0000 7f75e667c640 1 Processor -- start 2023-08-11T15:52:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.129+0000 7f75e667c640 1 -- start start 2023-08-11T15:52:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.130+0000 7f75e667c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 0x7f75e01517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.130+0000 7f75e667c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75e00cb430 0x7f75e00c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.130+0000 7f75e667c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e015a140 0x7f75e00c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.130+0000 7f75e667c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0150710 con 0x7f75e006f5d0 2023-08-11T15:52:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.130+0000 7f75e667c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0150340 con 0x7f75e015a140 2023-08-11T15:52:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.130+0000 7f75e667c640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e014f2a0 con 0x7f75e00cb430 2023-08-11T15:52:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.131+0000 7f75dffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 0x7f75e01517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.131+0000 7f75dffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 0x7f75e01517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43020/0 (socket says 172.21.15.27:43020) 2023-08-11T15:52:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.131+0000 7f75dffff640 1 -- 172.21.15.27:0/213116503 learned_addr learned my addr 172.21.15.27:0/213116503 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.131+0000 7f75dffff640 1 -- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75e00cb430 msgr2=0x7f75e00c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.131+0000 7f75dffff640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75e00cb430 0x7f75e00c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.131+0000 7f75dffff640 1 -- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e015a140 msgr2=0x7f75e00c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.131+0000 7f75e4c19640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e015a140 0x7f75e00c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.131+0000 7f75dffff640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e015a140 0x7f75e00c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.131+0000 7f75dffff640 1 -- 172.21.15.27:0/213116503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d8009b30 con 0x7f75e006f5d0 2023-08-11T15:52:39.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.131+0000 7f75dffff640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 0x7f75e01517c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d7d0 tx=0x7f75d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:39.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.132+0000 7f75dcff9640 1 -- 172.21.15.27:0/213116503 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d00096c0 con 0x7f75e006f5d0 2023-08-11T15:52:39.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.132+0000 7f75dcff9640 1 -- 172.21.15.27:0/213116503 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0009820 con 0x7f75e006f5d0 2023-08-11T15:52:39.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.132+0000 7f75dcff9640 1 -- 172.21.15.27:0/213116503 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0009bd0 con 0x7f75e006f5d0 2023-08-11T15:52:39.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.132+0000 7f75e667c640 1 -- 172.21.15.27:0/213116503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e014f580 con 0x7f75e006f5d0 2023-08-11T15:52:39.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.132+0000 7f75e667c640 1 -- 172.21.15.27:0/213116503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e014fb40 con 0x7f75e006f5d0 2023-08-11T15:52:39.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.133+0000 7f75e667c640 1 -- 172.21.15.27:0/213116503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e006bd60 con 0x7f75e006f5d0 2023-08-11T15:52:39.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.134+0000 7f75dcff9640 1 -- 172.21.15.27:0/213116503 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f75d0009d30 con 0x7f75e006f5d0 2023-08-11T15:52:39.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.135+0000 7f75dcff9640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75cc06a050 0x7f75cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.136+0000 7f75dcff9640 1 -- 172.21.15.27:0/213116503 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f75d00890d0 con 0x7f75e006f5d0 2023-08-11T15:52:39.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.136+0000 7f75df7fe640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75cc06a050 0x7f75cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.137+0000 7f75df7fe640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75cc06a050 0x7f75cc06c510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f75e00c7510 tx=0x7f75d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:39.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.140+0000 7f75dcff9640 1 -- 172.21.15.27:0/213116503 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f75d0057e80 con 0x7f75e006f5d0 2023-08-11T15:52:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.336+0000 7f75e667c640 1 -- 172.21.15.27:0/213116503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75e0073990 con 0x7f75e006f5d0 2023-08-11T15:52:39.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.340+0000 7f75dcff9640 1 -- 172.21.15.27:0/213116503 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f75d0057a10 con 0x7f75e006f5d0 2023-08-11T15:52:39.342 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:39.343 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 -- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75cc06a050 msgr2=0x7f75cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75cc06a050 0x7f75cc06c510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f75e00c7510 tx=0x7f75d4008040 comp rx=0 tx=0).stop 2023-08-11T15:52:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 -- 172.21.15.27:0/213116503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 msgr2=0x7f75e01517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 0x7f75e01517c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d7d0 tx=0x7f75d000dc90 comp rx=0 tx=0).stop 2023-08-11T15:52:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 -- 172.21.15.27:0/213116503 shutdown_connections 2023-08-11T15:52:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75cc06a050 0x7f75cc06c510 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75e00cb430 0x7f75e00c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e015a140 0x7f75e00c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 --2- 172.21.15.27:0/213116503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e006f5d0 0x7f75e01517c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 -- 172.21.15.27:0/213116503 >> 172.21.15.27:0/213116503 conn(0x7f75e00afb70 msgr2=0x7f75e00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 -- 172.21.15.27:0/213116503 shutdown_connections 2023-08-11T15:52:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.343+0000 7f75e667c640 1 -- 172.21.15.27:0/213116503 wait complete. 2023-08-11T15:52:39.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:39.365 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:39.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.588+0000 7f22a88ec640 1 Processor -- start 2023-08-11T15:52:39.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.588+0000 7f22a88ec640 1 -- start start 2023-08-11T15:52:39.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.590+0000 7f22a88ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22a413ce80 0x7f22a413f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.590+0000 7f22a88ec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a40cd470 con 0x7f22a413d260 2023-08-11T15:52:39.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.590+0000 7f22a88ec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a40cd750 con 0x7f22a40cce30 2023-08-11T15:52:39.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.590+0000 7f22a88ec640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a4140670 con 0x7f22a413ce80 2023-08-11T15:52:39.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.590+0000 7f22a2ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f22a413d260 0x7f22a40cc720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57456/0 (socket says 172.21.15.27:57456) 2023-08-11T15:52:39.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.590+0000 7f22a2ffd640 1 -- 172.21.15.27:0/3523686817 learned_addr learned my addr 172.21.15.27:0/3523686817 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:39.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a37fe640 1 --2- 172.21.15.27:0/3523686817 >> v2:172.21.15.119:3300/0 conn(0x7f22a413ce80 0x7f22a413f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850734569 0 0) 0x7f22a40cd470 con 0x7f22a413d260 2023-08-11T15:52:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2290003520 con 0x7f22a413d260 2023-08-11T15:52:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3122797432 0 0) 0x7f2290003520 con 0x7f22a413d260 2023-08-11T15:52:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 >> v2:172.21.15.119:3300/0 conn(0x7f22a413ce80 msgr2=0x7f22a413f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 --2- 172.21.15.27:0/3523686817 >> v2:172.21.15.119:3300/0 conn(0x7f22a413ce80 0x7f22a413f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 >> v1:172.21.15.119:6789/0 conn(0x7f22a40cce30 legacy=0x7f22a40adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a40ad7d0 con 0x7f22a413d260 2023-08-11T15:52:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f228c001670 con 0x7f22a413d260 2023-08-11T15:52:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 >> v1:172.21.15.27:6789/0 conn(0x7f22a413d260 legacy=0x7f22a40cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2290003970 0x7f2290003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22900047c0 0x7f2290004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2290005180 0x7f229000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4140670 con 0x7f22900047c0 2023-08-11T15:52:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229000c0b0 con 0x7f2290005180 2023-08-11T15:52:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.591+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229000c270 con 0x7f2290003970 2023-08-11T15:52:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.592+0000 7f22a2ffd640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22900047c0 0x7f2290004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.592+0000 7f22a3fff640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2290005180 0x7f229000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.592+0000 7f22a37fe640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2290003970 0x7f2290003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.592+0000 7f22a2ffd640 1 -- 172.21.15.27:0/3523686817 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2290003970 msgr2=0x7f2290003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.592+0000 7f22a2ffd640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2290003970 0x7f2290003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.592+0000 7f22a2ffd640 1 -- 172.21.15.27:0/3523686817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2290005180 msgr2=0x7f229000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.592+0000 7f22a2ffd640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2290005180 0x7f229000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.592+0000 7f22a2ffd640 1 -- 172.21.15.27:0/3523686817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2290003520 con 0x7f22900047c0 2023-08-11T15:52:39.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.592+0000 7f22a3fff640 1 -- 172.21.15.27:0/3523686817 reap_dead start 2023-08-11T15:52:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.593+0000 7f22a2ffd640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22900047c0 0x7f2290004c40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f228c001eb0 tx=0x7f22900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5333029fa717bc8b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.594+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c005ad0 con 0x7f22900047c0 2023-08-11T15:52:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.594+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c005c50 con 0x7f22900047c0 2023-08-11T15:52:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.594+0000 7f22a27fc640 1 -- 172.21.15.27:0/3523686817 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c02c890 con 0x7f22900047c0 2023-08-11T15:52:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.594+0000 7f22a88ec640 1 -- 172.21.15.27:0/3523686817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22900047c0 msgr2=0x7f2290004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.594+0000 7f22a88ec640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22900047c0 0x7f2290004c40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f228c001eb0 tx=0x7f22900057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.594+0000 7f22a88ec640 1 -- 172.21.15.27:0/3523686817 shutdown_connections 2023-08-11T15:52:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.594+0000 7f22a88ec640 1 --2- 172.21.15.27:0/3523686817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22900047c0 0x7f2290004c40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.594+0000 7f22a88ec640 1 -- 172.21.15.27:0/3523686817 >> 172.21.15.27:0/3523686817 conn(0x7f22a4069800 msgr2=0x7f22a4069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:39.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.594+0000 7f22a88ec640 1 -- 172.21.15.27:0/3523686817 shutdown_connections 2023-08-11T15:52:39.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.594+0000 7f22a88ec640 1 -- 172.21.15.27:0/3523686817 wait complete. 2023-08-11T15:52:39.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.596+0000 7f22a88ec640 1 Processor -- start 2023-08-11T15:52:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.596+0000 7f22a88ec640 1 -- start start 2023-08-11T15:52:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.597+0000 7f22a88ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22900047c0 0x7f22a41378e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.597+0000 7f22a88ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22a413bf60 0x7f22a4154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.597+0000 7f22a88ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22a4156810 0x7f22a4156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.597+0000 7f22a88ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4137e20 con 0x7f22a413bf60 2023-08-11T15:52:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.597+0000 7f22a88ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a41388e0 con 0x7f22a4156810 2023-08-11T15:52:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.597+0000 7f22a88ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4138320 con 0x7f22900047c0 2023-08-11T15:52:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.597+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22a413bf60 0x7f22a4154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.597+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22a413bf60 0x7f22a4154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43030/0 (socket says 172.21.15.27:43030) 2023-08-11T15:52:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.597+0000 7f22a2ffd640 1 -- 172.21.15.27:0/3449743987 learned_addr learned my addr 172.21.15.27:0/3449743987 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.598+0000 7f22a37fe640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22900047c0 0x7f22a41378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.598+0000 7f22a3fff640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22a4156810 0x7f22a4156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.598+0000 7f22a2ffd640 1 -- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22900047c0 msgr2=0x7f22a41378e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.598+0000 7f22a2ffd640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22900047c0 0x7f22a41378e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.598+0000 7f22a2ffd640 1 -- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22a4156810 msgr2=0x7f22a4156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.598+0000 7f22a2ffd640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22a4156810 0x7f22a4156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.598+0000 7f22a2ffd640 1 -- 172.21.15.27:0/3449743987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4138600 con 0x7f22a413bf60 2023-08-11T15:52:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.598+0000 7f22a2ffd640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22a413bf60 0x7f22a4154db0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f228c02c430 tx=0x7f228c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.599+0000 7f229bfff640 1 -- 172.21.15.27:0/3449743987 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c03c040 con 0x7f22a413bf60 2023-08-11T15:52:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.599+0000 7f229bfff640 1 -- 172.21.15.27:0/3449743987 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c02cd60 con 0x7f22a413bf60 2023-08-11T15:52:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.599+0000 7f229bfff640 1 -- 172.21.15.27:0/3449743987 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c007270 con 0x7f22a413bf60 2023-08-11T15:52:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.599+0000 7f22a88ec640 1 -- 172.21.15.27:0/3449743987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a41574e0 con 0x7f22a413bf60 2023-08-11T15:52:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.599+0000 7f22a88ec640 1 -- 172.21.15.27:0/3449743987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a41571d0 con 0x7f22a413bf60 2023-08-11T15:52:39.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.600+0000 7f22a88ec640 1 -- 172.21.15.27:0/3449743987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a4066e80 con 0x7f22a413bf60 2023-08-11T15:52:39.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.601+0000 7f229bfff640 1 -- 172.21.15.27:0/3449743987 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f228c032070 con 0x7f22a413bf60 2023-08-11T15:52:39.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.602+0000 7f229bfff640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f228406a080 0x7f228406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:39.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.602+0000 7f229bfff640 1 -- 172.21.15.27:0/3449743987 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f228c0a9d30 con 0x7f22a413bf60 2023-08-11T15:52:39.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.603+0000 7f22a37fe640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f228406a080 0x7f228406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:39.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.604+0000 7f22a37fe640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f228406a080 0x7f228406c540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2294006fd0 tx=0x7f2294008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:39.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.606+0000 7f229bfff640 1 -- 172.21.15.27:0/3449743987 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f228c0750f0 con 0x7f22a413bf60 2023-08-11T15:52:39.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.794+0000 7f22a88ec640 1 -- 172.21.15.27:0/3449743987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22a406a6e0 con 0x7f22a413bf60 2023-08-11T15:52:39.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.795+0000 7f229bfff640 1 -- 172.21.15.27:0/3449743987 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f228c078710 con 0x7f22a413bf60 2023-08-11T15:52:39.797 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.798+0000 7f22a88ec640 1 -- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f228406a080 msgr2=0x7f228406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.798+0000 7f22a88ec640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f228406a080 0x7f228406c540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2294006fd0 tx=0x7f2294008040 comp rx=0 tx=0).stop 2023-08-11T15:52:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.798+0000 7f22a88ec640 1 -- 172.21.15.27:0/3449743987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22a413bf60 msgr2=0x7f22a4154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.798+0000 7f22a88ec640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22a413bf60 0x7f22a4154db0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f228c02c430 tx=0x7f228c02e070 comp rx=0 tx=0).stop 2023-08-11T15:52:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.799+0000 7f22a88ec640 1 -- 172.21.15.27:0/3449743987 shutdown_connections 2023-08-11T15:52:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.799+0000 7f22a88ec640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f228406a080 0x7f228406c540 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.799+0000 7f22a88ec640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22a4156810 0x7f22a4156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.799+0000 7f22a88ec640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22a413bf60 0x7f22a4154db0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.799+0000 7f22a88ec640 1 --2- 172.21.15.27:0/3449743987 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22900047c0 0x7f22a41378e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:39.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.799+0000 7f22a88ec640 1 -- 172.21.15.27:0/3449743987 >> 172.21.15.27:0/3449743987 conn(0x7f22a4069800 msgr2=0x7f22a4152ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:39.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.799+0000 7f22a88ec640 1 -- 172.21.15.27:0/3449743987 shutdown_connections 2023-08-11T15:52:39.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:39.799+0000 7f22a88ec640 1 -- 172.21.15.27:0/3449743987 wait complete. 2023-08-11T15:52:39.818 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:40.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.054+0000 7f609ca56640 1 Processor -- start 2023-08-11T15:52:40.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.054+0000 7f609ca56640 1 -- start start 2023-08-11T15:52:40.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.056+0000 7f609ca56640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f609806ffd0 0x7f60980b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.056+0000 7f609ca56640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 0x7f6098156580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.056+0000 7f609ca56640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60980b1330 con 0x7f60980703b0 2023-08-11T15:52:40.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.056+0000 7f609ca56640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60980b1f70 con 0x7f609806ffd0 2023-08-11T15:52:40.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.056+0000 7f609ca56640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60980b15d0 con 0x7f60980b2f60 2023-08-11T15:52:40.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.057+0000 7f6096d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f609806ffd0 0x7f60980b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:40.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.057+0000 7f6096d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f609806ffd0 0x7f60980b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43036/0 (socket says 172.21.15.27:43036) 2023-08-11T15:52:40.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.057+0000 7f6096d9d640 1 -- 172.21.15.27:0/1141769838 learned_addr learned my addr 172.21.15.27:0/1141769838 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:40.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.057+0000 7f609659c640 1 --2- 172.21.15.27:0/1141769838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 0x7f6098156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:40.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.057+0000 7f6096d9d640 1 -- 172.21.15.27:0/1141769838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 msgr2=0x7f6098156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:40.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.057+0000 7f6096d9d640 1 --2- 172.21.15.27:0/1141769838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 0x7f6098156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.057+0000 7f6096d9d640 1 -- 172.21.15.27:0/1141769838 >> v1:172.21.15.119:6789/0 conn(0x7f60980703b0 legacy=0x7f6098160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:40.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.057+0000 7f6096d9d640 1 -- 172.21.15.27:0/1141769838 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980b18b0 con 0x7f609806ffd0 2023-08-11T15:52:40.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.057+0000 7f609659c640 1 --2- 172.21.15.27:0/1141769838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 0x7f6098156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:40.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.059+0000 7f6096d9d640 1 --2- 172.21.15.27:0/1141769838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609806ffd0 0x7f60980b2a20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6088009870 tx=0x7f608802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c77b6fb9d3f4eb4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:40.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.059+0000 7f6095d9b640 1 -- 172.21.15.27:0/1141769838 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60880317c0 con 0x7f609806ffd0 2023-08-11T15:52:40.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.059+0000 7f6095d9b640 1 -- 172.21.15.27:0/1141769838 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6088031920 con 0x7f609806ffd0 2023-08-11T15:52:40.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.059+0000 7f6095d9b640 1 -- 172.21.15.27:0/1141769838 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088031cb0 con 0x7f609806ffd0 2023-08-11T15:52:40.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.060+0000 7f609ca56640 1 -- 172.21.15.27:0/1141769838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609806ffd0 msgr2=0x7f60980b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.060+0000 7f609ca56640 1 --2- 172.21.15.27:0/1141769838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609806ffd0 0x7f60980b2a20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6088009870 tx=0x7f608802b860 comp rx=0 tx=0).stop 2023-08-11T15:52:40.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.060+0000 7f609ca56640 1 -- 172.21.15.27:0/1141769838 shutdown_connections 2023-08-11T15:52:40.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.060+0000 7f609ca56640 1 --2- 172.21.15.27:0/1141769838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 0x7f6098156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.060+0000 7f609ca56640 1 --2- 172.21.15.27:0/1141769838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609806ffd0 0x7f60980b2a20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.060+0000 7f609ca56640 1 -- 172.21.15.27:0/1141769838 >> 172.21.15.27:0/1141769838 conn(0x7f6098069800 msgr2=0x7f6098069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.060+0000 7f609ca56640 1 -- 172.21.15.27:0/1141769838 shutdown_connections 2023-08-11T15:52:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.060+0000 7f609ca56640 1 -- 172.21.15.27:0/1141769838 wait complete. 2023-08-11T15:52:40.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.062+0000 7f609ca56640 1 Processor -- start 2023-08-11T15:52:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.062+0000 7f609ca56640 1 -- start start 2023-08-11T15:52:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.063+0000 7f609ca56640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609806ffd0 0x7f6098158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.063+0000 7f609ca56640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60980703b0 0x7f60981599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.063+0000 7f609ca56640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 0x7f6098159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.063+0000 7f609ca56640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981f5dc0 con 0x7f60980703b0 2023-08-11T15:52:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.063+0000 7f609ca56640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981f5930 con 0x7f609806ffd0 2023-08-11T15:52:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.063+0000 7f609ca56640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609815a520 con 0x7f60980b2f60 2023-08-11T15:52:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.063+0000 7f609659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60980703b0 0x7f60981599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.063+0000 7f609659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60980703b0 0x7f60981599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43048/0 (socket says 172.21.15.27:43048) 2023-08-11T15:52:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.063+0000 7f609659c640 1 -- 172.21.15.27:0/3776366480 learned_addr learned my addr 172.21.15.27:0/3776366480 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f609659c640 1 -- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 msgr2=0x7f6098159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f609759e640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 0x7f6098159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f6096d9d640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609806ffd0 0x7f6098158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f609659c640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 0x7f6098159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f609659c640 1 -- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609806ffd0 msgr2=0x7f6098158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f609659c640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609806ffd0 0x7f6098158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f609659c640 1 -- 172.21.15.27:0/3776366480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60880094d0 con 0x7f60980703b0 2023-08-11T15:52:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f609659c640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60980703b0 0x7f60981599b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f608000c7f0 tx=0x7f608000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f608f7fe640 1 -- 172.21.15.27:0/3776366480 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080007a30 con 0x7f60980703b0 2023-08-11T15:52:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f608f7fe640 1 -- 172.21.15.27:0/3776366480 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6080007b90 con 0x7f60980703b0 2023-08-11T15:52:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f608f7fe640 1 -- 172.21.15.27:0/3776366480 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608000ce40 con 0x7f60980703b0 2023-08-11T15:52:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.064+0000 7f609ca56640 1 -- 172.21.15.27:0/3776366480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60981f5000 con 0x7f60980703b0 2023-08-11T15:52:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.065+0000 7f609ca56640 1 -- 172.21.15.27:0/3776366480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60981f55c0 con 0x7f60980703b0 2023-08-11T15:52:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.066+0000 7f609ca56640 1 -- 172.21.15.27:0/3776366480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6098066e80 con 0x7f60980703b0 2023-08-11T15:52:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.067+0000 7f608f7fe640 1 -- 172.21.15.27:0/3776366480 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f60800113f0 con 0x7f60980703b0 2023-08-11T15:52:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.068+0000 7f608f7fe640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f607806a050 0x7f607806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.068+0000 7f608f7fe640 1 -- 172.21.15.27:0/3776366480 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6080088980 con 0x7f60980703b0 2023-08-11T15:52:40.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.069+0000 7f6096d9d640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f607806a050 0x7f607806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:40.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.070+0000 7f6096d9d640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f607806a050 0x7f607806c510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6088036000 tx=0x7f608803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.072+0000 7f608f7fe640 1 -- 172.21.15.27:0/3776366480 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6080053dd0 con 0x7f60980703b0 2023-08-11T15:52:40.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.263+0000 7f609ca56640 1 -- 172.21.15.27:0/3776366480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6098160270 con 0x7f60980703b0 2023-08-11T15:52:40.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.264+0000 7f608f7fe640 1 -- 172.21.15.27:0/3776366480 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f60800573f0 con 0x7f60980703b0 2023-08-11T15:52:40.266 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.266+0000 7f609ca56640 1 -- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f607806a050 msgr2=0x7f607806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.266+0000 7f609ca56640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f607806a050 0x7f607806c510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6088036000 tx=0x7f608803d040 comp rx=0 tx=0).stop 2023-08-11T15:52:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.266+0000 7f609ca56640 1 -- 172.21.15.27:0/3776366480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60980703b0 msgr2=0x7f60981599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.266+0000 7f609ca56640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60980703b0 0x7f60981599b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f608000c7f0 tx=0x7f608000cbf0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.266+0000 7f609ca56640 1 -- 172.21.15.27:0/3776366480 shutdown_connections 2023-08-11T15:52:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.266+0000 7f609ca56640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f607806a050 0x7f607806c510 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.266+0000 7f609ca56640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60980b2f60 0x7f6098159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.266+0000 7f609ca56640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60980703b0 0x7f60981599b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.266+0000 7f609ca56640 1 --2- 172.21.15.27:0/3776366480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609806ffd0 0x7f6098158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.267+0000 7f609ca56640 1 -- 172.21.15.27:0/3776366480 >> 172.21.15.27:0/3776366480 conn(0x7f6098069800 msgr2=0x7f60981559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:40.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.267+0000 7f609ca56640 1 -- 172.21.15.27:0/3776366480 shutdown_connections 2023-08-11T15:52:40.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.267+0000 7f609ca56640 1 -- 172.21.15.27:0/3776366480 wait complete. 2023-08-11T15:52:40.288 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:40.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.527+0000 7f694b36d640 1 Processor -- start 2023-08-11T15:52:40.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.527+0000 7f694b36d640 1 -- start start 2023-08-11T15:52:40.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.528+0000 7f694b36d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69440ccbc0 0x7f69440afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.528+0000 7f694b36d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f694406c960 0x7f694406ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.529+0000 7f694b36d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69440b0420 con 0x7f69441592e0 2023-08-11T15:52:40.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.529+0000 7f694b36d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694406d350 con 0x7f694406c960 2023-08-11T15:52:40.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.529+0000 7f694b36d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69441596c0 con 0x7f69440ccbc0 2023-08-11T15:52:40.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.529+0000 7f694a36b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f69441592e0 0x7f694415d730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57464/0 (socket says 172.21.15.27:57464) 2023-08-11T15:52:40.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.529+0000 7f694a36b640 1 -- 172.21.15.27:0/1008078225 learned_addr learned my addr 172.21.15.27:0/1008078225 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:40.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.529+0000 7f694ab6c640 1 --2- 172.21.15.27:0/1008078225 >> v2:172.21.15.27:3300/0 conn(0x7f694406c960 0x7f694406ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:40.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.529+0000 7f6949b6a640 1 --2- 172.21.15.27:0/1008078225 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69440ccbc0 0x7f69440afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.529+0000 7f6949369640 1 -- 172.21.15.27:0/1008078225 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508849145 0 0) 0x7f69440b0420 con 0x7f69441592e0 2023-08-11T15:52:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.529+0000 7f6949369640 1 -- 172.21.15.27:0/1008078225 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6930003520 con 0x7f69441592e0 2023-08-11T15:52:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.530+0000 7f694ab6c640 1 -- 172.21.15.27:0/1008078225 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69440ccbc0 msgr2=0x7f69440afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.530+0000 7f694ab6c640 1 --2- 172.21.15.27:0/1008078225 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69440ccbc0 0x7f69440afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.530+0000 7f694ab6c640 1 -- 172.21.15.27:0/1008078225 >> v1:172.21.15.27:6789/0 conn(0x7f69441592e0 legacy=0x7f694415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.530+0000 7f694ab6c640 1 -- 172.21.15.27:0/1008078225 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440b0700 con 0x7f694406c960 2023-08-11T15:52:40.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.531+0000 7f694ab6c640 1 --2- 172.21.15.27:0/1008078225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 0x7f694406ce10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f693c009d90 tx=0x7f693c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb7b50aecc0c16e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:40.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.531+0000 7f6949369640 1 -- 172.21.15.27:0/1008078225 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c02cbd0 con 0x7f694406c960 2023-08-11T15:52:40.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.531+0000 7f6949369640 1 -- 172.21.15.27:0/1008078225 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c02cd30 con 0x7f694406c960 2023-08-11T15:52:40.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.531+0000 7f6949369640 1 -- 172.21.15.27:0/1008078225 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c02cea0 con 0x7f694406c960 2023-08-11T15:52:40.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.532+0000 7f694b36d640 1 -- 172.21.15.27:0/1008078225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 msgr2=0x7f694406ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.532+0000 7f694b36d640 1 --2- 172.21.15.27:0/1008078225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 0x7f694406ce10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f693c009d90 tx=0x7f693c02be20 comp rx=0 tx=0).stop 2023-08-11T15:52:40.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.532+0000 7f694b36d640 1 -- 172.21.15.27:0/1008078225 shutdown_connections 2023-08-11T15:52:40.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.532+0000 7f694b36d640 1 --2- 172.21.15.27:0/1008078225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 0x7f694406ce10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.532+0000 7f694b36d640 1 --2- 172.21.15.27:0/1008078225 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69440ccbc0 0x7f69440afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.532+0000 7f694b36d640 1 -- 172.21.15.27:0/1008078225 >> 172.21.15.27:0/1008078225 conn(0x7f69440caab0 msgr2=0x7f69440cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.532+0000 7f694b36d640 1 -- 172.21.15.27:0/1008078225 shutdown_connections 2023-08-11T15:52:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.532+0000 7f694b36d640 1 -- 172.21.15.27:0/1008078225 wait complete. 2023-08-11T15:52:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.534+0000 7f694b36d640 1 Processor -- start 2023-08-11T15:52:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.534+0000 7f694b36d640 1 -- start start 2023-08-11T15:52:40.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694b36d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 0x7f69440b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694b36d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440ccbc0 0x7f69440b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694b36d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69441592e0 0x7f694415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694b36d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440c19c0 con 0x7f694406c960 2023-08-11T15:52:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694b36d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440c1ca0 con 0x7f69440ccbc0 2023-08-11T15:52:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694b36d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69440c1f80 con 0x7f69441592e0 2023-08-11T15:52:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694a36b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 0x7f69440b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694a36b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 0x7f69440b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43064/0 (socket says 172.21.15.27:43064) 2023-08-11T15:52:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694a36b640 1 -- 172.21.15.27:0/3575221772 learned_addr learned my addr 172.21.15.27:0/3575221772 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694a36b640 1 -- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69441592e0 msgr2=0x7f694415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694a36b640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69441592e0 0x7f694415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694a36b640 1 -- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440ccbc0 msgr2=0x7f69440b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694a36b640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440ccbc0 0x7f69440b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.535+0000 7f694a36b640 1 -- 172.21.15.27:0/3575221772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c009a70 con 0x7f694406c960 2023-08-11T15:52:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.536+0000 7f694a36b640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 0x7f69440b1650 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f694000c500 tx=0x7f694000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.536+0000 7f693affd640 1 -- 172.21.15.27:0/3575221772 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940005ce0 con 0x7f694406c960 2023-08-11T15:52:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.536+0000 7f693affd640 1 -- 172.21.15.27:0/3575221772 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6940005e40 con 0x7f694406c960 2023-08-11T15:52:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.536+0000 7f693affd640 1 -- 172.21.15.27:0/3575221772 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940007330 con 0x7f694406c960 2023-08-11T15:52:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.536+0000 7f694b36d640 1 -- 172.21.15.27:0/3575221772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69440c2260 con 0x7f694406c960 2023-08-11T15:52:40.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.536+0000 7f694b36d640 1 -- 172.21.15.27:0/3575221772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6944159810 con 0x7f694406c960 2023-08-11T15:52:40.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.537+0000 7f694b36d640 1 -- 172.21.15.27:0/3575221772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694415cf10 con 0x7f694406c960 2023-08-11T15:52:40.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.538+0000 7f693affd640 1 -- 172.21.15.27:0/3575221772 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6940007490 con 0x7f694406c960 2023-08-11T15:52:40.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.539+0000 7f693affd640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f692806a080 0x7f692806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.539+0000 7f693affd640 1 -- 172.21.15.27:0/3575221772 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6940087eb0 con 0x7f694406c960 2023-08-11T15:52:40.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.540+0000 7f6949b6a640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f692806a080 0x7f692806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:40.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.541+0000 7f6949b6a640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f692806a080 0x7f692806c540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6934006fd0 tx=0x7f6934008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:40.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.543+0000 7f693affd640 1 -- 172.21.15.27:0/3575221772 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f694415cf10 con 0x7f694406c960 2023-08-11T15:52:40.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.739+0000 7f694b36d640 1 -- 172.21.15.27:0/3575221772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69440c2780 con 0x7f694406c960 2023-08-11T15:52:40.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.740+0000 7f693affd640 1 -- 172.21.15.27:0/3575221772 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6940017020 con 0x7f694406c960 2023-08-11T15:52:40.742 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:40.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 -- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f692806a080 msgr2=0x7f692806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f692806a080 0x7f692806c540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6934006fd0 tx=0x7f6934008040 comp rx=0 tx=0).stop 2023-08-11T15:52:40.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 -- 172.21.15.27:0/3575221772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 msgr2=0x7f69440b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:40.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 0x7f69440b1650 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f694000c500 tx=0x7f694000c9c0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 -- 172.21.15.27:0/3575221772 shutdown_connections 2023-08-11T15:52:40.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f692806a080 0x7f692806c540 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69441592e0 0x7f694415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440ccbc0 0x7f69440b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 --2- 172.21.15.27:0/3575221772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694406c960 0x7f69440b1650 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 -- 172.21.15.27:0/3575221772 >> 172.21.15.27:0/3575221772 conn(0x7f69440caab0 msgr2=0x7f694415ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 -- 172.21.15.27:0/3575221772 shutdown_connections 2023-08-11T15:52:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.743+0000 7f694b36d640 1 -- 172.21.15.27:0/3575221772 wait complete. 2023-08-11T15:52:40.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:40.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.995+0000 7f346e06a640 1 Processor -- start 2023-08-11T15:52:40.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.995+0000 7f346e06a640 1 -- start start 2023-08-11T15:52:40.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f346e06a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3468151220 0x7f346806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f346e06a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f346815e680 0x7f34680ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:40.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f346e06a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346806c850 con 0x7f346806bdf0 2023-08-11T15:52:40.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f346e06a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346806c290 con 0x7f3468151220 2023-08-11T15:52:40.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f346e06a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346806c570 con 0x7f346815e680 2023-08-11T15:52:40.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f34677fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3468151220 0x7f346806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f34677fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3468151220 0x7f346806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43068/0 (socket says 172.21.15.27:43068) 2023-08-11T15:52:41.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f34677fe640 1 -- 172.21.15.27:0/2739498183 learned_addr learned my addr 172.21.15.27:0/2739498183 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:41.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f3467fff640 1 --2- 172.21.15.27:0/2739498183 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f346815e680 0x7f34680ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f34677fe640 1 -- 172.21.15.27:0/2739498183 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f346815e680 msgr2=0x7f34680ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:41.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f34677fe640 1 --2- 172.21.15.27:0/2739498183 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f346815e680 0x7f34680ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.997+0000 7f34677fe640 1 -- 172.21.15.27:0/2739498183 >> v1:172.21.15.119:6789/0 conn(0x7f346806bdf0 legacy=0x7f346815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.998+0000 7f34677fe640 1 -- 172.21.15.27:0/2739498183 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34680ca050 con 0x7f3468151220 2023-08-11T15:52:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.999+0000 7f34677fe640 1 --2- 172.21.15.27:0/2739498183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3468151220 0x7f346806b8b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3458009e50 tx=0x7f345802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0659a983b7f682 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.999+0000 7f34667fc640 1 -- 172.21.15.27:0/2739498183 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345802c950 con 0x7f3468151220 2023-08-11T15:52:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.999+0000 7f34667fc640 1 -- 172.21.15.27:0/2739498183 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345802cab0 con 0x7f3468151220 2023-08-11T15:52:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:40.999+0000 7f34667fc640 1 -- 172.21.15.27:0/2739498183 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345802ce40 con 0x7f3468151220 2023-08-11T15:52:41.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.000+0000 7f346e06a640 1 -- 172.21.15.27:0/2739498183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3468151220 msgr2=0x7f346806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.000+0000 7f346e06a640 1 --2- 172.21.15.27:0/2739498183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3468151220 0x7f346806b8b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3458009e50 tx=0x7f345802bdb0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.000+0000 7f346e06a640 1 -- 172.21.15.27:0/2739498183 shutdown_connections 2023-08-11T15:52:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.000+0000 7f346e06a640 1 --2- 172.21.15.27:0/2739498183 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f346815e680 0x7f34680ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.000+0000 7f346e06a640 1 --2- 172.21.15.27:0/2739498183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3468151220 0x7f346806b8b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.000+0000 7f346e06a640 1 -- 172.21.15.27:0/2739498183 >> 172.21.15.27:0/2739498183 conn(0x7f34680afbe0 msgr2=0x7f34680affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.000+0000 7f346e06a640 1 -- 172.21.15.27:0/2739498183 shutdown_connections 2023-08-11T15:52:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.000+0000 7f346e06a640 1 -- 172.21.15.27:0/2739498183 wait complete. 2023-08-11T15:52:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.002+0000 7f346e06a640 1 Processor -- start 2023-08-11T15:52:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.002+0000 7f346e06a640 1 -- start start 2023-08-11T15:52:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.003+0000 7f346e06a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f346806bdf0 0x7f346814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.003+0000 7f346e06a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3468151220 0x7f3468151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.003+0000 7f346e06a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f346815e680 0x7f3468159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.003+0000 7f346e06a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34681540f0 con 0x7f346806bdf0 2023-08-11T15:52:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.003+0000 7f346e06a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34681543d0 con 0x7f3468151220 2023-08-11T15:52:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.003+0000 7f346e06a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34681546b0 con 0x7f346815e680 2023-08-11T15:52:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.003+0000 7f34677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f346806bdf0 0x7f346814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.003+0000 7f34677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f346806bdf0 0x7f346814d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43076/0 (socket says 172.21.15.27:43076) 2023-08-11T15:52:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.003+0000 7f34677fe640 1 -- 172.21.15.27:0/4191044402 learned_addr learned my addr 172.21.15.27:0/4191044402 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.004+0000 7f34677fe640 1 -- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f346815e680 msgr2=0x7f3468159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.004+0000 7f34677fe640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f346815e680 0x7f3468159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.004+0000 7f34677fe640 1 -- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3468151220 msgr2=0x7f3468151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.004+0000 7f34677fe640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3468151220 0x7f3468151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.004+0000 7f34677fe640 1 -- 172.21.15.27:0/4191044402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3458009b30 con 0x7f346806bdf0 2023-08-11T15:52:41.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.004+0000 7f34677fe640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f346806bdf0 0x7f346814d2c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3458009e50 tx=0x7f3458004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.004+0000 7f344ffff640 1 -- 172.21.15.27:0/4191044402 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345803e450 con 0x7f346806bdf0 2023-08-11T15:52:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.004+0000 7f344ffff640 1 -- 172.21.15.27:0/4191044402 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345803e5b0 con 0x7f346806bdf0 2023-08-11T15:52:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.004+0000 7f344ffff640 1 -- 172.21.15.27:0/4191044402 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345803e960 con 0x7f346806bdf0 2023-08-11T15:52:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.004+0000 7f346e06a640 1 -- 172.21.15.27:0/4191044402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3468154990 con 0x7f346806bdf0 2023-08-11T15:52:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.005+0000 7f346e06a640 1 -- 172.21.15.27:0/4191044402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3468153d10 con 0x7f346806bdf0 2023-08-11T15:52:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.006+0000 7f346e06a640 1 -- 172.21.15.27:0/4191044402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3468153ea0 con 0x7f346806bdf0 2023-08-11T15:52:41.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.007+0000 7f344ffff640 1 -- 172.21.15.27:0/4191044402 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f345803eac0 con 0x7f346806bdf0 2023-08-11T15:52:41.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.008+0000 7f344ffff640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f345406a050 0x7f345406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.008+0000 7f344ffff640 1 -- 172.21.15.27:0/4191044402 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f34580ab940 con 0x7f346806bdf0 2023-08-11T15:52:41.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.008+0000 7f3466ffd640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f345406a050 0x7f345406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.009+0000 7f3466ffd640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f345406a050 0x7f345406c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f34681f52a0 tx=0x7f345c0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:41.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.012+0000 7f344ffff640 1 -- 172.21.15.27:0/4191044402 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f3458076c60 con 0x7f346806bdf0 2023-08-11T15:52:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.215+0000 7f346e06a640 1 -- 172.21.15.27:0/4191044402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3468154230 con 0x7f346806bdf0 2023-08-11T15:52:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.216+0000 7f344ffff640 1 -- 172.21.15.27:0/4191044402 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3468154230 con 0x7f346806bdf0 2023-08-11T15:52:41.218 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 -- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f345406a050 msgr2=0x7f345406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f345406a050 0x7f345406c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f34681f52a0 tx=0x7f345c0079b0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 -- 172.21.15.27:0/4191044402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f346806bdf0 msgr2=0x7f346814d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f346806bdf0 0x7f346814d2c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3458009e50 tx=0x7f3458004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 -- 172.21.15.27:0/4191044402 shutdown_connections 2023-08-11T15:52:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f345406a050 0x7f345406c510 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f346815e680 0x7f3468159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3468151220 0x7f3468151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 --2- 172.21.15.27:0/4191044402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f346806bdf0 0x7f346814d2c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 -- 172.21.15.27:0/4191044402 >> 172.21.15.27:0/4191044402 conn(0x7f34680afbe0 msgr2=0x7f34680aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 -- 172.21.15.27:0/4191044402 shutdown_connections 2023-08-11T15:52:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.219+0000 7f346e06a640 1 -- 172.21.15.27:0/4191044402 wait complete. 2023-08-11T15:52:41.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:41.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.482+0000 7f0e8b056640 1 Processor -- start 2023-08-11T15:52:41.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.482+0000 7f0e8b056640 1 -- start start 2023-08-11T15:52:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.483+0000 7f0e8b056640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0e8415c5b0 0x7f0e840d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.484+0000 7f0e8b056640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840d0790 0x7f0e840cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.484+0000 7f0e8b056640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e840d0dc0 con 0x7f0e840cf140 2023-08-11T15:52:41.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.484+0000 7f0e8b056640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e840cf5d0 con 0x7f0e8415c5b0 2023-08-11T15:52:41.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.484+0000 7f0e8b056640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840d10a0 con 0x7f0e840d0790 2023-08-11T15:52:41.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.484+0000 7f0e8a855640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0e840cf140 0x7f0e840b3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57478/0 (socket says 172.21.15.27:57478) 2023-08-11T15:52:41.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.484+0000 7f0e8a855640 1 -- 172.21.15.27:0/2370761351 learned_addr learned my addr 172.21.15.27:0/2370761351 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:41.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.484+0000 7f0e89853640 1 --2- 172.21.15.27:0/2370761351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840d0790 0x7f0e840cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.484+0000 7f0e8a054640 1 --2- 172.21.15.27:0/2370761351 >> v2:172.21.15.119:3300/0 conn(0x7f0e8415c5b0 0x7f0e840d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.485+0000 7f0e89052640 1 -- 172.21.15.27:0/2370761351 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425185585 0 0) 0x7f0e840d0dc0 con 0x7f0e840cf140 2023-08-11T15:52:41.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.485+0000 7f0e89052640 1 -- 172.21.15.27:0/2370761351 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e70003520 con 0x7f0e840cf140 2023-08-11T15:52:41.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.485+0000 7f0e8a054640 1 -- 172.21.15.27:0/2370761351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840d0790 msgr2=0x7f0e840cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:41.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.485+0000 7f0e8a054640 1 --2- 172.21.15.27:0/2370761351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840d0790 0x7f0e840cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.485+0000 7f0e8a054640 1 -- 172.21.15.27:0/2370761351 >> v1:172.21.15.27:6789/0 conn(0x7f0e840cf140 legacy=0x7f0e840b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.485+0000 7f0e8a054640 1 -- 172.21.15.27:0/2370761351 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e840d1380 con 0x7f0e8415c5b0 2023-08-11T15:52:41.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.485+0000 7f0e89853640 1 --2- 172.21.15.27:0/2370761351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840d0790 0x7f0e840cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:41.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.487+0000 7f0e8a054640 1 --2- 172.21.15.27:0/2370761351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e8415c5b0 0x7f0e840d0250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009d90 tx=0x7f0e8002bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e6e6edbeea75285 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:41.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.487+0000 7f0e89052640 1 -- 172.21.15.27:0/2370761351 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8002ccf0 con 0x7f0e8415c5b0 2023-08-11T15:52:41.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.487+0000 7f0e89052640 1 -- 172.21.15.27:0/2370761351 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8002ce50 con 0x7f0e8415c5b0 2023-08-11T15:52:41.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.487+0000 7f0e89052640 1 -- 172.21.15.27:0/2370761351 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80034640 con 0x7f0e8415c5b0 2023-08-11T15:52:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.488+0000 7f0e8b056640 1 -- 172.21.15.27:0/2370761351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e8415c5b0 msgr2=0x7f0e840d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.488+0000 7f0e8b056640 1 --2- 172.21.15.27:0/2370761351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e8415c5b0 0x7f0e840d0250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009d90 tx=0x7f0e8002bf40 comp rx=0 tx=0).stop 2023-08-11T15:52:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.488+0000 7f0e8b056640 1 -- 172.21.15.27:0/2370761351 shutdown_connections 2023-08-11T15:52:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.488+0000 7f0e8b056640 1 --2- 172.21.15.27:0/2370761351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840d0790 0x7f0e840cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.488+0000 7f0e8b056640 1 --2- 172.21.15.27:0/2370761351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e8415c5b0 0x7f0e840d0250 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.488+0000 7f0e8b056640 1 -- 172.21.15.27:0/2370761351 >> 172.21.15.27:0/2370761351 conn(0x7f0e8406a270 msgr2=0x7f0e8406a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.488+0000 7f0e8b056640 1 -- 172.21.15.27:0/2370761351 shutdown_connections 2023-08-11T15:52:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.488+0000 7f0e8b056640 1 -- 172.21.15.27:0/2370761351 wait complete. 2023-08-11T15:52:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.490+0000 7f0e8b056640 1 Processor -- start 2023-08-11T15:52:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.490+0000 7f0e8b056640 1 -- start start 2023-08-11T15:52:41.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.491+0000 7f0e8b056640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840cf140 0x7f0e841f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.491+0000 7f0e8b056640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e840d0790 0x7f0e840c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.491+0000 7f0e8b056640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8415c5b0 0x7f0e8415ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.491+0000 7f0e8b056640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8415e790 con 0x7f0e8415c5b0 2023-08-11T15:52:41.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.491+0000 7f0e8b056640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e841f1870 con 0x7f0e840d0790 2023-08-11T15:52:41.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.491+0000 7f0e8b056640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8415db80 con 0x7f0e840cf140 2023-08-11T15:52:41.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.491+0000 7f0e8a855640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8415c5b0 0x7f0e8415ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.491+0000 7f0e8a855640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8415c5b0 0x7f0e8415ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43090/0 (socket says 172.21.15.27:43090) 2023-08-11T15:52:41.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.491+0000 7f0e8a855640 1 -- 172.21.15.27:0/728444979 learned_addr learned my addr 172.21.15.27:0/728444979 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.492+0000 7f0e89853640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e840d0790 0x7f0e840c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.492+0000 7f0e8a054640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840cf140 0x7f0e841f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.492+0000 7f0e8a855640 1 -- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840cf140 msgr2=0x7f0e841f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.492+0000 7f0e8a855640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840cf140 0x7f0e841f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.492+0000 7f0e8a855640 1 -- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e840d0790 msgr2=0x7f0e840c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.492+0000 7f0e8a855640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e840d0790 0x7f0e840c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.492+0000 7f0e8a855640 1 -- 172.21.15.27:0/728444979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e74009400 con 0x7f0e8415c5b0 2023-08-11T15:52:41.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.492+0000 7f0e8a855640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8415c5b0 0x7f0e8415ec20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c430 tx=0x7f0e7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:41.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.493+0000 7f0e7affd640 1 -- 172.21.15.27:0/728444979 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c005ce0 con 0x7f0e8415c5b0 2023-08-11T15:52:41.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.493+0000 7f0e7affd640 1 -- 172.21.15.27:0/728444979 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c005e40 con 0x7f0e8415c5b0 2023-08-11T15:52:41.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.493+0000 7f0e7affd640 1 -- 172.21.15.27:0/728444979 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c007310 con 0x7f0e8415c5b0 2023-08-11T15:52:41.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.493+0000 7f0e8b056640 1 -- 172.21.15.27:0/728444979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e80009a70 con 0x7f0e8415c5b0 2023-08-11T15:52:41.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.493+0000 7f0e8b056640 1 -- 172.21.15.27:0/728444979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8415e140 con 0x7f0e8415c5b0 2023-08-11T15:52:41.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.494+0000 7f0e8b056640 1 -- 172.21.15.27:0/728444979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e840d18f0 con 0x7f0e8415c5b0 2023-08-11T15:52:41.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.495+0000 7f0e7affd640 1 -- 172.21.15.27:0/728444979 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0e7c007470 con 0x7f0e8415c5b0 2023-08-11T15:52:41.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.496+0000 7f0e7affd640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a050 0x7f0e6806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.497+0000 7f0e7affd640 1 -- 172.21.15.27:0/728444979 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0e7c087f10 con 0x7f0e8415c5b0 2023-08-11T15:52:41.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.497+0000 7f0e8a054640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a050 0x7f0e6806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.498+0000 7f0e8a054640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a050 0x7f0e6806c510 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e80036000 tx=0x7f0e80030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:41.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.501+0000 7f0e7affd640 1 -- 172.21.15.27:0/728444979 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0e7c0532d0 con 0x7f0e8415c5b0 2023-08-11T15:52:41.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.699+0000 7f0e8b056640 1 -- 172.21.15.27:0/728444979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0e8404e420 con 0x7f0e8415c5b0 2023-08-11T15:52:41.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.700+0000 7f0e7affd640 1 -- 172.21.15.27:0/728444979 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0e7c017020 con 0x7f0e8415c5b0 2023-08-11T15:52:41.702 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.702+0000 7f0e8b056640 1 -- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a050 msgr2=0x7f0e6806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.702+0000 7f0e8b056640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a050 0x7f0e6806c510 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e80036000 tx=0x7f0e80030000 comp rx=0 tx=0).stop 2023-08-11T15:52:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.703+0000 7f0e8b056640 1 -- 172.21.15.27:0/728444979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8415c5b0 msgr2=0x7f0e8415ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.703+0000 7f0e8b056640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8415c5b0 0x7f0e8415ec20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c430 tx=0x7f0e7c00c8f0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.703+0000 7f0e8b056640 1 -- 172.21.15.27:0/728444979 shutdown_connections 2023-08-11T15:52:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.703+0000 7f0e8b056640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0e6806a050 0x7f0e6806c510 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.703+0000 7f0e8b056640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8415c5b0 0x7f0e8415ec20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.703+0000 7f0e8b056640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e840d0790 0x7f0e840c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.703+0000 7f0e8b056640 1 --2- 172.21.15.27:0/728444979 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e840cf140 0x7f0e841f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.703+0000 7f0e8b056640 1 -- 172.21.15.27:0/728444979 >> 172.21.15.27:0/728444979 conn(0x7f0e8406a270 msgr2=0x7f0e840cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.703+0000 7f0e8b056640 1 -- 172.21.15.27:0/728444979 shutdown_connections 2023-08-11T15:52:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.703+0000 7f0e8b056640 1 -- 172.21.15.27:0/728444979 wait complete. 2023-08-11T15:52:41.726 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:52:41.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.970+0000 7f50b2452640 1 Processor -- start 2023-08-11T15:52:41.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.970+0000 7f50b2452640 1 -- start start 2023-08-11T15:52:41.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.971+0000 7f50b2452640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f50ac15c3c0 0x7f50ac15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.972+0000 7f50b2452640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f50ac0cd500 0x7f50ac0d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.972+0000 7f50b2452640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ac0cf670 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.972+0000 7f50b2452640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ac160e50 con 0x7f50ac15c3c0 2023-08-11T15:52:41.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.972+0000 7f50b2452640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ac15c7a0 con 0x7f50ac0cd500 2023-08-11T15:52:41.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.972+0000 7f50b1450640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f50ac15c3c0 0x7f50ac15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.972+0000 7f50b1450640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f50ac15c3c0 0x7f50ac15feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43100/0 (socket says 172.21.15.27:43100) 2023-08-11T15:52:41.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.972+0000 7f50b1450640 1 -- 172.21.15.27:0/233076523 learned_addr learned my addr 172.21.15.27:0/233076523 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:41.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.972+0000 7f50b0c4f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f50ac0b7ab0 0x7f50ac0ccdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57494/0 (socket says 172.21.15.27:57494) 2023-08-11T15:52:41.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.972+0000 7f50b1c51640 1 --2- 172.21.15.27:0/233076523 >> v2:172.21.15.119:3300/0 conn(0x7f50ac0cd500 0x7f50ac0d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.973+0000 7f50b1450640 1 -- 172.21.15.27:0/233076523 >> v2:172.21.15.119:3300/0 conn(0x7f50ac0cd500 msgr2=0x7f50ac0d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:41.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.973+0000 7f50b1450640 1 --2- 172.21.15.27:0/233076523 >> v2:172.21.15.119:3300/0 conn(0x7f50ac0cd500 0x7f50ac0d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.973+0000 7f50b1450640 1 -- 172.21.15.27:0/233076523 >> v1:172.21.15.27:6789/0 conn(0x7f50ac0b7ab0 legacy=0x7f50ac0ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.973+0000 7f50b1450640 1 -- 172.21.15.27:0/233076523 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac1605b0 con 0x7f50ac15c3c0 2023-08-11T15:52:41.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.973+0000 7f50abfff640 1 -- 172.21.15.27:0/233076523 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865360153 0 0) 0x7f50ac0cf670 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.974+0000 7f50b1450640 1 --2- 172.21.15.27:0/233076523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac15c3c0 0x7f50ac15feb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0b5090 tx=0x7f509c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d23b80a539a241 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:41.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.974+0000 7f50abfff640 1 -- 172.21.15.27:0/233076523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c02ca80 con 0x7f50ac15c3c0 2023-08-11T15:52:41.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.974+0000 7f50abfff640 1 -- 172.21.15.27:0/233076523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c02cbe0 con 0x7f50ac15c3c0 2023-08-11T15:52:41.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.974+0000 7f50abfff640 1 -- 172.21.15.27:0/233076523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c0343f0 con 0x7f50ac15c3c0 2023-08-11T15:52:41.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.975+0000 7f50b2452640 1 -- 172.21.15.27:0/233076523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac15c3c0 msgr2=0x7f50ac15feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.975+0000 7f50b2452640 1 --2- 172.21.15.27:0/233076523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac15c3c0 0x7f50ac15feb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0b5090 tx=0x7f509c02bee0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.975+0000 7f50b2452640 1 -- 172.21.15.27:0/233076523 shutdown_connections 2023-08-11T15:52:41.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.975+0000 7f50b2452640 1 --2- 172.21.15.27:0/233076523 >> v2:172.21.15.119:3300/0 conn(0x7f50ac0cd500 0x7f50ac0d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.975+0000 7f50b2452640 1 --2- 172.21.15.27:0/233076523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac15c3c0 0x7f50ac15feb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.975+0000 7f50b2452640 1 -- 172.21.15.27:0/233076523 >> 172.21.15.27:0/233076523 conn(0x7f50ac06a190 msgr2=0x7f50ac06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:41.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.975+0000 7f50b2452640 1 -- 172.21.15.27:0/233076523 shutdown_connections 2023-08-11T15:52:41.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.975+0000 7f50b2452640 1 -- 172.21.15.27:0/233076523 wait complete. 2023-08-11T15:52:41.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.977+0000 7f50b2452640 1 Processor -- start 2023-08-11T15:52:41.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.977+0000 7f50b2452640 1 -- start start 2023-08-11T15:52:41.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.978+0000 7f50b2452640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac0b7ab0 0x7f50ac0b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.978+0000 7f50b2452640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50ac0cd500 0x7f50ac1f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.978+0000 7f50b2452640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f50ac15c3c0 0x7f50ac1f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.978+0000 7f50b2452640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0b08e0 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.978+0000 7f50b2452640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0b1b50 con 0x7f50ac0cd500 2023-08-11T15:52:41.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.978+0000 7f50b2452640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1f5a20 con 0x7f50ac15c3c0 2023-08-11T15:52:41.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.978+0000 7f50b1450640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac0b7ab0 0x7f50ac0b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.978+0000 7f50b1450640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac0b7ab0 0x7f50ac0b03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43108/0 (socket says 172.21.15.27:43108) 2023-08-11T15:52:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.978+0000 7f50b1450640 1 -- 172.21.15.27:0/981539998 learned_addr learned my addr 172.21.15.27:0/981539998 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.979+0000 7f50b0c4f640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50ac0cd500 0x7f50ac1f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.979+0000 7f50b1c51640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f50ac15c3c0 0x7f50ac1f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.979+0000 7f50b1450640 1 -- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f50ac15c3c0 msgr2=0x7f50ac1f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.979+0000 7f50b1450640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f50ac15c3c0 0x7f50ac1f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.979+0000 7f50b1450640 1 -- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50ac0cd500 msgr2=0x7f50ac1f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.979+0000 7f50b1450640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50ac0cd500 0x7f50ac1f1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.979+0000 7f50b1450640 1 -- 172.21.15.27:0/981539998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c009b30 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.979+0000 7f50b1c51640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f50ac15c3c0 0x7f50ac1f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.979+0000 7f50b1450640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac0b7ab0 0x7f50ac0b03a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f509c000f80 tx=0x7f509c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.980+0000 7f50a9ffb640 1 -- 172.21.15.27:0/981539998 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c009510 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.980+0000 7f50a9ffb640 1 -- 172.21.15.27:0/981539998 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c03e450 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.980+0000 7f50b2452640 1 -- 172.21.15.27:0/981539998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac1f5d00 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.980+0000 7f50a9ffb640 1 -- 172.21.15.27:0/981539998 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c03e600 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.980+0000 7f50b2452640 1 -- 172.21.15.27:0/981539998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac0b17e0 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.981+0000 7f50b2452640 1 -- 172.21.15.27:0/981539998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac1f5e90 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.982+0000 7f50a9ffb640 1 -- 172.21.15.27:0/981539998 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f509c02cd50 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.983+0000 7f50a9ffb640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f509806a080 0x7f509806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.983+0000 7f50a9ffb640 1 -- 172.21.15.27:0/981539998 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f509c0aafd0 con 0x7f50ac0b7ab0 2023-08-11T15:52:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.984+0000 7f50b0c4f640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f509806a080 0x7f509806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.985+0000 7f50b0c4f640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f509806a080 0x7f509806c540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f50ac1598e0 tx=0x7f50a0007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:41.987+0000 7f50a9ffb640 1 -- 172.21.15.27:0/981539998 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f509c076420 con 0x7f50ac0b7ab0 2023-08-11T15:52:42.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.193+0000 7f50b2452640 1 -- 172.21.15.27:0/981539998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f50ac066e80 con 0x7f50ac0b7ab0 2023-08-11T15:52:42.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.194+0000 7f50a9ffb640 1 -- 172.21.15.27:0/981539998 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f509c079a40 con 0x7f50ac0b7ab0 2023-08-11T15:52:42.196 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:52:42.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 -- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f509806a080 msgr2=0x7f509806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:42.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f509806a080 0x7f509806c540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f50ac1598e0 tx=0x7f50a0007c00 comp rx=0 tx=0).stop 2023-08-11T15:52:42.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 -- 172.21.15.27:0/981539998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac0b7ab0 msgr2=0x7f50ac0b03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:42.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac0b7ab0 0x7f50ac0b03a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f509c000f80 tx=0x7f509c004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:42.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 -- 172.21.15.27:0/981539998 shutdown_connections 2023-08-11T15:52:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f509806a080 0x7f509806c540 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f50ac15c3c0 0x7f50ac1f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50ac0cd500 0x7f50ac1f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 --2- 172.21.15.27:0/981539998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50ac0b7ab0 0x7f50ac0b03a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 -- 172.21.15.27:0/981539998 >> 172.21.15.27:0/981539998 conn(0x7f50ac06a190 msgr2=0x7f50ac072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 -- 172.21.15.27:0/981539998 shutdown_connections 2023-08-11T15:52:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.197+0000 7f50b2452640 1 -- 172.21.15.27:0/981539998 wait complete. 2023-08-11T15:52:42.219 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi119.front.sepia.ceph.com... 2023-08-11T15:52:42.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:52:42.220 DEBUG:teuthology.orchestra.run.smithi119:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-08-11T15:52:42.323 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:52:42.323 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:52:42.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:52:42.324 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-11T15:52:42.376 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-11T15:52:42.376 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:52:42.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.620+0000 7faafe0fa640 1 Processor -- start 2023-08-11T15:52:42.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.620+0000 7faafe0fa640 1 -- start start 2023-08-11T15:52:42.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.622+0000 7faafe0fa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faaf80cb0c0 0x7faaf815a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:42.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.622+0000 7faafe0fa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faaf80cbb70 0x7faaf80708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:42.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.622+0000 7faafe0fa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf80cc020 con 0x7faaf8156e50 2023-08-11T15:52:42.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.622+0000 7faafe0fa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf80ca460 con 0x7faaf80cb0c0 2023-08-11T15:52:42.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.622+0000 7faafe0fa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf8070470 con 0x7faaf80cbb70 2023-08-11T15:52:42.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.622+0000 7faaf77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faaf8156e50 0x7faaf80ca9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57506/0 (socket says 172.21.15.27:57506) 2023-08-11T15:52:42.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.622+0000 7faaf77fe640 1 -- 172.21.15.27:0/470933944 learned_addr learned my addr 172.21.15.27:0/470933944 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:42.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.622+0000 7faaf6ffd640 1 --2- 172.21.15.27:0/470933944 >> v2:172.21.15.27:3300/0 conn(0x7faaf80cb0c0 0x7faaf815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:42.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.623+0000 7faaf7fff640 1 --2- 172.21.15.27:0/470933944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faaf80cbb70 0x7faaf80708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:42.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.623+0000 7faaf6ffd640 1 -- 172.21.15.27:0/470933944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faaf80cbb70 msgr2=0x7faaf80708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:42.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.623+0000 7faaf6ffd640 1 --2- 172.21.15.27:0/470933944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faaf80cbb70 0x7faaf80708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.623+0000 7faaf6ffd640 1 -- 172.21.15.27:0/470933944 >> v1:172.21.15.27:6789/0 conn(0x7faaf8156e50 legacy=0x7faaf80ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:42.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.623+0000 7faaf6ffd640 1 -- 172.21.15.27:0/470933944 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf80cc270 con 0x7faaf80cb0c0 2023-08-11T15:52:42.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.623+0000 7faaf67fc640 1 -- 172.21.15.27:0/470933944 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821875436 0 0) 0x7faaf80cc020 con 0x7faaf8156e50 2023-08-11T15:52:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.624+0000 7faaf6ffd640 1 --2- 172.21.15.27:0/470933944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf80cb0c0 0x7faaf815a180 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faaec0097a0 tx=0x7faaec02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b04c4bd800f9db5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.624+0000 7faaf67fc640 1 -- 172.21.15.27:0/470933944 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec02d040 con 0x7faaf80cb0c0 2023-08-11T15:52:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.624+0000 7faaf67fc640 1 -- 172.21.15.27:0/470933944 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec031810 con 0x7faaf80cb0c0 2023-08-11T15:52:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.624+0000 7faaf67fc640 1 -- 172.21.15.27:0/470933944 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec031ba0 con 0x7faaf80cb0c0 2023-08-11T15:52:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.625+0000 7faafe0fa640 1 -- 172.21.15.27:0/470933944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf80cb0c0 msgr2=0x7faaf815a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.625+0000 7faafe0fa640 1 --2- 172.21.15.27:0/470933944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf80cb0c0 0x7faaf815a180 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faaec0097a0 tx=0x7faaec02b8b0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.625+0000 7faafe0fa640 1 -- 172.21.15.27:0/470933944 shutdown_connections 2023-08-11T15:52:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.625+0000 7faafe0fa640 1 --2- 172.21.15.27:0/470933944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faaf80cbb70 0x7faaf80708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.625+0000 7faafe0fa640 1 --2- 172.21.15.27:0/470933944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf80cb0c0 0x7faaf815a180 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.625+0000 7faafe0fa640 1 -- 172.21.15.27:0/470933944 >> 172.21.15.27:0/470933944 conn(0x7faaf80afb70 msgr2=0x7faaf80aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.625+0000 7faafe0fa640 1 -- 172.21.15.27:0/470933944 shutdown_connections 2023-08-11T15:52:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.625+0000 7faafe0fa640 1 -- 172.21.15.27:0/470933944 wait complete. 2023-08-11T15:52:42.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.627+0000 7faafe0fa640 1 Processor -- start 2023-08-11T15:52:42.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.627+0000 7faafe0fa640 1 -- start start 2023-08-11T15:52:42.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.628+0000 7faafe0fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf80cb0c0 0x7faaf814edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:42.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.628+0000 7faafe0fa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faaf80cbb70 0x7faaf80c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:42.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.628+0000 7faafe0fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf8156e50 0x7faaf80c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:42.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.628+0000 7faafe0fa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80c3fc0 con 0x7faaf8156e50 2023-08-11T15:52:42.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.628+0000 7faafe0fa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81ef850 con 0x7faaf80cb0c0 2023-08-11T15:52:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.628+0000 7faafe0fa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80ad360 con 0x7faaf80cbb70 2023-08-11T15:52:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.628+0000 7faaf7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf8156e50 0x7faaf80c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.628+0000 7faaf7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf8156e50 0x7faaf80c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43130/0 (socket says 172.21.15.27:43130) 2023-08-11T15:52:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.628+0000 7faaf7fff640 1 -- 172.21.15.27:0/332039418 learned_addr learned my addr 172.21.15.27:0/332039418 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.629+0000 7faaf77fe640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf80cb0c0 0x7faaf814edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.629+0000 7faaf6ffd640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faaf80cbb70 0x7faaf80c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.629+0000 7faaf7fff640 1 -- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faaf80cbb70 msgr2=0x7faaf80c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.629+0000 7faaf7fff640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faaf80cbb70 0x7faaf80c29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.629+0000 7faaf7fff640 1 -- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf80cb0c0 msgr2=0x7faaf814edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.629+0000 7faaf7fff640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf80cb0c0 0x7faaf814edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.629+0000 7faaf7fff640 1 -- 172.21.15.27:0/332039418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec009400 con 0x7faaf8156e50 2023-08-11T15:52:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.629+0000 7faaf7fff640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf8156e50 0x7faaf80c7080 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faaf0007e40 tx=0x7faaf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.630+0000 7faadffff640 1 -- 172.21.15.27:0/332039418 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0014620 con 0x7faaf8156e50 2023-08-11T15:52:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.630+0000 7faadffff640 1 -- 172.21.15.27:0/332039418 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf000ee90 con 0x7faaf8156e50 2023-08-11T15:52:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.630+0000 7faafe0fa640 1 -- 172.21.15.27:0/332039418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf80c4a40 con 0x7faaf8156e50 2023-08-11T15:52:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.630+0000 7faadffff640 1 -- 172.21.15.27:0/332039418 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf00149e0 con 0x7faaf8156e50 2023-08-11T15:52:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.630+0000 7faafe0fa640 1 -- 172.21.15.27:0/332039418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf80c4480 con 0x7faaf8156e50 2023-08-11T15:52:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.631+0000 7faafe0fa640 1 -- 172.21.15.27:0/332039418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf80cc810 con 0x7faaf8156e50 2023-08-11T15:52:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.632+0000 7faadffff640 1 -- 172.21.15.27:0/332039418 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faaf0014b40 con 0x7faaf8156e50 2023-08-11T15:52:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.633+0000 7faadffff640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faae406a050 0x7faae406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:42.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.633+0000 7faadffff640 1 -- 172.21.15.27:0/332039418 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7faaf00892b0 con 0x7faaf8156e50 2023-08-11T15:52:42.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.634+0000 7faaf77fe640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faae406a050 0x7faae406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.635+0000 7faaf77fe640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faae406a050 0x7faae406c510 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faae80010e0 tx=0x7faae8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:42.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.637+0000 7faadffff640 1 -- 172.21.15.27:0/332039418 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7faaf80cc810 con 0x7faaf8156e50 2023-08-11T15:52:42.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.869+0000 7faafe0fa640 1 -- 172.21.15.27:0/332039418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faaf80cc810 con 0x7faaf8156e50 2023-08-11T15:52:42.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.870+0000 7faadffff640 1 -- 172.21.15.27:0/332039418 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7faaf80cc810 con 0x7faaf8156e50 2023-08-11T15:52:42.872 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:42.872 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:51:01.220970+0000","modified":"2023-08-11T15:51:25.832166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4315":{"gid":4315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:52:42.873 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2023-08-11T15:52:42.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.873+0000 7faafe0fa640 1 -- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faae406a050 msgr2=0x7faae406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:42.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.873+0000 7faafe0fa640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faae406a050 0x7faae406c510 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faae80010e0 tx=0x7faae8009040 comp rx=0 tx=0).stop 2023-08-11T15:52:42.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.873+0000 7faafe0fa640 1 -- 172.21.15.27:0/332039418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf8156e50 msgr2=0x7faaf80c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:42.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.873+0000 7faafe0fa640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf8156e50 0x7faaf80c7080 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faaf0007e40 tx=0x7faaf000e4b0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.873+0000 7faafe0fa640 1 -- 172.21.15.27:0/332039418 shutdown_connections 2023-08-11T15:52:42.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.873+0000 7faafe0fa640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faae406a050 0x7faae406c510 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.873+0000 7faafe0fa640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faaf80cbb70 0x7faaf80c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.873+0000 7faafe0fa640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf8156e50 0x7faaf80c7080 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.873+0000 7faafe0fa640 1 --2- 172.21.15.27:0/332039418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf80cb0c0 0x7faaf814edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.873+0000 7faafe0fa640 1 -- 172.21.15.27:0/332039418 >> 172.21.15.27:0/332039418 conn(0x7faaf80afb70 msgr2=0x7faaf80c8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.874+0000 7faafe0fa640 1 -- 172.21.15.27:0/332039418 shutdown_connections 2023-08-11T15:52:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:42.874+0000 7faafe0fa640 1 -- 172.21.15.27:0/332039418 wait complete. 2023-08-11T15:52:42.892 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:43.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.102+0000 7f8692a58640 1 Processor -- start 2023-08-11T15:52:43.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.103+0000 7f8692a58640 1 -- start start 2023-08-11T15:52:43.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.104+0000 7f8692a58640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f868c0b7d40 0x7f868c0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.104+0000 7f8692a58640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868c0b5170 con 0x7f868c15c1f0 2023-08-11T15:52:43.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.104+0000 7f8692a58640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868c0b81a0 con 0x7f868c0d09f0 2023-08-11T15:52:43.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.104+0000 7f8692a58640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c15d080 con 0x7f868c0b7d40 2023-08-11T15:52:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.105+0000 7f8691a56640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f868c15c1f0 0x7f868c15c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57514/0 (socket says 172.21.15.27:57514) 2023-08-11T15:52:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.105+0000 7f8691a56640 1 -- 172.21.15.27:0/3012631643 learned_addr learned my addr 172.21.15.27:0/3012631643 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.105+0000 7f8691255640 1 --2- 172.21.15.27:0/3012631643 >> v2:172.21.15.27:3300/0 conn(0x7f868c0b7d40 0x7f868c0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.105+0000 7f8690a54640 1 -- 172.21.15.27:0/3012631643 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008686264 0 0) 0x7f868c0b5170 con 0x7f868c15c1f0 2023-08-11T15:52:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.105+0000 7f8690a54640 1 -- 172.21.15.27:0/3012631643 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8678003520 con 0x7f868c15c1f0 2023-08-11T15:52:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.105+0000 7f8691255640 1 -- 172.21.15.27:0/3012631643 >> v1:172.21.15.119:6789/0 conn(0x7f868c0d09f0 legacy=0x7f868c160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.105+0000 7f8691255640 1 -- 172.21.15.27:0/3012631643 >> v1:172.21.15.27:6789/0 conn(0x7f868c15c1f0 legacy=0x7f868c15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.105+0000 7f8690a54640 1 -- 172.21.15.27:0/3012631643 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1154443037 0 0) 0x7f8678003520 con 0x7f868c15c1f0 2023-08-11T15:52:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.105+0000 7f8691255640 1 -- 172.21.15.27:0/3012631643 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c15d360 con 0x7f868c0b7d40 2023-08-11T15:52:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.107+0000 7f8691255640 1 --2- 172.21.15.27:0/3012631643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0b7d40 0x7f868c0d04b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f867c0097a0 tx=0x7f867c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39daf542c4cced03 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.107+0000 7f8690a54640 1 -- 172.21.15.27:0/3012631643 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c02f050 con 0x7f868c0b7d40 2023-08-11T15:52:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.107+0000 7f8690a54640 1 -- 172.21.15.27:0/3012631643 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c031500 con 0x7f868c0b7d40 2023-08-11T15:52:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.107+0000 7f8690a54640 1 -- 172.21.15.27:0/3012631643 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688001670 con 0x7f868c0b7d40 2023-08-11T15:52:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.107+0000 7f8692a58640 1 -- 172.21.15.27:0/3012631643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0b7d40 msgr2=0x7f868c0d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.107+0000 7f8692a58640 1 --2- 172.21.15.27:0/3012631643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0b7d40 0x7f868c0d04b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f867c0097a0 tx=0x7f867c02b820 comp rx=0 tx=0).stop 2023-08-11T15:52:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.107+0000 7f8692a58640 1 -- 172.21.15.27:0/3012631643 shutdown_connections 2023-08-11T15:52:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.107+0000 7f8692a58640 1 --2- 172.21.15.27:0/3012631643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0b7d40 0x7f868c0d04b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.107+0000 7f8692a58640 1 -- 172.21.15.27:0/3012631643 >> 172.21.15.27:0/3012631643 conn(0x7f868c06a250 msgr2=0x7f868c06a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.108+0000 7f8692a58640 1 -- 172.21.15.27:0/3012631643 shutdown_connections 2023-08-11T15:52:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.108+0000 7f8692a58640 1 -- 172.21.15.27:0/3012631643 wait complete. 2023-08-11T15:52:43.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.109+0000 7f8692a58640 1 Processor -- start 2023-08-11T15:52:43.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.109+0000 7f8692a58640 1 -- start start 2023-08-11T15:52:43.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.110+0000 7f8692a58640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f868c0b7d40 0x7f868c1f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.110+0000 7f8692a58640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0d09f0 0x7f868c148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.110+0000 7f8692a58640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868c15c1f0 0x7f868c0c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.110+0000 7f8692a58640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1f21c0 con 0x7f868c0d09f0 2023-08-11T15:52:43.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.110+0000 7f8692a58640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1489c0 con 0x7f868c15c1f0 2023-08-11T15:52:43.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.110+0000 7f8692a58640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0ca410 con 0x7f868c0b7d40 2023-08-11T15:52:43.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.111+0000 7f8691255640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0d09f0 0x7f868c148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:43.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.111+0000 7f8691255640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0d09f0 0x7f868c148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43146/0 (socket says 172.21.15.27:43146) 2023-08-11T15:52:43.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.111+0000 7f8691255640 1 -- 172.21.15.27:0/4240700531 learned_addr learned my addr 172.21.15.27:0/4240700531 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:43.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.111+0000 7f8691a56640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f868c0b7d40 0x7f868c1f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:43.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.111+0000 7f8691255640 1 -- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f868c0b7d40 msgr2=0x7f868c1f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.111+0000 7f8691255640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f868c0b7d40 0x7f868c1f1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.111+0000 7f8691255640 1 -- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868c15c1f0 msgr2=0x7f868c0c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.111+0000 7f8691255640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868c15c1f0 0x7f868c0c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.111+0000 7f8691255640 1 -- 172.21.15.27:0/4240700531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c009400 con 0x7f868c0d09f0 2023-08-11T15:52:43.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.111+0000 7f8691255640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0d09f0 0x7f868c148480 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f867c000c00 tx=0x7f867c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:43.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.112+0000 7f86827fc640 1 -- 172.21.15.27:0/4240700531 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0339c0 con 0x7f868c0d09f0 2023-08-11T15:52:43.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.112+0000 7f86827fc640 1 -- 172.21.15.27:0/4240700531 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c033b20 con 0x7f868c0d09f0 2023-08-11T15:52:43.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.112+0000 7f86827fc640 1 -- 172.21.15.27:0/4240700531 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c03d470 con 0x7f868c0d09f0 2023-08-11T15:52:43.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.112+0000 7f8692a58640 1 -- 172.21.15.27:0/4240700531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c0ca6f0 con 0x7f868c0d09f0 2023-08-11T15:52:43.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.112+0000 7f8692a58640 1 -- 172.21.15.27:0/4240700531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c0cacb0 con 0x7f868c0d09f0 2023-08-11T15:52:43.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.113+0000 7f8692a58640 1 -- 172.21.15.27:0/4240700531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868c04e420 con 0x7f868c0d09f0 2023-08-11T15:52:43.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.114+0000 7f86827fc640 1 -- 172.21.15.27:0/4240700531 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f867c03a040 con 0x7f868c0d09f0 2023-08-11T15:52:43.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.115+0000 7f86827fc640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f867006a050 0x7f867006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.115+0000 7f86827fc640 1 -- 172.21.15.27:0/4240700531 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f867c0aa550 con 0x7f868c0d09f0 2023-08-11T15:52:43.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.116+0000 7f8691a56640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f867006a050 0x7f867006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:43.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.117+0000 7f8691a56640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f867006a050 0x7f867006c510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8688001eb0 tx=0x7f8688009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:43.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.120+0000 7f86827fc640 1 -- 172.21.15.27:0/4240700531 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f867c075a20 con 0x7f868c0d09f0 2023-08-11T15:52:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.320+0000 7f8692a58640 1 -- 172.21.15.27:0/4240700531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f868c0cfd80 con 0x7f868c0d09f0 2023-08-11T15:52:43.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.323+0000 7f86827fc640 1 -- 172.21.15.27:0/4240700531 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f867c079040 con 0x7f868c0d09f0 2023-08-11T15:52:43.326 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:43.326 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.325+0000 7f8692a58640 1 -- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f867006a050 msgr2=0x7f867006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.325+0000 7f8692a58640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f867006a050 0x7f867006c510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8688001eb0 tx=0x7f8688009040 comp rx=0 tx=0).stop 2023-08-11T15:52:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.326+0000 7f8692a58640 1 -- 172.21.15.27:0/4240700531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0d09f0 msgr2=0x7f868c148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.326+0000 7f8692a58640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0d09f0 0x7f868c148480 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f867c000c00 tx=0x7f867c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.326+0000 7f8692a58640 1 -- 172.21.15.27:0/4240700531 shutdown_connections 2023-08-11T15:52:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.326+0000 7f8692a58640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f867006a050 0x7f867006c510 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.326+0000 7f8692a58640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868c15c1f0 0x7f868c0c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.326+0000 7f8692a58640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f868c0d09f0 0x7f868c148480 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.326+0000 7f8692a58640 1 --2- 172.21.15.27:0/4240700531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f868c0b7d40 0x7f868c1f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.326+0000 7f8692a58640 1 -- 172.21.15.27:0/4240700531 >> 172.21.15.27:0/4240700531 conn(0x7f868c06a250 msgr2=0x7f868c156260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:43.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.326+0000 7f8692a58640 1 -- 172.21.15.27:0/4240700531 shutdown_connections 2023-08-11T15:52:43.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.326+0000 7f8692a58640 1 -- 172.21.15.27:0/4240700531 wait complete. 2023-08-11T15:52:43.347 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:50:59.019247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:51:00.247433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:51:03.863616+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:52:43.348 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:52:43.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.585+0000 7f863c539640 1 Processor -- start 2023-08-11T15:52:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.585+0000 7f863c539640 1 -- start start 2023-08-11T15:52:43.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.587+0000 7f863c539640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f86340cb700 0x7f863413fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.587+0000 7f863c539640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863415b6b0 0x7f86341401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.587+0000 7f863c539640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8634140ba0 con 0x7f8634154ac0 2023-08-11T15:52:43.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.587+0000 7f863c539640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8634071b10 con 0x7f86340cb700 2023-08-11T15:52:43.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.587+0000 7f863c539640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863415b220 con 0x7f863415b6b0 2023-08-11T15:52:43.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.588+0000 7f863a2d5640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8634154ac0 0x7f86340caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57520/0 (socket says 172.21.15.27:57520) 2023-08-11T15:52:43.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.588+0000 7f863a2d5640 1 -- 172.21.15.27:0/3973971484 learned_addr learned my addr 172.21.15.27:0/3973971484 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:43.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.588+0000 7f8639ad4640 1 --2- 172.21.15.27:0/3973971484 >> v2:172.21.15.27:3300/0 conn(0x7f86340cb700 0x7f863413fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:43.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.588+0000 7f863aad6640 1 --2- 172.21.15.27:0/3973971484 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863415b6b0 0x7f86341401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:43.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.588+0000 7f86392d3640 1 -- 172.21.15.27:0/3973971484 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637317630 0 0) 0x7f8634140ba0 con 0x7f8634154ac0 2023-08-11T15:52:43.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.588+0000 7f86392d3640 1 -- 172.21.15.27:0/3973971484 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8620003520 con 0x7f8634154ac0 2023-08-11T15:52:43.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.588+0000 7f8639ad4640 1 -- 172.21.15.27:0/3973971484 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863415b6b0 msgr2=0x7f86341401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.588+0000 7f8639ad4640 1 --2- 172.21.15.27:0/3973971484 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863415b6b0 0x7f86341401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.588+0000 7f8639ad4640 1 -- 172.21.15.27:0/3973971484 >> v1:172.21.15.27:6789/0 conn(0x7f8634154ac0 legacy=0x7f86340caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.588+0000 7f8639ad4640 1 -- 172.21.15.27:0/3973971484 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86341408c0 con 0x7f86340cb700 2023-08-11T15:52:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.589+0000 7f8639ad4640 1 --2- 172.21.15.27:0/3973971484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 0x7f863413fc90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f86280097a0 tx=0x7f862802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873800b4be8ee7eb server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:43.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.590+0000 7f86392d3640 1 -- 172.21.15.27:0/3973971484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628031780 con 0x7f86340cb700 2023-08-11T15:52:43.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.590+0000 7f86392d3640 1 -- 172.21.15.27:0/3973971484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86280318e0 con 0x7f86340cb700 2023-08-11T15:52:43.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.590+0000 7f86392d3640 1 -- 172.21.15.27:0/3973971484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628031c50 con 0x7f86340cb700 2023-08-11T15:52:43.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.590+0000 7f863c539640 1 -- 172.21.15.27:0/3973971484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 msgr2=0x7f863413fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.590+0000 7f863c539640 1 --2- 172.21.15.27:0/3973971484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 0x7f863413fc90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f86280097a0 tx=0x7f862802b820 comp rx=0 tx=0).stop 2023-08-11T15:52:43.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.590+0000 7f863c539640 1 -- 172.21.15.27:0/3973971484 shutdown_connections 2023-08-11T15:52:43.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.590+0000 7f863c539640 1 --2- 172.21.15.27:0/3973971484 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863415b6b0 0x7f86341401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.590+0000 7f863c539640 1 --2- 172.21.15.27:0/3973971484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 0x7f863413fc90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.590+0000 7f863c539640 1 -- 172.21.15.27:0/3973971484 >> 172.21.15.27:0/3973971484 conn(0x7f8634069bf0 msgr2=0x7f863406a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:43.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.591+0000 7f863c539640 1 -- 172.21.15.27:0/3973971484 shutdown_connections 2023-08-11T15:52:43.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.591+0000 7f863c539640 1 -- 172.21.15.27:0/3973971484 wait complete. 2023-08-11T15:52:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.592+0000 7f863c539640 1 Processor -- start 2023-08-11T15:52:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.593+0000 7f863c539640 1 -- start start 2023-08-11T15:52:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863c539640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 0x7f8634153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863c539640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8634154ac0 0x7f86340c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863c539640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863415b6b0 0x7f86340ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863c539640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86340b23a0 con 0x7f86340cb700 2023-08-11T15:52:43.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863c539640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86340aea10 con 0x7f8634154ac0 2023-08-11T15:52:43.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863c539640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86340b1a00 con 0x7f863415b6b0 2023-08-11T15:52:43.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863a2d5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 0x7f8634153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:43.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863a2d5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 0x7f8634153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43162/0 (socket says 172.21.15.27:43162) 2023-08-11T15:52:43.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863a2d5640 1 -- 172.21.15.27:0/2780730116 learned_addr learned my addr 172.21.15.27:0/2780730116 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:43.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f8639ad4640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8634154ac0 0x7f86340c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:43.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863a2d5640 1 -- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863415b6b0 msgr2=0x7f86340ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863a2d5640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863415b6b0 0x7f86340ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863a2d5640 1 -- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8634154ac0 msgr2=0x7f86340c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863a2d5640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8634154ac0 0x7f86340c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.594+0000 7f863a2d5640 1 -- 172.21.15.27:0/2780730116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628009400 con 0x7f86340cb700 2023-08-11T15:52:43.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.595+0000 7f863a2d5640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 0x7f8634153760 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f863000c430 tx=0x7f863000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:43.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.595+0000 7f8626ffd640 1 -- 172.21.15.27:0/2780730116 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630005ce0 con 0x7f86340cb700 2023-08-11T15:52:43.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.595+0000 7f8626ffd640 1 -- 172.21.15.27:0/2780730116 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8630005e40 con 0x7f86340cb700 2023-08-11T15:52:43.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.595+0000 7f8626ffd640 1 -- 172.21.15.27:0/2780730116 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300072f0 con 0x7f86340cb700 2023-08-11T15:52:43.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.595+0000 7f863c539640 1 -- 172.21.15.27:0/2780730116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86340b1ce0 con 0x7f86340cb700 2023-08-11T15:52:43.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.595+0000 7f863c539640 1 -- 172.21.15.27:0/2780730116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86340cc050 con 0x7f86340cb700 2023-08-11T15:52:43.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.596+0000 7f863c539640 1 -- 172.21.15.27:0/2780730116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86340b24e0 con 0x7f86340cb700 2023-08-11T15:52:43.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.598+0000 7f8626ffd640 1 -- 172.21.15.27:0/2780730116 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8630007450 con 0x7f86340cb700 2023-08-11T15:52:43.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.599+0000 7f8626ffd640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f861806a050 0x7f861806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:43.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.599+0000 7f8626ffd640 1 -- 172.21.15.27:0/2780730116 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8630087db0 con 0x7f86340cb700 2023-08-11T15:52:43.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.600+0000 7f8639ad4640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f861806a050 0x7f861806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.601+0000 7f8639ad4640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f861806a050 0x7f861806c510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8628036000 tx=0x7f862803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.602+0000 7f8626ffd640 1 -- 172.21.15.27:0/2780730116 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f8630053260 con 0x7f86340cb700 2023-08-11T15:52:43.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.789+0000 7f863c539640 1 -- 172.21.15.27:0/2780730116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86340b24e0 con 0x7f86340cb700 2023-08-11T15:52:43.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.790+0000 7f8626ffd640 1 -- 172.21.15.27:0/2780730116 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86340b24e0 con 0x7f86340cb700 2023-08-11T15:52:43.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:52:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.792+0000 7f863c539640 1 -- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f861806a050 msgr2=0x7f861806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.792+0000 7f863c539640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f861806a050 0x7f861806c510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8628036000 tx=0x7f862803d040 comp rx=0 tx=0).stop 2023-08-11T15:52:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.792+0000 7f863c539640 1 -- 172.21.15.27:0/2780730116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 msgr2=0x7f8634153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.792+0000 7f863c539640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 0x7f8634153760 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f863000c430 tx=0x7f863000c8f0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.792+0000 7f863c539640 1 -- 172.21.15.27:0/2780730116 shutdown_connections 2023-08-11T15:52:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.792+0000 7f863c539640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f861806a050 0x7f861806c510 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.792+0000 7f863c539640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863415b6b0 0x7f86340ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.792+0000 7f863c539640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8634154ac0 0x7f86340c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.792+0000 7f863c539640 1 --2- 172.21.15.27:0/2780730116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86340cb700 0x7f8634153760 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.792+0000 7f863c539640 1 -- 172.21.15.27:0/2780730116 >> 172.21.15.27:0/2780730116 conn(0x7f8634069bf0 msgr2=0x7f86340c89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.793+0000 7f863c539640 1 -- 172.21.15.27:0/2780730116 shutdown_connections 2023-08-11T15:52:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:43.793+0000 7f863c539640 1 -- 172.21.15.27:0/2780730116 wait complete. 2023-08-11T15:52:43.813 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:52:44.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.047+0000 7fc4f8eec640 1 Processor -- start 2023-08-11T15:52:44.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.047+0000 7fc4f8eec640 1 -- start start 2023-08-11T15:52:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.049+0000 7fc4f8eec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4f4154ac0 0x7fc4f40caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.049+0000 7fc4f8eec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f415b6b0 0x7fc4f40cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.049+0000 7fc4f8eec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f4140ba0 con 0x7fc4f413fc90 2023-08-11T15:52:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.049+0000 7fc4f8eec640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f4071b10 con 0x7fc4f4154ac0 2023-08-11T15:52:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.049+0000 7fc4f8eec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f415b220 con 0x7fc4f415b6b0 2023-08-11T15:52:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.050+0000 7fc4f2d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc4f413fc90 0x7fc4f40b2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57522/0 (socket says 172.21.15.27:57522) 2023-08-11T15:52:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.050+0000 7fc4f2d9d640 1 -- 172.21.15.27:0/1882625916 learned_addr learned my addr 172.21.15.27:0/1882625916 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:44.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.050+0000 7fc4f259c640 1 --2- 172.21.15.27:0/1882625916 >> v2:172.21.15.119:3300/0 conn(0x7fc4f4154ac0 0x7fc4f40caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.050+0000 7fc4ebfff640 1 --2- 172.21.15.27:0/1882625916 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f415b6b0 0x7fc4f40cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.050+0000 7fc4f1d9b640 1 -- 172.21.15.27:0/1882625916 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312307574 0 0) 0x7fc4f4140ba0 con 0x7fc4f413fc90 2023-08-11T15:52:44.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.050+0000 7fc4f1d9b640 1 -- 172.21.15.27:0/1882625916 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e0003520 con 0x7fc4f413fc90 2023-08-11T15:52:44.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.051+0000 7fc4f1d9b640 1 -- 172.21.15.27:0/1882625916 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3868990157 0 0) 0x7fc4e0003520 con 0x7fc4f413fc90 2023-08-11T15:52:44.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.051+0000 7fc4ebfff640 1 -- 172.21.15.27:0/1882625916 >> v2:172.21.15.119:3300/0 conn(0x7fc4f4154ac0 msgr2=0x7fc4f40caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:44.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.051+0000 7fc4ebfff640 1 --2- 172.21.15.27:0/1882625916 >> v2:172.21.15.119:3300/0 conn(0x7fc4f4154ac0 0x7fc4f40caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.051+0000 7fc4ebfff640 1 -- 172.21.15.27:0/1882625916 >> v1:172.21.15.27:6789/0 conn(0x7fc4f413fc90 legacy=0x7fc4f40b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.051+0000 7fc4ebfff640 1 -- 172.21.15.27:0/1882625916 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f41408c0 con 0x7fc4f415b6b0 2023-08-11T15:52:44.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.052+0000 7fc4ebfff640 1 --2- 172.21.15.27:0/1882625916 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f415b6b0 0x7fc4f40cb530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0097d0 tx=0x7fc4dc02b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e266eaca9be93867 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:44.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.052+0000 7fc4f1d9b640 1 -- 172.21.15.27:0/1882625916 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0019f0 con 0x7fc4f415b6b0 2023-08-11T15:52:44.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.053+0000 7fc4f1d9b640 1 -- 172.21.15.27:0/1882625916 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc02e050 con 0x7fc4f415b6b0 2023-08-11T15:52:44.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.053+0000 7fc4f1d9b640 1 -- 172.21.15.27:0/1882625916 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc030710 con 0x7fc4f415b6b0 2023-08-11T15:52:44.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.053+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1882625916 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f415b6b0 msgr2=0x7fc4f40cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.053+0000 7fc4f8eec640 1 --2- 172.21.15.27:0/1882625916 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f415b6b0 0x7fc4f40cb530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0097d0 tx=0x7fc4dc02b6f0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.053+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1882625916 shutdown_connections 2023-08-11T15:52:44.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.053+0000 7fc4f8eec640 1 --2- 172.21.15.27:0/1882625916 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f415b6b0 0x7fc4f40cb530 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.053+0000 7fc4f8eec640 1 --2- 172.21.15.27:0/1882625916 >> v2:172.21.15.119:3300/0 conn(0x7fc4f4154ac0 0x7fc4f40caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.053+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1882625916 >> 172.21.15.27:0/1882625916 conn(0x7fc4f4069bf0 msgr2=0x7fc4f406a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:44.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.053+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1882625916 shutdown_connections 2023-08-11T15:52:44.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.053+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1882625916 wait complete. 2023-08-11T15:52:44.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.055+0000 7fc4f8eec640 1 Processor -- start 2023-08-11T15:52:44.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.055+0000 7fc4f8eec640 1 -- start start 2023-08-11T15:52:44.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.056+0000 7fc4f8eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f413fc90 0x7fc4f40c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.056+0000 7fc4f8eec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f4154ac0 0x7fc4f40adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.056+0000 7fc4f8eec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4f415b6b0 0x7fc4f40adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.056+0000 7fc4f8eec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f415eec0 con 0x7fc4f415b6b0 2023-08-11T15:52:44.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.056+0000 7fc4f8eec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f40b2740 con 0x7fc4f413fc90 2023-08-11T15:52:44.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.056+0000 7fc4f8eec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f40b00b0 con 0x7fc4f4154ac0 2023-08-11T15:52:44.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.057+0000 7fc4f2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4f415b6b0 0x7fc4f40adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.057+0000 7fc4f2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4f415b6b0 0x7fc4f40adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43172/0 (socket says 172.21.15.27:43172) 2023-08-11T15:52:44.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.057+0000 7fc4f2d9d640 1 -- 172.21.15.27:0/1069059736 learned_addr learned my addr 172.21.15.27:0/1069059736 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:44.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.057+0000 7fc4ebfff640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f4154ac0 0x7fc4f40adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.057+0000 7fc4f259c640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f413fc90 0x7fc4f40c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.057+0000 7fc4f2d9d640 1 -- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f4154ac0 msgr2=0x7fc4f40adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.057+0000 7fc4f2d9d640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f4154ac0 0x7fc4f40adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.057+0000 7fc4f2d9d640 1 -- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f413fc90 msgr2=0x7fc4f40c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.057+0000 7fc4f2d9d640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f413fc90 0x7fc4f40c5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.057+0000 7fc4f2d9d640 1 -- 172.21.15.27:0/1069059736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc009400 con 0x7fc4f415b6b0 2023-08-11T15:52:44.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.058+0000 7fc4f2d9d640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4f415b6b0 0x7fc4f40adff0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00cc00 tx=0x7fc4ec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:44.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.058+0000 7fc4eaffd640 1 -- 172.21.15.27:0/1069059736 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00ce10 con 0x7fc4f415b6b0 2023-08-11T15:52:44.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.058+0000 7fc4eaffd640 1 -- 172.21.15.27:0/1069059736 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec012400 con 0x7fc4f415b6b0 2023-08-11T15:52:44.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.058+0000 7fc4eaffd640 1 -- 172.21.15.27:0/1069059736 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0127b0 con 0x7fc4f415b6b0 2023-08-11T15:52:44.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.058+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1069059736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f40b0390 con 0x7fc4f415b6b0 2023-08-11T15:52:44.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.058+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1069059736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f40b0950 con 0x7fc4f415b6b0 2023-08-11T15:52:44.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.059+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1069059736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f404e420 con 0x7fc4f415b6b0 2023-08-11T15:52:44.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.060+0000 7fc4eaffd640 1 -- 172.21.15.27:0/1069059736 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc4ec012910 con 0x7fc4f415b6b0 2023-08-11T15:52:44.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.062+0000 7fc4eaffd640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc4d406a050 0x7fc4d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.062+0000 7fc4eaffd640 1 -- 172.21.15.27:0/1069059736 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc4ec088fc0 con 0x7fc4f415b6b0 2023-08-11T15:52:44.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.063+0000 7fc4f259c640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc4d406a050 0x7fc4d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.064+0000 7fc4f259c640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc4d406a050 0x7fc4d406c510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009ad0 tx=0x7fc4e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:44.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.067+0000 7fc4eaffd640 1 -- 172.21.15.27:0/1069059736 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fc4ec0bb7c0 con 0x7fc4f415b6b0 2023-08-11T15:52:44.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.265+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1069059736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc4f406c020 con 0x7fc4f415b6b0 2023-08-11T15:52:44.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.266+0000 7fc4eaffd640 1 -- 172.21.15.27:0/1069059736 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc4ec054410 con 0x7fc4f415b6b0 2023-08-11T15:52:44.268 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:52:44.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc4d406a050 msgr2=0x7fc4d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc4d406a050 0x7fc4d406c510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009ad0 tx=0x7fc4e4008040 comp rx=0 tx=0).stop 2023-08-11T15:52:44.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1069059736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4f415b6b0 msgr2=0x7fc4f40adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4f415b6b0 0x7fc4f40adff0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00cc00 tx=0x7fc4ec00a430 comp rx=0 tx=0).stop 2023-08-11T15:52:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1069059736 shutdown_connections 2023-08-11T15:52:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc4d406a050 0x7fc4d406c510 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4f4154ac0 0x7fc4f40adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4f415b6b0 0x7fc4f40adff0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 --2- 172.21.15.27:0/1069059736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4f413fc90 0x7fc4f40c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1069059736 >> 172.21.15.27:0/1069059736 conn(0x7fc4f4069bf0 msgr2=0x7fc4f40c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1069059736 shutdown_connections 2023-08-11T15:52:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.269+0000 7fc4f8eec640 1 -- 172.21.15.27:0/1069059736 wait complete. 2023-08-11T15:52:44.290 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:52:44.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.527+0000 7f5831ddd640 1 Processor -- start 2023-08-11T15:52:44.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.527+0000 7f5831ddd640 1 -- start start 2023-08-11T15:52:44.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5831ddd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f582c15c5b0 0x7f582c0d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5831ddd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f582c0d0790 0x7f582c0cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5831ddd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f582c0cf140 0x7f582c0b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5831ddd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c0cf5d0 con 0x7f582c15c5b0 2023-08-11T15:52:44.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5831ddd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c0d0dc0 con 0x7f582c0d0790 2023-08-11T15:52:44.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5831ddd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0d10a0 con 0x7f582c0cf140 2023-08-11T15:52:44.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5830ddb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f582c15c5b0 0x7f582c0d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5830ddb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f582c15c5b0 0x7f582c0d0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43176/0 (socket says 172.21.15.27:43176) 2023-08-11T15:52:44.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5830ddb640 1 -- 172.21.15.27:0/2846349232 learned_addr learned my addr 172.21.15.27:0/2846349232 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:44.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5823fff640 1 --2- 172.21.15.27:0/2846349232 >> v2:172.21.15.119:3300/0 conn(0x7f582c0d0790 0x7f582c0cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f58315dc640 1 --2- 172.21.15.27:0/2846349232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f582c0cf140 0x7f582c0b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5830ddb640 1 -- 172.21.15.27:0/2846349232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f582c0cf140 msgr2=0x7f582c0b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:44.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5830ddb640 1 --2- 172.21.15.27:0/2846349232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f582c0cf140 0x7f582c0b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5830ddb640 1 -- 172.21.15.27:0/2846349232 >> v2:172.21.15.119:3300/0 conn(0x7f582c0d0790 msgr2=0x7f582c0cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:44.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5830ddb640 1 --2- 172.21.15.27:0/2846349232 >> v2:172.21.15.119:3300/0 conn(0x7f582c0d0790 0x7f582c0cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.529+0000 7f5830ddb640 1 -- 172.21.15.27:0/2846349232 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0d1380 con 0x7f582c15c5b0 2023-08-11T15:52:44.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.531+0000 7f5830ddb640 1 --2- 172.21.15.27:0/2846349232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c15c5b0 0x7f582c0d0250 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f582c0cf850 tx=0x7f582802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6bd2b9ce03dd70 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:44.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.531+0000 7f58237fe640 1 -- 172.21.15.27:0/2846349232 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828030850 con 0x7f582c15c5b0 2023-08-11T15:52:44.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.531+0000 7f58237fe640 1 -- 172.21.15.27:0/2846349232 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58280309b0 con 0x7f582c15c5b0 2023-08-11T15:52:44.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.531+0000 7f58237fe640 1 -- 172.21.15.27:0/2846349232 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828030d40 con 0x7f582c15c5b0 2023-08-11T15:52:44.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.532+0000 7f5831ddd640 1 -- 172.21.15.27:0/2846349232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c15c5b0 msgr2=0x7f582c0d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.532+0000 7f5831ddd640 1 --2- 172.21.15.27:0/2846349232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c15c5b0 0x7f582c0d0250 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f582c0cf850 tx=0x7f582802b990 comp rx=0 tx=0).stop 2023-08-11T15:52:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.532+0000 7f5831ddd640 1 -- 172.21.15.27:0/2846349232 shutdown_connections 2023-08-11T15:52:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.532+0000 7f5831ddd640 1 --2- 172.21.15.27:0/2846349232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f582c0cf140 0x7f582c0b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.532+0000 7f5831ddd640 1 --2- 172.21.15.27:0/2846349232 >> v2:172.21.15.119:3300/0 conn(0x7f582c0d0790 0x7f582c0cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.532+0000 7f5831ddd640 1 --2- 172.21.15.27:0/2846349232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c15c5b0 0x7f582c0d0250 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.532+0000 7f5831ddd640 1 -- 172.21.15.27:0/2846349232 >> 172.21.15.27:0/2846349232 conn(0x7f582c06a270 msgr2=0x7f582c06a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:44.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.532+0000 7f5831ddd640 1 -- 172.21.15.27:0/2846349232 shutdown_connections 2023-08-11T15:52:44.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.532+0000 7f5831ddd640 1 -- 172.21.15.27:0/2846349232 wait complete. 2023-08-11T15:52:44.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.534+0000 7f5831ddd640 1 Processor -- start 2023-08-11T15:52:44.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.534+0000 7f5831ddd640 1 -- start start 2023-08-11T15:52:44.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.535+0000 7f5831ddd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c0cf140 0x7f582c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.535+0000 7f5831ddd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f582c0d0790 0x7f582c1f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.535+0000 7f5831ddd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f582c15c5b0 0x7f582c1f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.535+0000 7f5831ddd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c15f0b0 con 0x7f582c0cf140 2023-08-11T15:52:44.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.535+0000 7f5831ddd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c15e4a0 con 0x7f582c15c5b0 2023-08-11T15:52:44.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.535+0000 7f5831ddd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c15e780 con 0x7f582c0d0790 2023-08-11T15:52:44.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.535+0000 7f5830ddb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c0cf140 0x7f582c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.535+0000 7f5830ddb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c0cf140 0x7f582c15d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43188/0 (socket says 172.21.15.27:43188) 2023-08-11T15:52:44.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.535+0000 7f5830ddb640 1 -- 172.21.15.27:0/29495130 learned_addr learned my addr 172.21.15.27:0/29495130 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:44.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f58315dc640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f582c15c5b0 0x7f582c1f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f5823fff640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f582c0d0790 0x7f582c1f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f5830ddb640 1 -- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f582c0d0790 msgr2=0x7f582c1f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f5830ddb640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f582c0d0790 0x7f582c1f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f5830ddb640 1 -- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f582c15c5b0 msgr2=0x7f582c1f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f5830ddb640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f582c15c5b0 0x7f582c1f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f5830ddb640 1 -- 172.21.15.27:0/29495130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58280094d0 con 0x7f582c0cf140 2023-08-11T15:52:44.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f5830ddb640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c0cf140 0x7f582c15d9e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5828000c00 tx=0x7f5828004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:44.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f58217fa640 1 -- 172.21.15.27:0/29495130 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828039b60 con 0x7f582c0cf140 2023-08-11T15:52:44.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f58217fa640 1 -- 172.21.15.27:0/29495130 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5828039cc0 con 0x7f582c0cf140 2023-08-11T15:52:44.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f5831ddd640 1 -- 172.21.15.27:0/29495130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c15ea60 con 0x7f582c0cf140 2023-08-11T15:52:44.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.536+0000 7f58217fa640 1 -- 172.21.15.27:0/29495130 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828043610 con 0x7f582c0cf140 2023-08-11T15:52:44.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.537+0000 7f5831ddd640 1 -- 172.21.15.27:0/29495130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c1542e0 con 0x7f582c0cf140 2023-08-11T15:52:44.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.538+0000 7f5831ddd640 1 -- 172.21.15.27:0/29495130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c04e420 con 0x7f582c0cf140 2023-08-11T15:52:44.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.539+0000 7f58217fa640 1 -- 172.21.15.27:0/29495130 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f582802e070 con 0x7f582c0cf140 2023-08-11T15:52:44.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.540+0000 7f58217fa640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f581806a050 0x7f581806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.540+0000 7f58217fa640 1 -- 172.21.15.27:0/29495130 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f58280aa930 con 0x7f582c0cf140 2023-08-11T15:52:44.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.541+0000 7f5823fff640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f581806a050 0x7f581806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.542+0000 7f5823fff640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f581806a050 0x7f581806c510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f581c006fd0 tx=0x7f581c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:44.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.544+0000 7f58217fa640 1 -- 172.21.15.27:0/29495130 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f5828075cf0 con 0x7f582c0cf140 2023-08-11T15:52:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.729+0000 7f5831ddd640 1 -- 172.21.15.27:0/29495130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f582c0cd420 con 0x7f582c0cf140 2023-08-11T15:52:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.730+0000 7f58217fa640 1 -- 172.21.15.27:0/29495130 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5828079310 con 0x7f582c0cf140 2023-08-11T15:52:44.731 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:44.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.732+0000 7f5831ddd640 1 -- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f581806a050 msgr2=0x7f581806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.732+0000 7f5831ddd640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f581806a050 0x7f581806c510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f581c006fd0 tx=0x7f581c008040 comp rx=0 tx=0).stop 2023-08-11T15:52:44.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.732+0000 7f5831ddd640 1 -- 172.21.15.27:0/29495130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c0cf140 msgr2=0x7f582c15d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.732+0000 7f5831ddd640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c0cf140 0x7f582c15d9e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5828000c00 tx=0x7f5828004730 comp rx=0 tx=0).stop 2023-08-11T15:52:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.732+0000 7f5831ddd640 1 -- 172.21.15.27:0/29495130 shutdown_connections 2023-08-11T15:52:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.732+0000 7f5831ddd640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f581806a050 0x7f581806c510 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.732+0000 7f5831ddd640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f582c0d0790 0x7f582c1f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.732+0000 7f5831ddd640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f582c15c5b0 0x7f582c1f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.732+0000 7f5831ddd640 1 --2- 172.21.15.27:0/29495130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f582c0cf140 0x7f582c15d9e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.732+0000 7f5831ddd640 1 -- 172.21.15.27:0/29495130 >> 172.21.15.27:0/29495130 conn(0x7f582c06a270 msgr2=0x7f582c0b6210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.733+0000 7f5831ddd640 1 -- 172.21.15.27:0/29495130 shutdown_connections 2023-08-11T15:52:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.733+0000 7f5831ddd640 1 -- 172.21.15.27:0/29495130 wait complete. 2023-08-11T15:52:44.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:52:44.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.986+0000 7f911bfff640 1 Processor -- start 2023-08-11T15:52:44.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.986+0000 7f911bfff640 1 -- start start 2023-08-11T15:52:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.988+0000 7f911bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f911c06c960 0x7f911c06cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.988+0000 7f911bfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c0b01c0 con 0x7f911c1592e0 2023-08-11T15:52:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.988+0000 7f911bfff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c0b0420 con 0x7f911c0ccbc0 2023-08-11T15:52:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.988+0000 7f911bfff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911c1596c0 con 0x7f911c06c960 2023-08-11T15:52:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.988+0000 7f911affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f911c1592e0 0x7f911c15d730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57536/0 (socket says 172.21.15.27:57536) 2023-08-11T15:52:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.988+0000 7f911affd640 1 -- 172.21.15.27:0/950004662 learned_addr learned my addr 172.21.15.27:0/950004662 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.988+0000 7f911b7fe640 1 --2- 172.21.15.27:0/950004662 >> v2:172.21.15.27:3300/0 conn(0x7f911c06c960 0x7f911c06cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.989+0000 7f9119ffb640 1 -- 172.21.15.27:0/950004662 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156125109 0 0) 0x7f911c0b01c0 con 0x7f911c1592e0 2023-08-11T15:52:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.989+0000 7f9119ffb640 1 -- 172.21.15.27:0/950004662 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9104005730 con 0x7f911c1592e0 2023-08-11T15:52:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.989+0000 7f911b7fe640 1 -- 172.21.15.27:0/950004662 >> v1:172.21.15.119:6789/0 conn(0x7f911c0ccbc0 legacy=0x7f911c0afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.989+0000 7f911b7fe640 1 -- 172.21.15.27:0/950004662 >> v1:172.21.15.27:6789/0 conn(0x7f911c1592e0 legacy=0x7f911c15d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.989+0000 7f911b7fe640 1 -- 172.21.15.27:0/950004662 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0b0700 con 0x7f911c06c960 2023-08-11T15:52:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.990+0000 7f911b7fe640 1 --2- 172.21.15.27:0/950004662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 0x7f911c06cdf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f91140097a0 tx=0x7f911402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778a8328ef29a7b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.990+0000 7f9119ffb640 1 -- 172.21.15.27:0/950004662 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911402e070 con 0x7f911c06c960 2023-08-11T15:52:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.990+0000 7f9119ffb640 1 -- 172.21.15.27:0/950004662 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91140315a0 con 0x7f911c06c960 2023-08-11T15:52:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.990+0000 7f9119ffb640 1 -- 172.21.15.27:0/950004662 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114031930 con 0x7f911c06c960 2023-08-11T15:52:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.991+0000 7f911bfff640 1 -- 172.21.15.27:0/950004662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 msgr2=0x7f911c06cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.991+0000 7f911bfff640 1 --2- 172.21.15.27:0/950004662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 0x7f911c06cdf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f91140097a0 tx=0x7f911402b840 comp rx=0 tx=0).stop 2023-08-11T15:52:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.991+0000 7f911bfff640 1 -- 172.21.15.27:0/950004662 shutdown_connections 2023-08-11T15:52:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.991+0000 7f911bfff640 1 --2- 172.21.15.27:0/950004662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 0x7f911c06cdf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.991+0000 7f911bfff640 1 -- 172.21.15.27:0/950004662 >> 172.21.15.27:0/950004662 conn(0x7f911c0caab0 msgr2=0x7f911c0cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.991+0000 7f911bfff640 1 -- 172.21.15.27:0/950004662 shutdown_connections 2023-08-11T15:52:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.991+0000 7f911bfff640 1 -- 172.21.15.27:0/950004662 wait complete. 2023-08-11T15:52:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.993+0000 7f911bfff640 1 Processor -- start 2023-08-11T15:52:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.993+0000 7f911bfff640 1 -- start start 2023-08-11T15:52:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.994+0000 7f911bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 0x7f911c0af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.994+0000 7f911bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911c0ccbc0 0x7f911c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.994+0000 7f911bfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f911c1592e0 0x7f911c1480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.994+0000 7f911bfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0c6c70 con 0x7f911c06c960 2023-08-11T15:52:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.994+0000 7f911bfff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0b2080 con 0x7f911c0ccbc0 2023-08-11T15:52:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.994+0000 7f911bfff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0b2360 con 0x7f911c1592e0 2023-08-11T15:52:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.995+0000 7f911affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 0x7f911c0af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.995+0000 7f911affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 0x7f911c0af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43208/0 (socket says 172.21.15.27:43208) 2023-08-11T15:52:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.995+0000 7f911affd640 1 -- 172.21.15.27:0/952398081 learned_addr learned my addr 172.21.15.27:0/952398081 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.995+0000 7f911a7fc640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911c0ccbc0 0x7f911c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.995+0000 7f911affd640 1 -- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f911c1592e0 msgr2=0x7f911c1480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.995+0000 7f911affd640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f911c1592e0 0x7f911c1480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.995+0000 7f911affd640 1 -- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911c0ccbc0 msgr2=0x7f911c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.995+0000 7f911affd640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911c0ccbc0 0x7f911c0b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.995+0000 7f911affd640 1 -- 172.21.15.27:0/952398081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9114009400 con 0x7f911c06c960 2023-08-11T15:52:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.995+0000 7f911affd640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 0x7f911c0af220 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f910c00cbc0 tx=0x7f910c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.996+0000 7f91037fe640 1 -- 172.21.15.27:0/952398081 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c00ad40 con 0x7f911c06c960 2023-08-11T15:52:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.996+0000 7f91037fe640 1 -- 172.21.15.27:0/952398081 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c012400 con 0x7f911c06c960 2023-08-11T15:52:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.996+0000 7f91037fe640 1 -- 172.21.15.27:0/952398081 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c00aea0 con 0x7f911c06c960 2023-08-11T15:52:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.996+0000 7f911bfff640 1 -- 172.21.15.27:0/952398081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c0c65b0 con 0x7f911c06c960 2023-08-11T15:52:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.996+0000 7f911bfff640 1 -- 172.21.15.27:0/952398081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c158930 con 0x7f911c06c960 2023-08-11T15:52:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.997+0000 7f911bfff640 1 -- 172.21.15.27:0/952398081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c04e420 con 0x7f911c06c960 2023-08-11T15:52:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.998+0000 7f91037fe640 1 -- 172.21.15.27:0/952398081 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f910c007070 con 0x7f911c06c960 2023-08-11T15:52:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.999+0000 7f91037fe640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f910806a050 0x7f910806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:44.999+0000 7f91037fe640 1 -- 172.21.15.27:0/952398081 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f910c0889c0 con 0x7f911c06c960 2023-08-11T15:52:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.000+0000 7f911a7fc640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f910806a050 0x7f910806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.001+0000 7f911a7fc640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f910806a050 0x7f910806c510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f911c0c61c0 tx=0x7f9110009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.004+0000 7f91037fe640 1 -- 172.21.15.27:0/952398081 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f910c053c50 con 0x7f911c06c960 2023-08-11T15:52:45.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.199+0000 7f911bfff640 1 -- 172.21.15.27:0/952398081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f911c0cc910 con 0x7f911c06c960 2023-08-11T15:52:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.200+0000 7f91037fe640 1 -- 172.21.15.27:0/952398081 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f910c017020 con 0x7f911c06c960 2023-08-11T15:52:45.201 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:52:45.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.202+0000 7f911bfff640 1 -- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f910806a050 msgr2=0x7f910806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.202+0000 7f911bfff640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f910806a050 0x7f910806c510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f911c0c61c0 tx=0x7f9110009040 comp rx=0 tx=0).stop 2023-08-11T15:52:45.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.202+0000 7f911bfff640 1 -- 172.21.15.27:0/952398081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 msgr2=0x7f911c0af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.202+0000 7f911bfff640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 0x7f911c0af220 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f910c00cbc0 tx=0x7f910c00a430 comp rx=0 tx=0).stop 2023-08-11T15:52:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.202+0000 7f911bfff640 1 -- 172.21.15.27:0/952398081 shutdown_connections 2023-08-11T15:52:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.202+0000 7f911bfff640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f910806a050 0x7f910806c510 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.202+0000 7f911bfff640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f911c1592e0 0x7f911c1480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.202+0000 7f911bfff640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911c0ccbc0 0x7f911c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.202+0000 7f911bfff640 1 --2- 172.21.15.27:0/952398081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f911c06c960 0x7f911c0af220 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.203+0000 7f911bfff640 1 -- 172.21.15.27:0/952398081 >> 172.21.15.27:0/952398081 conn(0x7f911c0caab0 msgr2=0x7f911c0cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.203+0000 7f911bfff640 1 -- 172.21.15.27:0/952398081 shutdown_connections 2023-08-11T15:52:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.203+0000 7f911bfff640 1 -- 172.21.15.27:0/952398081 wait complete. 2023-08-11T15:52:45.225 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:52:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.456+0000 7fe548ac5640 1 Processor -- start 2023-08-11T15:52:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.456+0000 7fe548ac5640 1 -- start start 2023-08-11T15:52:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.458+0000 7fe548ac5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe544151220 0x7fe54406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.458+0000 7fe548ac5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe54406bdf0 0x7fe54415df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.458+0000 7fe548ac5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54406c850 con 0x7fe544151600 2023-08-11T15:52:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.458+0000 7fe548ac5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54406c290 con 0x7fe544151220 2023-08-11T15:52:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.458+0000 7fe548ac5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54406c570 con 0x7fe54406bdf0 2023-08-11T15:52:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.459+0000 7fe54259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe544151220 0x7fe54406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.459+0000 7fe54259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe544151220 0x7fe54406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43220/0 (socket says 172.21.15.27:43220) 2023-08-11T15:52:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.459+0000 7fe54259c640 1 -- 172.21.15.27:0/119704744 learned_addr learned my addr 172.21.15.27:0/119704744 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.459+0000 7fe541d9b640 1 --2- 172.21.15.27:0/119704744 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe54406bdf0 0x7fe54415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.459+0000 7fe54259c640 1 -- 172.21.15.27:0/119704744 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe54406bdf0 msgr2=0x7fe54415df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.460+0000 7fe54259c640 1 --2- 172.21.15.27:0/119704744 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe54406bdf0 0x7fe54415df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.460+0000 7fe54259c640 1 -- 172.21.15.27:0/119704744 >> v1:172.21.15.119:6789/0 conn(0x7fe544151600 legacy=0x7fe5440ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.460+0000 7fe54259c640 1 -- 172.21.15.27:0/119704744 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5440ca050 con 0x7fe544151220 2023-08-11T15:52:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.461+0000 7fe54259c640 1 --2- 172.21.15.27:0/119704744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151220 0x7fe54406b8b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe534009870 tx=0x7fe53402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9ed441968af75e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.461+0000 7fe54159a640 1 -- 172.21.15.27:0/119704744 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5340317c0 con 0x7fe544151220 2023-08-11T15:52:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.461+0000 7fe54159a640 1 -- 172.21.15.27:0/119704744 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe534031920 con 0x7fe544151220 2023-08-11T15:52:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.461+0000 7fe54159a640 1 -- 172.21.15.27:0/119704744 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534031cb0 con 0x7fe544151220 2023-08-11T15:52:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.462+0000 7fe548ac5640 1 -- 172.21.15.27:0/119704744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151220 msgr2=0x7fe54406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.462+0000 7fe548ac5640 1 --2- 172.21.15.27:0/119704744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151220 0x7fe54406b8b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe534009870 tx=0x7fe53402b860 comp rx=0 tx=0).stop 2023-08-11T15:52:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.462+0000 7fe548ac5640 1 -- 172.21.15.27:0/119704744 shutdown_connections 2023-08-11T15:52:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.462+0000 7fe548ac5640 1 --2- 172.21.15.27:0/119704744 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe54406bdf0 0x7fe54415df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.462+0000 7fe548ac5640 1 --2- 172.21.15.27:0/119704744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151220 0x7fe54406b8b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.462+0000 7fe548ac5640 1 -- 172.21.15.27:0/119704744 >> 172.21.15.27:0/119704744 conn(0x7fe5440afbe0 msgr2=0x7fe5440affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.462+0000 7fe548ac5640 1 -- 172.21.15.27:0/119704744 shutdown_connections 2023-08-11T15:52:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.462+0000 7fe548ac5640 1 -- 172.21.15.27:0/119704744 wait complete. 2023-08-11T15:52:45.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.465+0000 7fe548ac5640 1 Processor -- start 2023-08-11T15:52:45.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.465+0000 7fe548ac5640 1 -- start start 2023-08-11T15:52:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe548ac5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe54406bdf0 0x7fe544142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe548ac5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe544151220 0x7fe5440c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe548ac5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151600 0x7fe5440c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe548ac5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440c93f0 con 0x7fe544151600 2023-08-11T15:52:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe548ac5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440c50c0 con 0x7fe54406bdf0 2023-08-11T15:52:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe548ac5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440c5680 con 0x7fe544151220 2023-08-11T15:52:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe542d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151600 0x7fe5440c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe542d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151600 0x7fe5440c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43222/0 (socket says 172.21.15.27:43222) 2023-08-11T15:52:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe542d9d640 1 -- 172.21.15.27:0/4038058161 learned_addr learned my addr 172.21.15.27:0/4038058161 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe54259c640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe54406bdf0 0x7fe544142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.467+0000 7fe541d9b640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe544151220 0x7fe5440c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe542d9d640 1 -- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe544151220 msgr2=0x7fe5440c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe542d9d640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe544151220 0x7fe5440c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe542d9d640 1 -- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe54406bdf0 msgr2=0x7fe544142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe542d9d640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe54406bdf0 0x7fe544142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe542d9d640 1 -- 172.21.15.27:0/4038058161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5340094d0 con 0x7fe544151600 2023-08-11T15:52:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe54259c640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe54406bdf0 0x7fe544142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe542d9d640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151600 0x7fe5440c8eb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c730 tx=0x7fe53c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe532ffd640 1 -- 172.21.15.27:0/4038058161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c003b90 con 0x7fe544151600 2023-08-11T15:52:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe532ffd640 1 -- 172.21.15.27:0/4038058161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53c003cf0 con 0x7fe544151600 2023-08-11T15:52:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe548ac5640 1 -- 172.21.15.27:0/4038058161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5440c7510 con 0x7fe544151600 2023-08-11T15:52:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.468+0000 7fe532ffd640 1 -- 172.21.15.27:0/4038058161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c00aa90 con 0x7fe544151600 2023-08-11T15:52:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.469+0000 7fe548ac5640 1 -- 172.21.15.27:0/4038058161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5440c5310 con 0x7fe544151600 2023-08-11T15:52:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.470+0000 7fe548ac5640 1 -- 172.21.15.27:0/4038058161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5440c5830 con 0x7fe544151600 2023-08-11T15:52:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.471+0000 7fe532ffd640 1 -- 172.21.15.27:0/4038058161 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe53c00abf0 con 0x7fe544151600 2023-08-11T15:52:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.472+0000 7fe532ffd640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe52406a050 0x7fe52406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.472+0000 7fe532ffd640 1 -- 172.21.15.27:0/4038058161 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe53c0886c0 con 0x7fe544151600 2023-08-11T15:52:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.472+0000 7fe54259c640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe52406a050 0x7fe52406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.474+0000 7fe54259c640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe52406a050 0x7fe52406c510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe534036000 tx=0x7fe53403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:45.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.476+0000 7fe532ffd640 1 -- 172.21.15.27:0/4038058161 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe5440c5830 con 0x7fe544151600 2023-08-11T15:52:45.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.674+0000 7fe548ac5640 1 -- 172.21.15.27:0/4038058161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe5440c57f0 con 0x7fe544151600 2023-08-11T15:52:45.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.675+0000 7fe532ffd640 1 -- 172.21.15.27:0/4038058161 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe53c053a90 con 0x7fe544151600 2023-08-11T15:52:45.677 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:52:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.678+0000 7fe548ac5640 1 -- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe52406a050 msgr2=0x7fe52406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.678+0000 7fe548ac5640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe52406a050 0x7fe52406c510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe534036000 tx=0x7fe53403d040 comp rx=0 tx=0).stop 2023-08-11T15:52:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.678+0000 7fe548ac5640 1 -- 172.21.15.27:0/4038058161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151600 msgr2=0x7fe5440c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.678+0000 7fe548ac5640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151600 0x7fe5440c8eb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c730 tx=0x7fe53c00cbf0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.678+0000 7fe548ac5640 1 -- 172.21.15.27:0/4038058161 shutdown_connections 2023-08-11T15:52:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.678+0000 7fe548ac5640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe52406a050 0x7fe52406c510 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.678+0000 7fe548ac5640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe544151220 0x7fe5440c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.678+0000 7fe548ac5640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544151600 0x7fe5440c8eb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.678+0000 7fe548ac5640 1 --2- 172.21.15.27:0/4038058161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe54406bdf0 0x7fe544142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.678+0000 7fe548ac5640 1 -- 172.21.15.27:0/4038058161 >> 172.21.15.27:0/4038058161 conn(0x7fe5440afbe0 msgr2=0x7fe5440b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.679+0000 7fe548ac5640 1 -- 172.21.15.27:0/4038058161 shutdown_connections 2023-08-11T15:52:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.679+0000 7fe548ac5640 1 -- 172.21.15.27:0/4038058161 wait complete. 2023-08-11T15:52:45.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:52:45.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.935+0000 7fafbcd62640 1 Processor -- start 2023-08-11T15:52:45.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.936+0000 7fafbcd62640 1 -- start start 2023-08-11T15:52:45.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.937+0000 7fafbcd62640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb80b55c0 0x7fafb80cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.937+0000 7fafbcd62640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb80cda30 con 0x7fafb80cf930 2023-08-11T15:52:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.937+0000 7fafbcd62640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb8161110 con 0x7fafb815c620 2023-08-11T15:52:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.937+0000 7fafbcd62640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80ccc10 con 0x7fafb80b55c0 2023-08-11T15:52:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb6d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fafb80cf930 0x7fafb80d0b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57542/0 (socket says 172.21.15.27:57542) 2023-08-11T15:52:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb6d9d640 1 -- 172.21.15.27:0/3282017626 learned_addr learned my addr 172.21.15.27:0/3282017626 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb5d9b640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb80b55c0 0x7fafb80cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755135462 0 0) 0x7fafb80cda30 con 0x7fafb80cf930 2023-08-11T15:52:45.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa0003520 con 0x7fafb80cf930 2023-08-11T15:52:45.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1505561344 0 0) 0x7fafa0003520 con 0x7fafb80cf930 2023-08-11T15:52:45.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb80b55c0 msgr2=0x7fafb80cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:45.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb559a640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb80b55c0 0x7fafb80cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 >> v1:172.21.15.119:6789/0 conn(0x7fafb815c620 legacy=0x7fafb80b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb80cce20 con 0x7fafb80cf930 2023-08-11T15:52:45.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.938+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fafb00019f0 con 0x7fafb80cf930 2023-08-11T15:52:45.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.939+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 >> v1:172.21.15.27:6789/0 conn(0x7fafb80cf930 legacy=0x7fafb80d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.939+0000 7fafb559a640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 0x7fafa0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.939+0000 7fafb559a640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb80b55c0 0x7fafa00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.939+0000 7fafb559a640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa0004d00 0x7fafa0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.939+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80ccc10 con 0x7fafa00039a0 2023-08-11T15:52:45.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.939+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa000ba30 con 0x7fafa0004d00 2023-08-11T15:52:45.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.939+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa000bbf0 con 0x7fafb80b55c0 2023-08-11T15:52:45.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.939+0000 7fafb659c640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 0x7fafa0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.939+0000 7fafb6d9d640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa0004d00 0x7fafa0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.939+0000 7fafb5d9b640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb80b55c0 0x7fafa00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.940+0000 7fafb659c640 1 -- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb80b55c0 msgr2=0x7fafa00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:45.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.940+0000 7fafb659c640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb80b55c0 0x7fafa00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.940+0000 7fafb659c640 1 -- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa0004d00 msgr2=0x7fafa0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:45.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.940+0000 7fafb659c640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa0004d00 0x7fafa0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.940+0000 7fafb659c640 1 -- 172.21.15.27:0/3282017626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa0003520 con 0x7fafa00039a0 2023-08-11T15:52:45.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.940+0000 7fafb5d9b640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb80b55c0 0x7fafa00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:45.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.941+0000 7fafb659c640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 0x7fafa0003dc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fafa8001720 tx=0x7fafa00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d455db0d523619 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:45.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.941+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa80052d0 con 0x7fafa00039a0 2023-08-11T15:52:45.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.941+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa802e050 con 0x7fafa00039a0 2023-08-11T15:52:45.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.941+0000 7fafb559a640 1 -- 172.21.15.27:0/3282017626 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa80035f0 con 0x7fafa00039a0 2023-08-11T15:52:45.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.942+0000 7fafbcd62640 1 -- 172.21.15.27:0/3282017626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 msgr2=0x7fafa0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.942+0000 7fafbcd62640 1 --2- 172.21.15.27:0/3282017626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 0x7fafa0003dc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fafa8001720 tx=0x7fafa00057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.942+0000 7fafb6d9d640 1 -- 172.21.15.27:0/3282017626 reap_dead start 2023-08-11T15:52:45.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.942+0000 7fafbcd62640 1 -- 172.21.15.27:0/3282017626 shutdown_connections 2023-08-11T15:52:45.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.942+0000 7fafbcd62640 1 -- 172.21.15.27:0/3282017626 >> 172.21.15.27:0/3282017626 conn(0x7fafb8071110 msgr2=0x7fafb80714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:45.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.942+0000 7fafbcd62640 1 -- 172.21.15.27:0/3282017626 shutdown_connections 2023-08-11T15:52:45.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.942+0000 7fafbcd62640 1 -- 172.21.15.27:0/3282017626 wait complete. 2023-08-11T15:52:45.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.944+0000 7fafbcd62640 1 Processor -- start 2023-08-11T15:52:45.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.944+0000 7fafbcd62640 1 -- start start 2023-08-11T15:52:45.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.945+0000 7fafbcd62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 0x7fafb806f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.945+0000 7fafbcd62640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb814c560 0x7fafb814c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.945+0000 7fafbcd62640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb80ca560 0x7fafb80ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.945+0000 7fafbcd62640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80cb5a0 con 0x7fafa00039a0 2023-08-11T15:52:45.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.945+0000 7fafbcd62640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb814cf20 con 0x7fafb80ca560 2023-08-11T15:52:45.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.945+0000 7fafbcd62640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80cb2c0 con 0x7fafb814c560 2023-08-11T15:52:45.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.945+0000 7fafb659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 0x7fafb806f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.945+0000 7fafb659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 0x7fafb806f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43238/0 (socket says 172.21.15.27:43238) 2023-08-11T15:52:45.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.945+0000 7fafb659c640 1 -- 172.21.15.27:0/402556212 learned_addr learned my addr 172.21.15.27:0/402556212 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:45.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.946+0000 7fafb6d9d640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb80ca560 0x7fafb80ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.946+0000 7fafb5d9b640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb814c560 0x7fafb814c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.946+0000 7fafb659c640 1 -- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb814c560 msgr2=0x7fafb814c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.946+0000 7fafb659c640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb814c560 0x7fafb814c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.946+0000 7fafb659c640 1 -- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb80ca560 msgr2=0x7fafb80ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:45.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.946+0000 7fafb659c640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb80ca560 0x7fafb80ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:45.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.946+0000 7fafb659c640 1 -- 172.21.15.27:0/402556212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb81507d0 con 0x7fafa00039a0 2023-08-11T15:52:45.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.946+0000 7fafb659c640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 0x7fafb806f630 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fafa8008fd0 tx=0x7fafa8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:45.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.947+0000 7fafa6ffd640 1 -- 172.21.15.27:0/402556212 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa80076c0 con 0x7fafa00039a0 2023-08-11T15:52:45.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.947+0000 7fafa6ffd640 1 -- 172.21.15.27:0/402556212 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa8007a30 con 0x7fafa00039a0 2023-08-11T15:52:45.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.947+0000 7fafbcd62640 1 -- 172.21.15.27:0/402556212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb8150ab0 con 0x7fafa00039a0 2023-08-11T15:52:45.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.947+0000 7fafa6ffd640 1 -- 172.21.15.27:0/402556212 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa8006be0 con 0x7fafa00039a0 2023-08-11T15:52:45.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.947+0000 7fafbcd62640 1 -- 172.21.15.27:0/402556212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb80caf20 con 0x7fafa00039a0 2023-08-11T15:52:45.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.948+0000 7fafbcd62640 1 -- 172.21.15.27:0/402556212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb8073990 con 0x7fafa00039a0 2023-08-11T15:52:45.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.949+0000 7fafa6ffd640 1 -- 172.21.15.27:0/402556212 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fafa8006d40 con 0x7fafa00039a0 2023-08-11T15:52:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.950+0000 7fafa6ffd640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf9806a050 0x7faf9806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.950+0000 7fafa6ffd640 1 -- 172.21.15.27:0/402556212 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fafa80aa990 con 0x7fafa00039a0 2023-08-11T15:52:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.951+0000 7fafb5d9b640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf9806a050 0x7faf9806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.952+0000 7fafb5d9b640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf9806a050 0x7faf9806c510 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fafb8152cb0 tx=0x7fafac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:45.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:45.954+0000 7fafa6ffd640 1 -- 172.21.15.27:0/402556212 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fafa8079870 con 0x7fafa00039a0 2023-08-11T15:52:46.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.184+0000 7fafbcd62640 1 -- 172.21.15.27:0/402556212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafb80cb5a0 con 0x7fafa00039a0 2023-08-11T15:52:46.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.186+0000 7fafa6ffd640 1 -- 172.21.15.27:0/402556212 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fafa8079400 con 0x7fafa00039a0 2023-08-11T15:52:46.188 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:46.189 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:51:01.220970+0000","modified":"2023-08-11T15:51:25.832166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4315":{"gid":4315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:52:46.189 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2023-08-11T15:52:46.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.188+0000 7fafbcd62640 1 -- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf9806a050 msgr2=0x7faf9806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.188+0000 7fafbcd62640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf9806a050 0x7faf9806c510 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fafb8152cb0 tx=0x7fafac008040 comp rx=0 tx=0).stop 2023-08-11T15:52:46.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.188+0000 7fafbcd62640 1 -- 172.21.15.27:0/402556212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 msgr2=0x7fafb806f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.188+0000 7fafbcd62640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 0x7fafb806f630 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fafa8008fd0 tx=0x7fafa8001120 comp rx=0 tx=0).stop 2023-08-11T15:52:46.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.189+0000 7fafbcd62640 1 -- 172.21.15.27:0/402556212 shutdown_connections 2023-08-11T15:52:46.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.189+0000 7fafbcd62640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf9806a050 0x7faf9806c510 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.189+0000 7fafbcd62640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fafb814c560 0x7fafb814c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.189+0000 7fafbcd62640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafb80ca560 0x7fafb80ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.189+0000 7fafbcd62640 1 --2- 172.21.15.27:0/402556212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa00039a0 0x7fafb806f630 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.189+0000 7fafbcd62640 1 -- 172.21.15.27:0/402556212 >> 172.21.15.27:0/402556212 conn(0x7fafb8071110 msgr2=0x7fafb815bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:46.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.189+0000 7fafbcd62640 1 -- 172.21.15.27:0/402556212 shutdown_connections 2023-08-11T15:52:46.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.189+0000 7fafbcd62640 1 -- 172.21.15.27:0/402556212 wait complete. 2023-08-11T15:52:46.210 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:46.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.442+0000 7f6cf8afa640 1 Processor -- start 2023-08-11T15:52:46.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.443+0000 7f6cf8afa640 1 -- start start 2023-08-11T15:52:46.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.444+0000 7f6cf8afa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6cf406f5d0 0x7f6cf406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.444+0000 7f6cf8afa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf40c8a30 con 0x7f6cf40cb430 2023-08-11T15:52:46.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.444+0000 7f6cf8afa640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf406c0c0 con 0x7f6cf415a140 2023-08-11T15:52:46.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.444+0000 7f6cf8afa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf406ffa0 con 0x7f6cf406f5d0 2023-08-11T15:52:46.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.444+0000 7f6cf2d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6cf406f5d0 0x7f6cf406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.444+0000 7f6cebfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6cf40cb430 0x7f6cf40b0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57552/0 (socket says 172.21.15.27:57552) 2023-08-11T15:52:46.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.444+0000 7f6cebfff640 1 -- 172.21.15.27:0/2335972210 learned_addr learned my addr 172.21.15.27:0/2335972210 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:46.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.444+0000 7f6cf2d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6cf406f5d0 0x7f6cf406fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43250/0 (socket says 172.21.15.27:43250) 2023-08-11T15:52:46.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.445+0000 7f6cf2d9d640 1 -- 172.21.15.27:0/2335972210 >> v1:172.21.15.119:6789/0 conn(0x7f6cf415a140 legacy=0x7f6cf406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:46.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.445+0000 7f6cf2d9d640 1 -- 172.21.15.27:0/2335972210 >> v1:172.21.15.27:6789/0 conn(0x7f6cf40cb430 legacy=0x7f6cf40b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.445+0000 7f6cf2d9d640 1 -- 172.21.15.27:0/2335972210 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf406c3a0 con 0x7f6cf406f5d0 2023-08-11T15:52:46.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.445+0000 7f6cf1d9b640 1 -- 172.21.15.27:0/2335972210 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211418829 0 0) 0x7f6cf40c8a30 con 0x7f6cf40cb430 2023-08-11T15:52:46.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.446+0000 7f6cf2d9d640 1 --2- 172.21.15.27:0/2335972210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf406f5d0 0x7f6cf406fa60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009d90 tx=0x7f6cec02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed284bda2c7a406a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:46.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.446+0000 7f6cf1d9b640 1 -- 172.21.15.27:0/2335972210 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec02cc60 con 0x7f6cf406f5d0 2023-08-11T15:52:46.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.446+0000 7f6cf1d9b640 1 -- 172.21.15.27:0/2335972210 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec02cdc0 con 0x7f6cf406f5d0 2023-08-11T15:52:46.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.446+0000 7f6cf1d9b640 1 -- 172.21.15.27:0/2335972210 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec034660 con 0x7f6cf406f5d0 2023-08-11T15:52:46.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.447+0000 7f6cf8afa640 1 -- 172.21.15.27:0/2335972210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf406f5d0 msgr2=0x7f6cf406fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.447+0000 7f6cf8afa640 1 --2- 172.21.15.27:0/2335972210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf406f5d0 0x7f6cf406fa60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009d90 tx=0x7f6cec02beb0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.447+0000 7f6cf8afa640 1 -- 172.21.15.27:0/2335972210 shutdown_connections 2023-08-11T15:52:46.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.447+0000 7f6cf8afa640 1 --2- 172.21.15.27:0/2335972210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf406f5d0 0x7f6cf406fa60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.447+0000 7f6cf8afa640 1 -- 172.21.15.27:0/2335972210 >> 172.21.15.27:0/2335972210 conn(0x7f6cf40afb70 msgr2=0x7f6cf40aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.447+0000 7f6cf8afa640 1 -- 172.21.15.27:0/2335972210 shutdown_connections 2023-08-11T15:52:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.447+0000 7f6cf8afa640 1 -- 172.21.15.27:0/2335972210 wait complete. 2023-08-11T15:52:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.449+0000 7f6cf8afa640 1 Processor -- start 2023-08-11T15:52:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.449+0000 7f6cf8afa640 1 -- start start 2023-08-11T15:52:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.450+0000 7f6cf8afa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cf406f5d0 0x7f6cf4151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.450+0000 7f6cf8afa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf40cb430 0x7f6cf415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.450+0000 7f6cf8afa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf415a140 0x7f6cf415e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.450+0000 7f6cf8afa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf40c7d60 con 0x7f6cf415a140 2023-08-11T15:52:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.450+0000 7f6cf8afa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf40c3660 con 0x7f6cf40cb430 2023-08-11T15:52:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.450+0000 7f6cf8afa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf40c3940 con 0x7f6cf406f5d0 2023-08-11T15:52:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.450+0000 7f6cf2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf415a140 0x7f6cf415e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.450+0000 7f6cf2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf415a140 0x7f6cf415e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43266/0 (socket says 172.21.15.27:43266) 2023-08-11T15:52:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.450+0000 7f6cf2d9d640 1 -- 172.21.15.27:0/137514719 learned_addr learned my addr 172.21.15.27:0/137514719 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:46.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.451+0000 7f6cebfff640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf40cb430 0x7f6cf415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.451+0000 7f6cf259c640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cf406f5d0 0x7f6cf4151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.451+0000 7f6cf2d9d640 1 -- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cf406f5d0 msgr2=0x7f6cf4151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.451+0000 7f6cf2d9d640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cf406f5d0 0x7f6cf4151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.451+0000 7f6cf2d9d640 1 -- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf40cb430 msgr2=0x7f6cf415dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.451+0000 7f6cf2d9d640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf40cb430 0x7f6cf415dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.451+0000 7f6cf2d9d640 1 -- 172.21.15.27:0/137514719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec009a70 con 0x7f6cf415a140 2023-08-11T15:52:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.451+0000 7f6cf2d9d640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf415a140 0x7f6cf415e2d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009eb0 tx=0x7f6cec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.452+0000 7f6ceaffd640 1 -- 172.21.15.27:0/137514719 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec03f420 con 0x7f6cf415a140 2023-08-11T15:52:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.452+0000 7f6ceaffd640 1 -- 172.21.15.27:0/137514719 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec03f580 con 0x7f6cf415a140 2023-08-11T15:52:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.452+0000 7f6ceaffd640 1 -- 172.21.15.27:0/137514719 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec03f8f0 con 0x7f6cf415a140 2023-08-11T15:52:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.452+0000 7f6cf8afa640 1 -- 172.21.15.27:0/137514719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf40c3c20 con 0x7f6cf415a140 2023-08-11T15:52:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.452+0000 7f6cf8afa640 1 -- 172.21.15.27:0/137514719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf40c41e0 con 0x7f6cf415a140 2023-08-11T15:52:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.453+0000 7f6cf8afa640 1 -- 172.21.15.27:0/137514719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf40c43d0 con 0x7f6cf415a140 2023-08-11T15:52:46.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.454+0000 7f6ceaffd640 1 -- 172.21.15.27:0/137514719 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6cec03fa50 con 0x7f6cf415a140 2023-08-11T15:52:46.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.455+0000 7f6ceaffd640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6cd406a050 0x7f6cd406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.455+0000 7f6ceaffd640 1 -- 172.21.15.27:0/137514719 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6cec0ab740 con 0x7f6cf415a140 2023-08-11T15:52:46.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.456+0000 7f6cf259c640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6cd406a050 0x7f6cd406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.457+0000 7f6cf259c640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6cd406a050 0x7f6cd406c510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400a790 tx=0x7f6ce4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:46.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.459+0000 7f6ceaffd640 1 -- 172.21.15.27:0/137514719 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6cec0769d0 con 0x7f6cf415a140 2023-08-11T15:52:46.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.656+0000 7f6cf8afa640 1 -- 172.21.15.27:0/137514719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cf4073990 con 0x7f6cf415a140 2023-08-11T15:52:46.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.660+0000 7f6ceaffd640 1 -- 172.21.15.27:0/137514719 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f6cec079ff0 con 0x7f6cf415a140 2023-08-11T15:52:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:46.662 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:46.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.663+0000 7f6cf8afa640 1 -- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6cd406a050 msgr2=0x7f6cd406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.663+0000 7f6cf8afa640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6cd406a050 0x7f6cd406c510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400a790 tx=0x7f6ce4008040 comp rx=0 tx=0).stop 2023-08-11T15:52:46.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.663+0000 7f6cf8afa640 1 -- 172.21.15.27:0/137514719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf415a140 msgr2=0x7f6cf415e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.663+0000 7f6cf8afa640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf415a140 0x7f6cf415e2d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009eb0 tx=0x7f6cec004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:46.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.663+0000 7f6cf8afa640 1 -- 172.21.15.27:0/137514719 shutdown_connections 2023-08-11T15:52:46.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.663+0000 7f6cf8afa640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6cd406a050 0x7f6cd406c510 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.663+0000 7f6cf8afa640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf415a140 0x7f6cf415e2d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.663+0000 7f6cf8afa640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf40cb430 0x7f6cf415dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.663+0000 7f6cf8afa640 1 --2- 172.21.15.27:0/137514719 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cf406f5d0 0x7f6cf4151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.663+0000 7f6cf8afa640 1 -- 172.21.15.27:0/137514719 >> 172.21.15.27:0/137514719 conn(0x7f6cf40afb70 msgr2=0x7f6cf40b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:46.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.664+0000 7f6cf8afa640 1 -- 172.21.15.27:0/137514719 shutdown_connections 2023-08-11T15:52:46.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.664+0000 7f6cf8afa640 1 -- 172.21.15.27:0/137514719 wait complete. 2023-08-11T15:52:46.686 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-11T15:52:46.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:52:46.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.925+0000 7f7093fff640 1 Processor -- start 2023-08-11T15:52:46.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.925+0000 7f7093fff640 1 -- start start 2023-08-11T15:52:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.926+0000 7f7093fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70940c9600 0x7f709415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.926+0000 7f7093fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f709406f5d0 0x7f709406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.926+0000 7f7093fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70940c85f0 con 0x7f709415a020 2023-08-11T15:52:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.926+0000 7f7093fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709415eae0 con 0x7f709406f5d0 2023-08-11T15:52:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.926+0000 7f7093fff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709406ff10 con 0x7f70940c9600 2023-08-11T15:52:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.926+0000 7f70937fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f709406f5d0 0x7f709406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.926+0000 7f7092ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f709415a020 0x7f7094071440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57554/0 (socket says 172.21.15.27:57554) 2023-08-11T15:52:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.926+0000 7f7092ffd640 1 -- 172.21.15.27:0/190799222 learned_addr learned my addr 172.21.15.27:0/190799222 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.926+0000 7f70937fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f709406f5d0 0x7f709406f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43282/0 (socket says 172.21.15.27:43282) 2023-08-11T15:52:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.927+0000 7f708bfff640 1 --2- 172.21.15.27:0/190799222 >> v2:172.21.15.119:3300/0 conn(0x7f70940c9600 0x7f709415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.927+0000 7f70937fe640 1 -- 172.21.15.27:0/190799222 >> v2:172.21.15.119:3300/0 conn(0x7f70940c9600 msgr2=0x7f709415e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.927+0000 7f70937fe640 1 --2- 172.21.15.27:0/190799222 >> v2:172.21.15.119:3300/0 conn(0x7f70940c9600 0x7f709415e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.927+0000 7f70937fe640 1 -- 172.21.15.27:0/190799222 >> v1:172.21.15.27:6789/0 conn(0x7f709415a020 legacy=0x7f7094071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.927+0000 7f70937fe640 1 -- 172.21.15.27:0/190799222 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70940c8840 con 0x7f709406f5d0 2023-08-11T15:52:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.927+0000 7f70927fc640 1 -- 172.21.15.27:0/190799222 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257735750 0 0) 0x7f70940c85f0 con 0x7f709415a020 2023-08-11T15:52:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.927+0000 7f708bfff640 1 --2- 172.21.15.27:0/190799222 >> v2:172.21.15.119:3300/0 conn(0x7f70940c9600 0x7f709415e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.928+0000 7f70937fe640 1 --2- 172.21.15.27:0/190799222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709406f5d0 0x7f709406f9d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7094071210 tx=0x7f708c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d26aa4a767c6b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.928+0000 7f70927fc640 1 -- 172.21.15.27:0/190799222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c02cbe0 con 0x7f709406f5d0 2023-08-11T15:52:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.928+0000 7f70927fc640 1 -- 172.21.15.27:0/190799222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c02cd40 con 0x7f709406f5d0 2023-08-11T15:52:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.928+0000 7f70927fc640 1 -- 172.21.15.27:0/190799222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c034660 con 0x7f709406f5d0 2023-08-11T15:52:46.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.929+0000 7f7093fff640 1 -- 172.21.15.27:0/190799222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709406f5d0 msgr2=0x7f709406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.929+0000 7f7093fff640 1 --2- 172.21.15.27:0/190799222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709406f5d0 0x7f709406f9d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7094071210 tx=0x7f708c02be30 comp rx=0 tx=0).stop 2023-08-11T15:52:46.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.929+0000 7f7093fff640 1 -- 172.21.15.27:0/190799222 shutdown_connections 2023-08-11T15:52:46.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.929+0000 7f7093fff640 1 --2- 172.21.15.27:0/190799222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709406f5d0 0x7f709406f9d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.929+0000 7f7093fff640 1 --2- 172.21.15.27:0/190799222 >> v2:172.21.15.119:3300/0 conn(0x7f70940c9600 0x7f709415e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.929+0000 7f7093fff640 1 -- 172.21.15.27:0/190799222 >> 172.21.15.27:0/190799222 conn(0x7f70940afb70 msgr2=0x7f70940aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.929+0000 7f7093fff640 1 -- 172.21.15.27:0/190799222 shutdown_connections 2023-08-11T15:52:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.929+0000 7f7093fff640 1 -- 172.21.15.27:0/190799222 wait complete. 2023-08-11T15:52:46.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.931+0000 7f7093fff640 1 Processor -- start 2023-08-11T15:52:46.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.931+0000 7f7093fff640 1 -- start start 2023-08-11T15:52:46.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.932+0000 7f7093fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f709406f5d0 0x7f70940c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.932+0000 7f7093fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70940c9600 0x7f7094158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.932+0000 7f7093fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709415a020 0x7f709415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.932+0000 7f7093fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70940c6360 con 0x7f709415a020 2023-08-11T15:52:46.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.932+0000 7f7093fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709415c070 con 0x7f70940c9600 2023-08-11T15:52:46.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.932+0000 7f7093fff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70940c5da0 con 0x7f709406f5d0 2023-08-11T15:52:46.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f70937fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709415a020 0x7f709415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f70937fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709415a020 0x7f709415cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43294/0 (socket says 172.21.15.27:43294) 2023-08-11T15:52:46.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f70937fe640 1 -- 172.21.15.27:0/124164178 learned_addr learned my addr 172.21.15.27:0/124164178 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f7092ffd640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f709406f5d0 0x7f70940c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f708bfff640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70940c9600 0x7f7094158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f70937fe640 1 -- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f709406f5d0 msgr2=0x7f70940c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f70937fe640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f709406f5d0 0x7f70940c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f70937fe640 1 -- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70940c9600 msgr2=0x7f7094158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f70937fe640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70940c9600 0x7f7094158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f70937fe640 1 -- 172.21.15.27:0/124164178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c009a70 con 0x7f709415a020 2023-08-11T15:52:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.933+0000 7f70937fe640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709415a020 0x7f709415cb10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f708c02c440 tx=0x7f708c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:46.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.934+0000 7f708b7fe640 1 -- 172.21.15.27:0/124164178 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c03e610 con 0x7f709415a020 2023-08-11T15:52:46.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.934+0000 7f708b7fe640 1 -- 172.21.15.27:0/124164178 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c03e770 con 0x7f709415a020 2023-08-11T15:52:46.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.934+0000 7f708b7fe640 1 -- 172.21.15.27:0/124164178 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c03eae0 con 0x7f709415a020 2023-08-11T15:52:46.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.934+0000 7f7093fff640 1 -- 172.21.15.27:0/124164178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70940c6080 con 0x7f709415a020 2023-08-11T15:52:46.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.934+0000 7f7093fff640 1 -- 172.21.15.27:0/124164178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709415c5f0 con 0x7f709415a020 2023-08-11T15:52:46.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.935+0000 7f7093fff640 1 -- 172.21.15.27:0/124164178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709404e420 con 0x7f709415a020 2023-08-11T15:52:46.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.936+0000 7f708b7fe640 1 -- 172.21.15.27:0/124164178 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f708c03b050 con 0x7f709415a020 2023-08-11T15:52:46.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.937+0000 7f708b7fe640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f707406a050 0x7f707406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:46.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.937+0000 7f708b7fe640 1 -- 172.21.15.27:0/124164178 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f708c0ab0d0 con 0x7f709415a020 2023-08-11T15:52:46.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.938+0000 7f7092ffd640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f707406a050 0x7f707406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:46.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.939+0000 7f7092ffd640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f707406a050 0x7f707406c510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7084000f60 tx=0x7f7084007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:46.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:46.941+0000 7f708b7fe640 1 -- 172.21.15.27:0/124164178 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f708c076510 con 0x7f709415a020 2023-08-11T15:52:47.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.175+0000 7f7093fff640 1 -- 172.21.15.27:0/124164178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f709415c7b0 con 0x7f709415a020 2023-08-11T15:52:47.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.176+0000 7f708b7fe640 1 -- 172.21.15.27:0/124164178 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f708c079b30 con 0x7f709415a020 2023-08-11T15:52:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:47.180 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:51:01.220970+0000","modified":"2023-08-11T15:51:25.832166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4315":{"gid":4315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:52:47.180 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2023-08-11T15:52:47.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.179+0000 7f7093fff640 1 -- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f707406a050 msgr2=0x7f707406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.179+0000 7f7093fff640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f707406a050 0x7f707406c510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7084000f60 tx=0x7f7084007c00 comp rx=0 tx=0).stop 2023-08-11T15:52:47.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.179+0000 7f7093fff640 1 -- 172.21.15.27:0/124164178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709415a020 msgr2=0x7f709415cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.179+0000 7f7093fff640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709415a020 0x7f709415cb10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f708c02c440 tx=0x7f708c004a40 comp rx=0 tx=0).stop 2023-08-11T15:52:47.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.179+0000 7f7093fff640 1 -- 172.21.15.27:0/124164178 shutdown_connections 2023-08-11T15:52:47.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.179+0000 7f7093fff640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f707406a050 0x7f707406c510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.179+0000 7f7093fff640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709415a020 0x7f709415cb10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.179+0000 7f7093fff640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70940c9600 0x7f7094158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.179+0000 7f7093fff640 1 --2- 172.21.15.27:0/124164178 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f709406f5d0 0x7f70940c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.179+0000 7f7093fff640 1 -- 172.21.15.27:0/124164178 >> 172.21.15.27:0/124164178 conn(0x7f70940afb70 msgr2=0x7f70940acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.180+0000 7f7093fff640 1 -- 172.21.15.27:0/124164178 shutdown_connections 2023-08-11T15:52:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.180+0000 7f7093fff640 1 -- 172.21.15.27:0/124164178 wait complete. 2023-08-11T15:52:47.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:52:47.284 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:52:47.293 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:52:47.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.430+0000 7f0a4c3c2640 1 Processor -- start 2023-08-11T15:52:47.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.430+0000 7f0a4c3c2640 1 -- start start 2023-08-11T15:52:47.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.432+0000 7f0a4c3c2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a44156e50 0x7f0a4415a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.432+0000 7f0a4c3c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0a440cbb70 0x7f0a440ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.432+0000 7f0a4c3c2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a440cbfb0 con 0x7f0a440caef0 2023-08-11T15:52:47.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.432+0000 7f0a4c3c2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a440ca460 con 0x7f0a440cbb70 2023-08-11T15:52:47.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.432+0000 7f0a4c3c2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440cb380 con 0x7f0a44156e50 2023-08-11T15:52:47.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.432+0000 7f0a4a95f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0a440caef0 0x7f0a4415ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57562/0 (socket says 172.21.15.27:57562) 2023-08-11T15:52:47.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.432+0000 7f0a4a95f640 1 -- 172.21.15.27:0/1362650758 learned_addr learned my addr 172.21.15.27:0/1362650758 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:47.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4a15e640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a44156e50 0x7f0a4415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4995d640 1 --2- 172.21.15.27:0/1362650758 >> v2:172.21.15.119:3300/0 conn(0x7f0a440cbb70 0x7f0a440ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646047127 0 0) 0x7f0a440cbfb0 con 0x7f0a440caef0 2023-08-11T15:52:47.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a30005730 con 0x7f0a440caef0 2023-08-11T15:52:47.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1543598924 0 0) 0x7f0a30005730 con 0x7f0a440caef0 2023-08-11T15:52:47.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a44156e50 msgr2=0x7f0a4415a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a44156e50 0x7f0a4415a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 >> v2:172.21.15.119:3300/0 conn(0x7f0a440cbb70 msgr2=0x7f0a440ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 --2- 172.21.15.27:0/1362650758 >> v2:172.21.15.119:3300/0 conn(0x7f0a440cbb70 0x7f0a440ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440cc270 con 0x7f0a440caef0 2023-08-11T15:52:47.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f0a3c0019f0 con 0x7f0a440caef0 2023-08-11T15:52:47.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 >> v1:172.21.15.27:6789/0 conn(0x7f0a440caef0 legacy=0x7f0a4415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a44156e50 0x7f0a30005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a300060c0 0x7f0a300069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a30006540 0x7f0a30006f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440ca460 con 0x7f0a300060c0 2023-08-11T15:52:47.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440cb380 con 0x7f0a30006540 2023-08-11T15:52:47.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.433+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a30007780 con 0x7f0a44156e50 2023-08-11T15:52:47.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.434+0000 7f0a4995d640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a300060c0 0x7f0a300069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.434+0000 7f0a4a15e640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a44156e50 0x7f0a30005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.434+0000 7f0a4a95f640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a30006540 0x7f0a30006f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.434+0000 7f0a4995d640 1 -- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a44156e50 msgr2=0x7f0a30005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.434+0000 7f0a4995d640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a44156e50 0x7f0a30005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.434+0000 7f0a4995d640 1 -- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a30006540 msgr2=0x7f0a30006f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.434+0000 7f0a4995d640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a30006540 0x7f0a30006f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.434+0000 7f0a4995d640 1 -- 172.21.15.27:0/1362650758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a30005730 con 0x7f0a300060c0 2023-08-11T15:52:47.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.435+0000 7f0a4995d640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a300060c0 0x7f0a300069d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009ac0 tx=0x7f0a30007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5125905a2dbb21d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:47.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.436+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3402c710 con 0x7f0a300060c0 2023-08-11T15:52:47.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.436+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3402c870 con 0x7f0a300060c0 2023-08-11T15:52:47.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.436+0000 7f0a4915c640 1 -- 172.21.15.27:0/1362650758 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3402cc00 con 0x7f0a300060c0 2023-08-11T15:52:47.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.436+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/1362650758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a300060c0 msgr2=0x7f0a300069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.436+0000 7f0a4c3c2640 1 --2- 172.21.15.27:0/1362650758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a300060c0 0x7f0a300069d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009ac0 tx=0x7f0a30007a00 comp rx=0 tx=0).stop 2023-08-11T15:52:47.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.436+0000 7f0a4a95f640 1 -- 172.21.15.27:0/1362650758 reap_dead start 2023-08-11T15:52:47.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.436+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/1362650758 shutdown_connections 2023-08-11T15:52:47.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.436+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/1362650758 >> 172.21.15.27:0/1362650758 conn(0x7f0a440afb70 msgr2=0x7f0a440aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:47.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.437+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/1362650758 shutdown_connections 2023-08-11T15:52:47.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.437+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/1362650758 wait complete. 2023-08-11T15:52:47.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.438+0000 7f0a4c3c2640 1 Processor -- start 2023-08-11T15:52:47.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.439+0000 7f0a4c3c2640 1 -- start start 2023-08-11T15:52:47.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.439+0000 7f0a4c3c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a300060c0 0x7f0a44142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.439+0000 7f0a4c3c2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a440b1480 0x7f0a440b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.439+0000 7f0a4c3c2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a440c5a10 0x7f0a440c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.439+0000 7f0a4c3c2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440b2ac0 con 0x7f0a440b1480 2023-08-11T15:52:47.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.439+0000 7f0a4c3c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440b1f40 con 0x7f0a300060c0 2023-08-11T15:52:47.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.439+0000 7f0a4c3c2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440b2220 con 0x7f0a440c5a10 2023-08-11T15:52:47.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.440+0000 7f0a4995d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a440b1480 0x7f0a440b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.440+0000 7f0a4995d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a440b1480 0x7f0a440b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43304/0 (socket says 172.21.15.27:43304) 2023-08-11T15:52:47.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.440+0000 7f0a4995d640 1 -- 172.21.15.27:0/4092566503 learned_addr learned my addr 172.21.15.27:0/4092566503 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:47.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.440+0000 7f0a4a15e640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a300060c0 0x7f0a44142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.440+0000 7f0a4a95f640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a440c5a10 0x7f0a440c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.440+0000 7f0a4995d640 1 -- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a440c5a10 msgr2=0x7f0a440c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.440+0000 7f0a4995d640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a440c5a10 0x7f0a440c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.440+0000 7f0a4995d640 1 -- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a300060c0 msgr2=0x7f0a44142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.440+0000 7f0a4995d640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a300060c0 0x7f0a44142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.440+0000 7f0a4995d640 1 -- 172.21.15.27:0/4092566503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440b2500 con 0x7f0a440b1480 2023-08-11T15:52:47.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.441+0000 7f0a4995d640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a440b1480 0x7f0a440b1900 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0a3402c2b0 tx=0x7f0a34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:47.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.441+0000 7f0a3affd640 1 -- 172.21.15.27:0/4092566503 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34003810 con 0x7f0a440b1480 2023-08-11T15:52:47.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.441+0000 7f0a3affd640 1 -- 172.21.15.27:0/4092566503 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a34003990 con 0x7f0a440b1480 2023-08-11T15:52:47.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.441+0000 7f0a3affd640 1 -- 172.21.15.27:0/4092566503 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34003d40 con 0x7f0a440b1480 2023-08-11T15:52:47.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.441+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/4092566503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a440b27e0 con 0x7f0a440b1480 2023-08-11T15:52:47.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.441+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/4092566503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a441426c0 con 0x7f0a440b1480 2023-08-11T15:52:47.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.443+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/4092566503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a44073990 con 0x7f0a440b1480 2023-08-11T15:52:47.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.443+0000 7f0a3affd640 1 -- 172.21.15.27:0/4092566503 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0a34040450 con 0x7f0a440b1480 2023-08-11T15:52:47.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.445+0000 7f0a3affd640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0a2806a050 0x7f0a2806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.445+0000 7f0a3affd640 1 -- 172.21.15.27:0/4092566503 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0a340aa970 con 0x7f0a440b1480 2023-08-11T15:52:47.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.445+0000 7f0a4a15e640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0a2806a050 0x7f0a2806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.447+0000 7f0a4a15e640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0a2806a050 0x7f0a2806c510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009920 tx=0x7f0a40008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.450+0000 7f0a3affd640 1 -- 172.21.15.27:0/4092566503 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0a34075c90 con 0x7f0a440b1480 2023-08-11T15:52:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.683+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/4092566503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a440ccae0 con 0x7f0a440b1480 2023-08-11T15:52:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.684+0000 7f0a3affd640 1 -- 172.21.15.27:0/4092566503 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f0a340792b0 con 0x7f0a440b1480 2023-08-11T15:52:47.686 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:47.686 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1904278400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1904278400},{"type":"v1","addr":"172.21.15.27:6835","nonce":1904278400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2537287732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2537287732},{"type":"v1","addr":"172.21.15.27:6834","nonce":2537287732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/116819137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":116819137},{"type":"v1","addr":"172.21.15.119:6837","nonce":116819137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:51:01.220970+0000","modified":"2023-08-11T15:51:25.832166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4315":{"gid":4315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2813789236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2813789236},{"type":"v1","addr":"172.21.15.119:6835","nonce":2813789236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:52:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2023-08-11T15:52:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0a2806a050 msgr2=0x7f0a2806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0a2806a050 0x7f0a2806c510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009920 tx=0x7f0a40008040 comp rx=0 tx=0).stop 2023-08-11T15:52:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/4092566503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a440b1480 msgr2=0x7f0a440b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a440b1480 0x7f0a440b1900 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0a3402c2b0 tx=0x7f0a34004730 comp rx=0 tx=0).stop 2023-08-11T15:52:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/4092566503 shutdown_connections 2023-08-11T15:52:47.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f0a2806a050 0x7f0a2806c510 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a440c5a10 0x7f0a440c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a440b1480 0x7f0a440b1900 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 --2- 172.21.15.27:0/4092566503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a300060c0 0x7f0a44142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/4092566503 >> 172.21.15.27:0/4092566503 conn(0x7f0a440afb70 msgr2=0x7f0a440cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:47.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/4092566503 shutdown_connections 2023-08-11T15:52:47.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.686+0000 7f0a4c3c2640 1 -- 172.21.15.27:0/4092566503 wait complete. 2023-08-11T15:52:47.707 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:52:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.987+0000 7f612e909640 1 Processor -- start 2023-08-11T15:52:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.987+0000 7f612e909640 1 -- start start 2023-08-11T15:52:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.989+0000 7f612e909640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612815c520 0x7f61280caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.989+0000 7f612e909640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f61280cc280 0x7f61280c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.989+0000 7f612e909640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612806b910 con 0x7f61280ad2a0 2023-08-11T15:52:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.989+0000 7f612e909640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612806bbf0 con 0x7f61280cc280 2023-08-11T15:52:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.989+0000 7f612e909640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612806bed0 con 0x7f612815c520 2023-08-11T15:52:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.989+0000 7f612d106640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f61280ad2a0 0x7f61280cbb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57566/0 (socket says 172.21.15.27:57566) 2023-08-11T15:52:48.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.989+0000 7f612d106640 1 -- 172.21.15.27:0/2426551080 learned_addr learned my addr 172.21.15.27:0/2426551080 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:48.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612d907640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612815c520 0x7f61280caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612e108640 1 --2- 172.21.15.27:0/2426551080 >> v2:172.21.15.119:3300/0 conn(0x7f61280cc280 0x7f61280c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668767645 0 0) 0x7f612806b910 con 0x7f61280ad2a0 2023-08-11T15:52:48.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6114003520 con 0x7f61280ad2a0 2023-08-11T15:52:48.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4043488766 0 0) 0x7f6114003520 con 0x7f61280ad2a0 2023-08-11T15:52:48.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612815c520 msgr2=0x7f61280caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:48.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612815c520 0x7f61280caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 >> v2:172.21.15.119:3300/0 conn(0x7f61280cc280 msgr2=0x7f61280c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 --2- 172.21.15.27:0/2426551080 >> v2:172.21.15.119:3300/0 conn(0x7f61280cc280 0x7f61280c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612806c1b0 con 0x7f61280ad2a0 2023-08-11T15:52:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f6118001670 con 0x7f61280ad2a0 2023-08-11T15:52:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 >> v1:172.21.15.27:6789/0 conn(0x7f61280ad2a0 legacy=0x7f61280cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6114003970 0x7f6114003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 0x7f6114004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612815c520 0x7f6114005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612806bbf0 con 0x7f61140047c0 2023-08-11T15:52:48.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612806bed0 con 0x7f6114003970 2023-08-11T15:52:48.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.990+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611400b980 con 0x7f612815c520 2023-08-11T15:52:48.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.991+0000 7f612d106640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 0x7f6114004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.991+0000 7f612d907640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6114003970 0x7f6114003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.991+0000 7f612e108640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612815c520 0x7f6114005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.991+0000 7f612d106640 1 -- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612815c520 msgr2=0x7f6114005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:48.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.991+0000 7f612d106640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f612815c520 0x7f6114005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.991+0000 7f612d106640 1 -- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6114003970 msgr2=0x7f6114003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:48.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.991+0000 7f612d106640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6114003970 0x7f6114003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.991+0000 7f612d106640 1 -- 172.21.15.27:0/2426551080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6114003520 con 0x7f61140047c0 2023-08-11T15:52:48.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.992+0000 7f612d106640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 0x7f6114004c40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f611800c270 tx=0x7f61140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9f17faaf43c529 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:48.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.993+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611802cae0 con 0x7f61140047c0 2023-08-11T15:52:48.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.993+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611802cc40 con 0x7f61140047c0 2023-08-11T15:52:48.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.993+0000 7f612c905640 1 -- 172.21.15.27:0/2426551080 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180058d0 con 0x7f61140047c0 2023-08-11T15:52:48.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.993+0000 7f612e909640 1 -- 172.21.15.27:0/2426551080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 msgr2=0x7f6114004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.993+0000 7f612e909640 1 --2- 172.21.15.27:0/2426551080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 0x7f6114004c40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f611800c270 tx=0x7f61140057f0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.993+0000 7f612e108640 1 -- 172.21.15.27:0/2426551080 reap_dead start 2023-08-11T15:52:48.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.993+0000 7f612e909640 1 -- 172.21.15.27:0/2426551080 shutdown_connections 2023-08-11T15:52:48.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.993+0000 7f612e909640 1 -- 172.21.15.27:0/2426551080 >> 172.21.15.27:0/2426551080 conn(0x7f61280afb70 msgr2=0x7f61280aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:48.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.993+0000 7f612e909640 1 -- 172.21.15.27:0/2426551080 shutdown_connections 2023-08-11T15:52:48.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.994+0000 7f612e909640 1 -- 172.21.15.27:0/2426551080 wait complete. 2023-08-11T15:52:48.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.995+0000 7f612e909640 1 Processor -- start 2023-08-11T15:52:48.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.995+0000 7f612e909640 1 -- start start 2023-08-11T15:52:48.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.996+0000 7f612e909640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 0x7f612814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.996+0000 7f612e909640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f61280abdb0 0x7f61280ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.996+0000 7f612e909640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6128150a60 0x7f6128150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.996+0000 7f612e909640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61280c88b0 con 0x7f61140047c0 2023-08-11T15:52:48.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.996+0000 7f612e909640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61280ac770 con 0x7f6128150a60 2023-08-11T15:52:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.996+0000 7f612e909640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612814e530 con 0x7f61280abdb0 2023-08-11T15:52:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.997+0000 7f612d907640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 0x7f612814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.997+0000 7f612d907640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 0x7f612814dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43320/0 (socket says 172.21.15.27:43320) 2023-08-11T15:52:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.997+0000 7f612d907640 1 -- 172.21.15.27:0/246735581 learned_addr learned my addr 172.21.15.27:0/246735581 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.997+0000 7f612d907640 1 -- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f61280abdb0 msgr2=0x7f61280ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.997+0000 7f612d106640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f61280abdb0 0x7f61280ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.997+0000 7f612e108640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6128150a60 0x7f6128150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.997+0000 7f612d907640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f61280abdb0 0x7f61280ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.997+0000 7f612d907640 1 -- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6128150a60 msgr2=0x7f6128150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.997+0000 7f612d907640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6128150a60 0x7f6128150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.997+0000 7f612d907640 1 -- 172.21.15.27:0/246735581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6128151790 con 0x7f61140047c0 2023-08-11T15:52:48.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.998+0000 7f612d907640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 0x7f612814dff0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6124011540 tx=0x7f6124011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:48.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.998+0000 7f611e7fc640 1 -- 172.21.15.27:0/246735581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124015930 con 0x7f61140047c0 2023-08-11T15:52:48.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.998+0000 7f611e7fc640 1 -- 172.21.15.27:0/246735581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124015a90 con 0x7f61140047c0 2023-08-11T15:52:48.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.998+0000 7f611e7fc640 1 -- 172.21.15.27:0/246735581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124015e20 con 0x7f61140047c0 2023-08-11T15:52:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.998+0000 7f612e909640 1 -- 172.21.15.27:0/246735581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61281f53c0 con 0x7f61140047c0 2023-08-11T15:52:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.998+0000 7f612e909640 1 -- 172.21.15.27:0/246735581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61281f5980 con 0x7f61140047c0 2023-08-11T15:52:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:47.999+0000 7f612e909640 1 -- 172.21.15.27:0/246735581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612804e420 con 0x7f61140047c0 2023-08-11T15:52:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.000+0000 7f611e7fc640 1 -- 172.21.15.27:0/246735581 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f612401d400 con 0x7f61140047c0 2023-08-11T15:52:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.001+0000 7f611e7fc640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a080 0x7f610c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.002+0000 7f611e7fc640 1 -- 172.21.15.27:0/246735581 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6124088fc0 con 0x7f61140047c0 2023-08-11T15:52:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.002+0000 7f612d106640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a080 0x7f610c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.003+0000 7f612d106640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a080 0x7f610c06c540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6118034000 tx=0x7f6118007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.006+0000 7f611e7fc640 1 -- 172.21.15.27:0/246735581 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f61240542e0 con 0x7f61140047c0 2023-08-11T15:52:48.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.199+0000 7f612e909640 1 -- 172.21.15.27:0/246735581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61280c9700 con 0x7f61140047c0 2023-08-11T15:52:48.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.201+0000 7f611e7fc640 1 -- 172.21.15.27:0/246735581 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20216 (secure 0 0 0) 0x7f6124057900 con 0x7f61140047c0 2023-08-11T15:52:48.203 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:52:48.203 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:51:40.657727+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:50:59.019247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-11T15:51:00.247433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:51:03.863616+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:52:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 -- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a080 msgr2=0x7f610c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a080 0x7f610c06c540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6118034000 tx=0x7f6118007000 comp rx=0 tx=0).stop 2023-08-11T15:52:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 -- 172.21.15.27:0/246735581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 msgr2=0x7f612814dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 0x7f612814dff0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6124011540 tx=0x7f6124011a00 comp rx=0 tx=0).stop 2023-08-11T15:52:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 -- 172.21.15.27:0/246735581 shutdown_connections 2023-08-11T15:52:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f610c06a080 0x7f610c06c540 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f61280abdb0 0x7f61280ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6128150a60 0x7f6128150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 --2- 172.21.15.27:0/246735581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61140047c0 0x7f612814dff0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 -- 172.21.15.27:0/246735581 >> 172.21.15.27:0/246735581 conn(0x7f61280afb70 msgr2=0x7f6128070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.203+0000 7f612e909640 1 -- 172.21.15.27:0/246735581 shutdown_connections 2023-08-11T15:52:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.204+0000 7f612e909640 1 -- 172.21.15.27:0/246735581 wait complete. 2023-08-11T15:52:48.228 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-11T15:52:48.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.462+0000 7f1b3ce52640 1 Processor -- start 2023-08-11T15:52:48.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.462+0000 7f1b3ce52640 1 -- start start 2023-08-11T15:52:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.463+0000 7f1b3ce52640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b380cd500 0x7f1b3815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.463+0000 7f1b3ce52640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b380cf670 con 0x7f1b380b7ab0 2023-08-11T15:52:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.463+0000 7f1b3ce52640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b38160e50 con 0x7f1b3815c3c0 2023-08-11T15:52:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.463+0000 7f1b3ce52640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b3815c7a0 con 0x7f1b380cd500 2023-08-11T15:52:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.464+0000 7f1b36ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b380cd500 0x7f1b3815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.464+0000 7f1b36ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b380cd500 0x7f1b3815feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43326/0 (socket says 172.21.15.27:43326) 2023-08-11T15:52:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.464+0000 7f1b36ffd640 1 -- 172.21.15.27:0/1179232056 learned_addr learned my addr 172.21.15.27:0/1179232056 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.464+0000 7f1b37fff640 1 --1- 172.21.15.27:0/1179232056 >> v1:172.21.15.27:6789/0 conn(0x7f1b380b7ab0 0x7f1b380d0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57578/0 (socket says 172.21.15.27:57578) 2023-08-11T15:52:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.464+0000 7f1b36ffd640 1 -- 172.21.15.27:0/1179232056 >> v1:172.21.15.119:6789/0 conn(0x7f1b3815c3c0 legacy=0x7f1b380ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.464+0000 7f1b36ffd640 1 -- 172.21.15.27:0/1179232056 >> v1:172.21.15.27:6789/0 conn(0x7f1b380b7ab0 legacy=0x7f1b380d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.464+0000 7f1b36ffd640 1 -- 172.21.15.27:0/1179232056 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b381605b0 con 0x7f1b380cd500 2023-08-11T15:52:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.466+0000 7f1b36ffd640 1 --2- 172.21.15.27:0/1179232056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 0x7f1b3815feb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009870 tx=0x7f1b2c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2780d6305904bdf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.466+0000 7f1b367fc640 1 -- 172.21.15.27:0/1179232056 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c030860 con 0x7f1b380cd500 2023-08-11T15:52:48.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.466+0000 7f1b367fc640 1 -- 172.21.15.27:0/1179232056 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c0309c0 con 0x7f1b380cd500 2023-08-11T15:52:48.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.466+0000 7f1b367fc640 1 -- 172.21.15.27:0/1179232056 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c030d50 con 0x7f1b380cd500 2023-08-11T15:52:48.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.467+0000 7f1b3ce52640 1 -- 172.21.15.27:0/1179232056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 msgr2=0x7f1b3815feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.467+0000 7f1b3ce52640 1 --2- 172.21.15.27:0/1179232056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 0x7f1b3815feb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009870 tx=0x7f1b2c02b9a0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.467+0000 7f1b3ce52640 1 -- 172.21.15.27:0/1179232056 shutdown_connections 2023-08-11T15:52:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.467+0000 7f1b3ce52640 1 --2- 172.21.15.27:0/1179232056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 0x7f1b3815feb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.467+0000 7f1b3ce52640 1 -- 172.21.15.27:0/1179232056 >> 172.21.15.27:0/1179232056 conn(0x7f1b3806a190 msgr2=0x7f1b3806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.467+0000 7f1b3ce52640 1 -- 172.21.15.27:0/1179232056 shutdown_connections 2023-08-11T15:52:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.467+0000 7f1b3ce52640 1 -- 172.21.15.27:0/1179232056 wait complete. 2023-08-11T15:52:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.469+0000 7f1b3ce52640 1 Processor -- start 2023-08-11T15:52:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.469+0000 7f1b3ce52640 1 -- start start 2023-08-11T15:52:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.470+0000 7f1b3ce52640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1b380b7ab0 0x7f1b381495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.470+0000 7f1b3ce52640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 0x7f1b380b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.470+0000 7f1b3ce52640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3815c3c0 0x7f1b380c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.470+0000 7f1b3ce52640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3815ea50 con 0x7f1b380cd500 2023-08-11T15:52:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.470+0000 7f1b3ce52640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b38149ae0 con 0x7f1b3815c3c0 2023-08-11T15:52:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.470+0000 7f1b3ce52640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3815e0b0 con 0x7f1b380b7ab0 2023-08-11T15:52:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b36ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 0x7f1b380b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b36ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 0x7f1b380b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43332/0 (socket says 172.21.15.27:43332) 2023-08-11T15:52:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b36ffd640 1 -- 172.21.15.27:0/226303596 learned_addr learned my addr 172.21.15.27:0/226303596 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b377fe640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1b380b7ab0 0x7f1b381495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b37fff640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3815c3c0 0x7f1b380c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b36ffd640 1 -- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1b380b7ab0 msgr2=0x7f1b381495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b36ffd640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1b380b7ab0 0x7f1b381495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b36ffd640 1 -- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3815c3c0 msgr2=0x7f1b380c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b36ffd640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3815c3c0 0x7f1b380c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b36ffd640 1 -- 172.21.15.27:0/226303596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c0094d0 con 0x7f1b380cd500 2023-08-11T15:52:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.471+0000 7f1b377fe640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1b380b7ab0 0x7f1b381495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.472+0000 7f1b36ffd640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 0x7f1b380b1690 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c000c00 tx=0x7f1b2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.472+0000 7f1b1ffff640 1 -- 172.21.15.27:0/226303596 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c030830 con 0x7f1b380cd500 2023-08-11T15:52:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.472+0000 7f1b1ffff640 1 -- 172.21.15.27:0/226303596 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c039d90 con 0x7f1b380cd500 2023-08-11T15:52:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.472+0000 7f1b1ffff640 1 -- 172.21.15.27:0/226303596 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c043670 con 0x7f1b380cd500 2023-08-11T15:52:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.472+0000 7f1b3ce52640 1 -- 172.21.15.27:0/226303596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3815e390 con 0x7f1b380cd500 2023-08-11T15:52:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.472+0000 7f1b3ce52640 1 -- 172.21.15.27:0/226303596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b380cc350 con 0x7f1b380cd500 2023-08-11T15:52:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.473+0000 7f1b3ce52640 1 -- 172.21.15.27:0/226303596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b380cc5a0 con 0x7f1b380cd500 2023-08-11T15:52:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.475+0000 7f1b1ffff640 1 -- 172.21.15.27:0/226303596 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1b2c02f3b0 con 0x7f1b380cd500 2023-08-11T15:52:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.475+0000 7f1b1ffff640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1b2406a050 0x7f1b2406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.476+0000 7f1b1ffff640 1 -- 172.21.15.27:0/226303596 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1b2c0aacd0 con 0x7f1b380cd500 2023-08-11T15:52:48.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.476+0000 7f1b377fe640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1b2406a050 0x7f1b2406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.478+0000 7f1b377fe640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1b2406a050 0x7f1b2406c510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1b280012e0 tx=0x7f1b28009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:48.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.480+0000 7f1b1ffff640 1 -- 172.21.15.27:0/226303596 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f1b2c076240 con 0x7f1b380cd500 2023-08-11T15:52:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.716+0000 7f1b3ce52640 1 -- 172.21.15.27:0/226303596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1b3815eb90 con 0x7f1b380cd500 2023-08-11T15:52:48.719 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-08-11T15:52:48.718+0000 7f6b67b3c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-11T15:52:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.730+0000 7f1b1ffff640 1 -- 172.21.15.27:0/226303596 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1b3815eb90 con 0x7f1b380cd500 2023-08-11T15:52:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-11T15:52:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 -- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1b2406a050 msgr2=0x7f1b2406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1b2406a050 0x7f1b2406c510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1b280012e0 tx=0x7f1b28009040 comp rx=0 tx=0).stop 2023-08-11T15:52:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 -- 172.21.15.27:0/226303596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 msgr2=0x7f1b380b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 0x7f1b380b1690 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c000c00 tx=0x7f1b2c004730 comp rx=0 tx=0).stop 2023-08-11T15:52:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 -- 172.21.15.27:0/226303596 shutdown_connections 2023-08-11T15:52:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1b2406a050 0x7f1b2406c510 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3815c3c0 0x7f1b380c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b380cd500 0x7f1b380b1690 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 --2- 172.21.15.27:0/226303596 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1b380b7ab0 0x7f1b381495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 -- 172.21.15.27:0/226303596 >> 172.21.15.27:0/226303596 conn(0x7f1b3806a190 msgr2=0x7f1b380723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 -- 172.21.15.27:0/226303596 shutdown_connections 2023-08-11T15:52:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.734+0000 7f1b3ce52640 1 -- 172.21.15.27:0/226303596 wait complete. 2023-08-11T15:52:48.754 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-11T15:52:48.765 INFO:tasks.ceph.mds.b.smithi119.stderr: -5890> 2023-08-11T15:51:05.885+0000 7f599d7c5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-11T15:52:48.855 INFO:tasks.ceph.mds.b.smithi119.stdout:starting mds.b at 2023-08-11T15:52:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.993+0000 7f3932e45640 1 Processor -- start 2023-08-11T15:52:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.993+0000 7f3932e45640 1 -- start start 2023-08-11T15:52:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.995+0000 7f3932e45640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f392c0b5390 0x7f392c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.995+0000 7f3932e45640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0cd620 0x7f392c0d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.995+0000 7f3932e45640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392c0d1dc0 con 0x7f392c15f8d0 2023-08-11T15:52:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.995+0000 7f3932e45640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392c0d2050 con 0x7f392c0b5390 2023-08-11T15:52:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.995+0000 7f3932e45640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c0b3df0 con 0x7f392c0cd620 2023-08-11T15:52:48.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.995+0000 7f3931642640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f392c0b5390 0x7f392c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.995+0000 7f3932644640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0cd620 0x7f392c0d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:48.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.996+0000 7f3931642640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f392c0b5390 0x7f392c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35084/0 (socket says 172.21.15.27:35084) 2023-08-11T15:52:48.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.996+0000 7f3931642640 1 -- 172.21.15.27:0/2546880261 learned_addr learned my addr 172.21.15.27:0/2546880261 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:48.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.996+0000 7f3932644640 1 -- 172.21.15.27:0/2546880261 >> v2:172.21.15.119:3300/0 conn(0x7f392c0b5390 msgr2=0x7f392c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:48.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.996+0000 7f3932644640 1 --2- 172.21.15.27:0/2546880261 >> v2:172.21.15.119:3300/0 conn(0x7f392c0b5390 0x7f392c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:48.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.996+0000 7f3932644640 1 -- 172.21.15.27:0/2546880261 >> v1:172.21.15.119:6789/0 conn(0x7f392c15f8d0 legacy=0x7f392c0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:48.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.996+0000 7f3932644640 1 -- 172.21.15.27:0/2546880261 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c15fcb0 con 0x7f392c0cd620 2023-08-11T15:52:48.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.998+0000 7f3932644640 1 --2- 172.21.15.27:0/2546880261 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0cd620 0x7f392c0d1880 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39240097a0 tx=0x7f392402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1256d48e3bdeb128 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:49.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.998+0000 7f3930e41640 1 -- 172.21.15.27:0/2546880261 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924031830 con 0x7f392c0cd620 2023-08-11T15:52:49.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.998+0000 7f3930e41640 1 -- 172.21.15.27:0/2546880261 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3924031990 con 0x7f392c0cd620 2023-08-11T15:52:49.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.998+0000 7f3930e41640 1 -- 172.21.15.27:0/2546880261 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924031d20 con 0x7f392c0cd620 2023-08-11T15:52:49.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.999+0000 7f3932e45640 1 -- 172.21.15.27:0/2546880261 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0cd620 msgr2=0x7f392c0d1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:49.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.999+0000 7f3932e45640 1 --2- 172.21.15.27:0/2546880261 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0cd620 0x7f392c0d1880 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39240097a0 tx=0x7f392402b8d0 comp rx=0 tx=0).stop 2023-08-11T15:52:49.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.999+0000 7f3932e45640 1 -- 172.21.15.27:0/2546880261 shutdown_connections 2023-08-11T15:52:49.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.999+0000 7f3932e45640 1 --2- 172.21.15.27:0/2546880261 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0cd620 0x7f392c0d1880 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:49.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.999+0000 7f3932e45640 1 --2- 172.21.15.27:0/2546880261 >> v2:172.21.15.119:3300/0 conn(0x7f392c0b5390 0x7f392c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:49.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.999+0000 7f3932e45640 1 -- 172.21.15.27:0/2546880261 >> 172.21.15.27:0/2546880261 conn(0x7f392c06a250 msgr2=0x7f392c06a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:49.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.999+0000 7f3932e45640 1 -- 172.21.15.27:0/2546880261 shutdown_connections 2023-08-11T15:52:49.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:48.999+0000 7f3932e45640 1 -- 172.21.15.27:0/2546880261 wait complete. 2023-08-11T15:52:49.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.001+0000 7f3932e45640 1 Processor -- start 2023-08-11T15:52:49.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.001+0000 7f3932e45640 1 -- start start 2023-08-11T15:52:49.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.002+0000 7f3932e45640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0b5390 0x7f392c158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:49.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.002+0000 7f3932e45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392c0cd620 0x7f392c157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:49.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.002+0000 7f3932e45640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392c15f8d0 0x7f392c0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:49.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.002+0000 7f3932e45640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c15a070 con 0x7f392c15f8d0 2023-08-11T15:52:49.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.002+0000 7f3932e45640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c159810 con 0x7f392c0cd620 2023-08-11T15:52:49.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.002+0000 7f3932e45640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c159ab0 con 0x7f392c0b5390 2023-08-11T15:52:49.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.003+0000 7f3932644640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392c15f8d0 0x7f392c0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:49.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.003+0000 7f3932644640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392c15f8d0 0x7f392c0ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43344/0 (socket says 172.21.15.27:43344) 2023-08-11T15:52:49.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.003+0000 7f3932644640 1 -- 172.21.15.27:0/1966747271 learned_addr learned my addr 172.21.15.27:0/1966747271 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:49.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.003+0000 7f3931642640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392c0cd620 0x7f392c157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:49.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.003+0000 7f3931e43640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0b5390 0x7f392c158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:49.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.003+0000 7f3932644640 1 -- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0b5390 msgr2=0x7f392c158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:49.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.003+0000 7f3932644640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0b5390 0x7f392c158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.003+0000 7f3932644640 1 -- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392c0cd620 msgr2=0x7f392c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.003+0000 7f3932644640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392c0cd620 0x7f392c157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.003+0000 7f3932644640 1 -- 172.21.15.27:0/1966747271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924009400 con 0x7f392c15f8d0 2023-08-11T15:52:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.004+0000 7f3932644640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392c15f8d0 0x7f392c0ca2f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f39240097a0 tx=0x7f39240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.004+0000 7f39227fc640 1 -- 172.21.15.27:0/1966747271 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392403a630 con 0x7f392c15f8d0 2023-08-11T15:52:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.004+0000 7f39227fc640 1 -- 172.21.15.27:0/1966747271 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392403a790 con 0x7f392c15f8d0 2023-08-11T15:52:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.004+0000 7f39227fc640 1 -- 172.21.15.27:0/1966747271 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392403ab40 con 0x7f392c15f8d0 2023-08-11T15:52:49.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.004+0000 7f3932e45640 1 -- 172.21.15.27:0/1966747271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c159d90 con 0x7f392c15f8d0 2023-08-11T15:52:49.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.004+0000 7f3932e45640 1 -- 172.21.15.27:0/1966747271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c0c7240 con 0x7f392c15f8d0 2023-08-11T15:52:49.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.005+0000 7f3932e45640 1 -- 172.21.15.27:0/1966747271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c0c6dc0 con 0x7f392c15f8d0 2023-08-11T15:52:49.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.006+0000 7f39227fc640 1 -- 172.21.15.27:0/1966747271 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f392403aca0 con 0x7f392c15f8d0 2023-08-11T15:52:49.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.007+0000 7f39227fc640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f391006a050 0x7f391006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:49.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.007+0000 7f39227fc640 1 -- 172.21.15.27:0/1966747271 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f39240aa250 con 0x7f392c15f8d0 2023-08-11T15:52:49.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.008+0000 7f3931e43640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f391006a050 0x7f391006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.009+0000 7f3931e43640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f391006a050 0x7f391006c510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39280012e0 tx=0x7f3928009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.012+0000 7f39227fc640 1 -- 172.21.15.27:0/1966747271 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f392c0c6dc0 con 0x7f392c15f8d0 2023-08-11T15:52:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.245+0000 7f3932e45640 1 -- 172.21.15.27:0/1966747271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f392c0c6dc0 con 0x7f392c15f8d0 2023-08-11T15:52:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.736+0000 7f39227fc640 1 -- 172.21.15.27:0/1966747271 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f392c0c6dc0 con 0x7f392c15f8d0 2023-08-11T15:52:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.739+0000 7f3932e45640 1 -- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f391006a050 msgr2=0x7f391006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.739+0000 7f3932e45640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f391006a050 0x7f391006c510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39280012e0 tx=0x7f3928009040 comp rx=0 tx=0).stop 2023-08-11T15:52:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.739+0000 7f3932e45640 1 -- 172.21.15.27:0/1966747271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392c15f8d0 msgr2=0x7f392c0ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.739+0000 7f3932e45640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392c15f8d0 0x7f392c0ca2f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f39240097a0 tx=0x7f39240046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.740+0000 7f3932e45640 1 -- 172.21.15.27:0/1966747271 shutdown_connections 2023-08-11T15:52:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.740+0000 7f3932e45640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f391006a050 0x7f391006c510 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.740+0000 7f3932e45640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392c15f8d0 0x7f392c0ca2f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.740+0000 7f3932e45640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392c0cd620 0x7f392c157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.740+0000 7f3932e45640 1 --2- 172.21.15.27:0/1966747271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f392c0b5390 0x7f392c158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.740+0000 7f3932e45640 1 -- 172.21.15.27:0/1966747271 >> 172.21.15.27:0/1966747271 conn(0x7f392c06a250 msgr2=0x7f392c0b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.740+0000 7f3932e45640 1 -- 172.21.15.27:0/1966747271 shutdown_connections 2023-08-11T15:52:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:49.740+0000 7f3932e45640 1 -- 172.21.15.27:0/1966747271 wait complete. 2023-08-11T15:52:49.762 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-11T15:52:50.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.008+0000 7f48f08ec640 1 Processor -- start 2023-08-11T15:52:50.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.008+0000 7f48f08ec640 1 -- start start 2023-08-11T15:52:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.009+0000 7f48f08ec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f48ec15c620 0x7f48ec0b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.010+0000 7f48f08ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 0x7f48ec0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.010+0000 7f48f08ec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec0cda30 con 0x7f48ec0b55c0 2023-08-11T15:52:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.010+0000 7f48f08ec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec0ccc10 con 0x7f48ec15c620 2023-08-11T15:52:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.010+0000 7f48f08ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec161110 con 0x7f48ec0b82f0 2023-08-11T15:52:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.010+0000 7f48ea59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f48ec15c620 0x7f48ec0b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.010+0000 7f48ea59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f48ec15c620 0x7f48ec0b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47470/0 (socket says 172.21.15.27:47470) 2023-08-11T15:52:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.010+0000 7f48ea59c640 1 -- 172.21.15.27:0/2277376649 learned_addr learned my addr 172.21.15.27:0/2277376649 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:50.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.010+0000 7f48ead9d640 1 --1- 172.21.15.27:0/2277376649 >> v1:172.21.15.27:6789/0 conn(0x7f48ec0b55c0 0x7f48ec0b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42710/0 (socket says 172.21.15.27:42710) 2023-08-11T15:52:50.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.010+0000 7f48e9d9b640 1 --2- 172.21.15.27:0/2277376649 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 0x7f48ec0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:50.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.011+0000 7f48ea59c640 1 -- 172.21.15.27:0/2277376649 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 msgr2=0x7f48ec0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:50.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.011+0000 7f48ea59c640 1 --2- 172.21.15.27:0/2277376649 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 0x7f48ec0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:50.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.011+0000 7f48ea59c640 1 -- 172.21.15.27:0/2277376649 >> v1:172.21.15.27:6789/0 conn(0x7f48ec0b55c0 legacy=0x7f48ec0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:50.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.011+0000 7f48ea59c640 1 -- 172.21.15.27:0/2277376649 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec0cce20 con 0x7f48ec15c620 2023-08-11T15:52:50.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.011+0000 7f48e9d9b640 1 --2- 172.21.15.27:0/2277376649 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 0x7f48ec0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:50.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.012+0000 7f48ea59c640 1 --2- 172.21.15.27:0/2277376649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 0x7f48ec0b7db0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009870 tx=0x7f48d402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c41bbe5ae5eb4d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.012+0000 7f48e959a640 1 -- 172.21.15.27:0/2277376649 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4030840 con 0x7f48ec15c620 2023-08-11T15:52:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.012+0000 7f48e959a640 1 -- 172.21.15.27:0/2277376649 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d40309a0 con 0x7f48ec15c620 2023-08-11T15:52:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.012+0000 7f48e959a640 1 -- 172.21.15.27:0/2277376649 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4030d30 con 0x7f48ec15c620 2023-08-11T15:52:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.013+0000 7f48f08ec640 1 -- 172.21.15.27:0/2277376649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 msgr2=0x7f48ec0b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.013+0000 7f48f08ec640 1 --2- 172.21.15.27:0/2277376649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 0x7f48ec0b7db0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009870 tx=0x7f48d402b980 comp rx=0 tx=0).stop 2023-08-11T15:52:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.013+0000 7f48f08ec640 1 -- 172.21.15.27:0/2277376649 shutdown_connections 2023-08-11T15:52:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.013+0000 7f48f08ec640 1 --2- 172.21.15.27:0/2277376649 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 0x7f48ec0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.013+0000 7f48f08ec640 1 --2- 172.21.15.27:0/2277376649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 0x7f48ec0b7db0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.013+0000 7f48f08ec640 1 -- 172.21.15.27:0/2277376649 >> 172.21.15.27:0/2277376649 conn(0x7f48ec071110 msgr2=0x7f48ec0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.013+0000 7f48f08ec640 1 -- 172.21.15.27:0/2277376649 shutdown_connections 2023-08-11T15:52:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.013+0000 7f48f08ec640 1 -- 172.21.15.27:0/2277376649 wait complete. 2023-08-11T15:52:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.015+0000 7f48f08ec640 1 Processor -- start 2023-08-11T15:52:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.015+0000 7f48f08ec640 1 -- start start 2023-08-11T15:52:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.016+0000 7f48f08ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48ec0b55c0 0x7f48ec158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.016+0000 7f48f08ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 0x7f48ec0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.016+0000 7f48f08ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 0x7f48ec0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.016+0000 7f48f08ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0ca1e0 con 0x7f48ec15c620 2023-08-11T15:52:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.016+0000 7f48f08ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0ca4c0 con 0x7f48ec0b55c0 2023-08-11T15:52:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.016+0000 7f48f08ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0caa20 con 0x7f48ec0b82f0 2023-08-11T15:52:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.016+0000 7f48ead9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 0x7f48ec0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.016+0000 7f48ead9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 0x7f48ec0c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47486/0 (socket says 172.21.15.27:47486) 2023-08-11T15:52:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.016+0000 7f48ead9d640 1 -- 172.21.15.27:0/2348183532 learned_addr learned my addr 172.21.15.27:0/2348183532 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48e9d9b640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 0x7f48ec0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48ead9d640 1 -- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 msgr2=0x7f48ec0c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48ead9d640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 0x7f48ec0c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48ea59c640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48ec0b55c0 0x7f48ec158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48ead9d640 1 -- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48ec0b55c0 msgr2=0x7f48ec158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48ead9d640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48ec0b55c0 0x7f48ec158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48ead9d640 1 -- 172.21.15.27:0/2348183532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d40094d0 con 0x7f48ec15c620 2023-08-11T15:52:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48ea59c640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48ec0b55c0 0x7f48ec158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48ead9d640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 0x7f48ec0c9bd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c400 tx=0x7f48e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48e2ffd640 1 -- 172.21.15.27:0/2348183532 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40074e0 con 0x7f48ec15c620 2023-08-11T15:52:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48e2ffd640 1 -- 172.21.15.27:0/2348183532 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e4007640 con 0x7f48ec15c620 2023-08-11T15:52:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48e2ffd640 1 -- 172.21.15.27:0/2348183532 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40049e0 con 0x7f48ec15c620 2023-08-11T15:52:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.017+0000 7f48f08ec640 1 -- 172.21.15.27:0/2348183532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec0cad00 con 0x7f48ec15c620 2023-08-11T15:52:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.018+0000 7f48f08ec640 1 -- 172.21.15.27:0/2348183532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec15faf0 con 0x7f48ec15c620 2023-08-11T15:52:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.019+0000 7f48f08ec640 1 -- 172.21.15.27:0/2348183532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec0cb220 con 0x7f48ec15c620 2023-08-11T15:52:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.020+0000 7f48e2ffd640 1 -- 172.21.15.27:0/2348183532 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f48e4004b40 con 0x7f48ec15c620 2023-08-11T15:52:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.021+0000 7f48e2ffd640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f48cc06a050 0x7f48cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.021+0000 7f48e2ffd640 1 -- 172.21.15.27:0/2348183532 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f48e4088250 con 0x7f48ec15c620 2023-08-11T15:52:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.021+0000 7f48ea59c640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f48cc06a050 0x7f48cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.023+0000 7f48ea59c640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f48cc06a050 0x7f48cc06c510 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f48d4036000 tx=0x7f48d403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.025+0000 7f48e2ffd640 1 -- 172.21.15.27:0/2348183532 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f48ec0cb220 con 0x7f48ec15c620 2023-08-11T15:52:50.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.222+0000 7f48f08ec640 1 -- 172.21.15.27:0/2348183532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48ec0cb220 con 0x7f48ec15c620 2023-08-11T15:52:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.743+0000 7f48e2ffd640 1 -- 172.21.15.27:0/2348183532 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) v1 ==== 181+0+0 (secure 0 0 0) 0x7f48ec0cb220 con 0x7f48ec15c620 2023-08-11T15:52:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-08-11T15:52:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.749+0000 7f48f08ec640 1 -- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f48cc06a050 msgr2=0x7f48cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.749+0000 7f48f08ec640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f48cc06a050 0x7f48cc06c510 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f48d4036000 tx=0x7f48d403b040 comp rx=0 tx=0).stop 2023-08-11T15:52:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.749+0000 7f48f08ec640 1 -- 172.21.15.27:0/2348183532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 msgr2=0x7f48ec0c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.749+0000 7f48f08ec640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 0x7f48ec0c9bd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c400 tx=0x7f48e400c8c0 comp rx=0 tx=0).stop 2023-08-11T15:52:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.749+0000 7f48f08ec640 1 -- 172.21.15.27:0/2348183532 shutdown_connections 2023-08-11T15:52:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.749+0000 7f48f08ec640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f48cc06a050 0x7f48cc06c510 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.749+0000 7f48f08ec640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48ec0b82f0 0x7f48ec0c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.749+0000 7f48f08ec640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48ec15c620 0x7f48ec0c9bd0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.749+0000 7f48f08ec640 1 --2- 172.21.15.27:0/2348183532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48ec0b55c0 0x7f48ec158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.749+0000 7f48f08ec640 1 -- 172.21.15.27:0/2348183532 >> 172.21.15.27:0/2348183532 conn(0x7f48ec071110 msgr2=0x7f48ec0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.750+0000 7f48f08ec640 1 -- 172.21.15.27:0/2348183532 shutdown_connections 2023-08-11T15:52:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:50.750+0000 7f48f08ec640 1 -- 172.21.15.27:0/2348183532 wait complete. 2023-08-11T15:52:50.775 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-11T15:52:51.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.012+0000 7f2d1708a640 1 Processor -- start 2023-08-11T15:52:51.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.013+0000 7f2d1708a640 1 -- start start 2023-08-11T15:52:51.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.014+0000 7f2d1708a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2d1015fba0 0x7f2d100cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:51.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.014+0000 7f2d1708a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d100d0df0 con 0x7f2d100b43a0 2023-08-11T15:52:51.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.014+0000 7f2d1708a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d100d1040 con 0x7f2d100cf820 2023-08-11T15:52:51.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.014+0000 7f2d1708a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100d1320 con 0x7f2d1015fba0 2023-08-11T15:52:51.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.014+0000 7f2d16088640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2d1015fba0 0x7f2d100cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:51.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.014+0000 7f2d16889640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2d100b43a0 0x7f2d100b5440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42716/0 (socket says 172.21.15.27:42716) 2023-08-11T15:52:51.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.014+0000 7f2d16889640 1 -- 172.21.15.27:0/1431844192 learned_addr learned my addr 172.21.15.27:0/1431844192 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:51.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.015+0000 7f2d16088640 1 -- 172.21.15.27:0/1431844192 >> v1:172.21.15.119:6789/0 conn(0x7f2d100cf820 legacy=0x7f2d100b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:51.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.015+0000 7f2d16088640 1 -- 172.21.15.27:0/1431844192 >> v1:172.21.15.27:6789/0 conn(0x7f2d100b43a0 legacy=0x7f2d100b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:51.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.015+0000 7f2d16088640 1 -- 172.21.15.27:0/1431844192 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100d1600 con 0x7f2d1015fba0 2023-08-11T15:52:51.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.015+0000 7f2d15086640 1 -- 172.21.15.27:0/1431844192 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785395005 0 0) 0x7f2d100d0df0 con 0x7f2d100b43a0 2023-08-11T15:52:51.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.016+0000 7f2d16088640 1 --2- 172.21.15.27:0/1431844192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d1015fba0 0x7f2d100cca70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0097a0 tx=0x7f2d0c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15757e5753da288 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:51.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.016+0000 7f2d15086640 1 -- 172.21.15.27:0/1431844192 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c030610 con 0x7f2d1015fba0 2023-08-11T15:52:51.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.016+0000 7f2d15086640 1 -- 172.21.15.27:0/1431844192 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c030770 con 0x7f2d1015fba0 2023-08-11T15:52:51.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.016+0000 7f2d15086640 1 -- 172.21.15.27:0/1431844192 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c030b00 con 0x7f2d1015fba0 2023-08-11T15:52:51.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.017+0000 7f2d1708a640 1 -- 172.21.15.27:0/1431844192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d1015fba0 msgr2=0x7f2d100cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:51.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.017+0000 7f2d1708a640 1 --2- 172.21.15.27:0/1431844192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d1015fba0 0x7f2d100cca70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0097a0 tx=0x7f2d0c02b960 comp rx=0 tx=0).stop 2023-08-11T15:52:51.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.017+0000 7f2d1708a640 1 -- 172.21.15.27:0/1431844192 shutdown_connections 2023-08-11T15:52:51.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.017+0000 7f2d1708a640 1 --2- 172.21.15.27:0/1431844192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d1015fba0 0x7f2d100cca70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:51.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.017+0000 7f2d1708a640 1 -- 172.21.15.27:0/1431844192 >> 172.21.15.27:0/1431844192 conn(0x7f2d100cec90 msgr2=0x7f2d100cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:51.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.017+0000 7f2d1708a640 1 -- 172.21.15.27:0/1431844192 shutdown_connections 2023-08-11T15:52:51.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.017+0000 7f2d1708a640 1 -- 172.21.15.27:0/1431844192 wait complete. 2023-08-11T15:52:51.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.019+0000 7f2d1708a640 1 Processor -- start 2023-08-11T15:52:51.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.019+0000 7f2d1708a640 1 -- start start 2023-08-11T15:52:51.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.020+0000 7f2d1708a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d100b43a0 0x7f2d101f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:51.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.020+0000 7f2d1708a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d100cf820 0x7f2d1015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:51.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.020+0000 7f2d1708a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d1015fba0 0x7f2d1015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:51.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.020+0000 7f2d1708a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100c5b70 con 0x7f2d100cf820 2023-08-11T15:52:51.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.020+0000 7f2d1708a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100c66f0 con 0x7f2d100b43a0 2023-08-11T15:52:51.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.020+0000 7f2d1708a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100c6b80 con 0x7f2d1015fba0 2023-08-11T15:52:51.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.020+0000 7f2d15887640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d100cf820 0x7f2d1015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:51.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.020+0000 7f2d15887640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d100cf820 0x7f2d1015d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47504/0 (socket says 172.21.15.27:47504) 2023-08-11T15:52:51.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d15887640 1 -- 172.21.15.27:0/1112474650 learned_addr learned my addr 172.21.15.27:0/1112474650 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:51.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d16088640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d100b43a0 0x7f2d101f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:51.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d15887640 1 -- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d1015fba0 msgr2=0x7f2d1015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:51.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d16889640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d1015fba0 0x7f2d1015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:51.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d15887640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d1015fba0 0x7f2d1015de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:51.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d15887640 1 -- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d100b43a0 msgr2=0x7f2d101f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:51.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d15887640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d100b43a0 0x7f2d101f58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:51.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d15887640 1 -- 172.21.15.27:0/1112474650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c009400 con 0x7f2d100cf820 2023-08-11T15:52:51.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d16088640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d100b43a0 0x7f2d101f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:51.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d16889640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d1015fba0 0x7f2d1015de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.021+0000 7f2d15887640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d100cf820 0x7f2d1015d8c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000c440 tx=0x7f2d0000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.022+0000 7f2d06ffd640 1 -- 172.21.15.27:0/1112474650 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00005480 con 0x7f2d100cf820 2023-08-11T15:52:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.022+0000 7f2d06ffd640 1 -- 172.21.15.27:0/1112474650 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d000035f0 con 0x7f2d100cf820 2023-08-11T15:52:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.022+0000 7f2d06ffd640 1 -- 172.21.15.27:0/1112474650 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d000039c0 con 0x7f2d100cf820 2023-08-11T15:52:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.022+0000 7f2d1708a640 1 -- 172.21.15.27:0/1112474650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d100c5dc0 con 0x7f2d100cf820 2023-08-11T15:52:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.022+0000 7f2d1708a640 1 -- 172.21.15.27:0/1112474650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d100c6380 con 0x7f2d100cf820 2023-08-11T15:52:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.023+0000 7f2d1708a640 1 -- 172.21.15.27:0/1112474650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1004e420 con 0x7f2d100cf820 2023-08-11T15:52:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.024+0000 7f2d06ffd640 1 -- 172.21.15.27:0/1112474650 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2d00003b20 con 0x7f2d100cf820 2023-08-11T15:52:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.025+0000 7f2d06ffd640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cf406a050 0x7f2cf406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.025+0000 7f2d06ffd640 1 -- 172.21.15.27:0/1112474650 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f2d000878c0 con 0x7f2d100cf820 2023-08-11T15:52:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.026+0000 7f2d16088640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cf406a050 0x7f2cf406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.027+0000 7f2d16088640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cf406a050 0x7f2cf406c510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c036000 tx=0x7f2d0c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.030+0000 7f2d06ffd640 1 -- 172.21.15.27:0/1112474650 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2d00056900 con 0x7f2d100cf820 2023-08-11T15:52:51.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.224+0000 7f2d1708a640 1 -- 172.21.15.27:0/1112474650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d100cd7d0 con 0x7f2d100cf820 2023-08-11T15:52:51.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.759+0000 7f2d06ffd640 1 -- 172.21.15.27:0/1112474650 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2d00056490 con 0x7f2d100cf820 2023-08-11T15:52:51.761 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-08-11T15:52:51.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.763+0000 7f2d1708a640 1 -- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cf406a050 msgr2=0x7f2cf406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:51.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.763+0000 7f2d1708a640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cf406a050 0x7f2cf406c510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c036000 tx=0x7f2d0c03b040 comp rx=0 tx=0).stop 2023-08-11T15:52:51.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.764+0000 7f2d1708a640 1 -- 172.21.15.27:0/1112474650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d100cf820 msgr2=0x7f2d1015d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:51.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.764+0000 7f2d1708a640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d100cf820 0x7f2d1015d8c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000c440 tx=0x7f2d0000c900 comp rx=0 tx=0).stop 2023-08-11T15:52:51.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.764+0000 7f2d1708a640 1 -- 172.21.15.27:0/1112474650 shutdown_connections 2023-08-11T15:52:51.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.764+0000 7f2d1708a640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2cf406a050 0x7f2cf406c510 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:51.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.764+0000 7f2d1708a640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2d1015fba0 0x7f2d1015de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:51.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.764+0000 7f2d1708a640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d100cf820 0x7f2d1015d8c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:51.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.764+0000 7f2d1708a640 1 --2- 172.21.15.27:0/1112474650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d100b43a0 0x7f2d101f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:51.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.764+0000 7f2d1708a640 1 -- 172.21.15.27:0/1112474650 >> 172.21.15.27:0/1112474650 conn(0x7f2d100cec90 msgr2=0x7f2d100b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:51.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.764+0000 7f2d1708a640 1 -- 172.21.15.27:0/1112474650 shutdown_connections 2023-08-11T15:52:51.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:51.764+0000 7f2d1708a640 1 -- 172.21.15.27:0/1112474650 wait complete. 2023-08-11T15:52:51.789 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-08-11T15:52:52.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.036+0000 7fdd019c2640 1 Processor -- start 2023-08-11T15:52:52.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.036+0000 7fdd019c2640 1 -- start start 2023-08-11T15:52:52.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.038+0000 7fdd019c2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc160a20 0x7fdcfc0ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:52.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.038+0000 7fdd019c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdcfc0c15c0 0x7fdcfc0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:52.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.038+0000 7fdd019c2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc0cb1e0 con 0x7fdcfc0c1070 2023-08-11T15:52:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.038+0000 7fdd019c2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc160e00 con 0x7fdcfc0c15c0 2023-08-11T15:52:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.038+0000 7fdd019c2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc1652a0 con 0x7fdcfc160a20 2023-08-11T15:52:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.039+0000 7fdcfaffd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc160a20 0x7fdcfc0ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.039+0000 7fdcfa7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdcfc0c15c0 0x7fdcfc0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.039+0000 7fdcfaffd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc160a20 0x7fdcfc0ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:40488/0 (socket says 172.21.15.27:40488) 2023-08-11T15:52:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.039+0000 7fdcfaffd640 1 -- 172.21.15.27:0/4139462308 learned_addr learned my addr 172.21.15.27:0/4139462308 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.039+0000 7fdcfaffd640 1 -- 172.21.15.27:0/4139462308 >> v2:172.21.15.119:3300/0 conn(0x7fdcfc0c15c0 msgr2=0x7fdcfc0caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.039+0000 7fdcfaffd640 1 --2- 172.21.15.27:0/4139462308 >> v2:172.21.15.119:3300/0 conn(0x7fdcfc0c15c0 0x7fdcfc0caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:52.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.039+0000 7fdcfaffd640 1 -- 172.21.15.27:0/4139462308 >> v1:172.21.15.119:6789/0 conn(0x7fdcfc0c1070 legacy=0x7fdcfc06b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:52.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.039+0000 7fdcfaffd640 1 -- 172.21.15.27:0/4139462308 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc0cbe00 con 0x7fdcfc160a20 2023-08-11T15:52:52.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.041+0000 7fdcfaffd640 1 --2- 172.21.15.27:0/4139462308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc160a20 0x7fdcfc0ca760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009850 tx=0x7fdcec02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1726a15975bbc9c5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:52.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.041+0000 7fdcf9ffb640 1 -- 172.21.15.27:0/4139462308 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec02e070 con 0x7fdcfc160a20 2023-08-11T15:52:52.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.041+0000 7fdcf9ffb640 1 -- 172.21.15.27:0/4139462308 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec004030 con 0x7fdcfc160a20 2023-08-11T15:52:52.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.041+0000 7fdcf9ffb640 1 -- 172.21.15.27:0/4139462308 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec004420 con 0x7fdcfc160a20 2023-08-11T15:52:52.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.042+0000 7fdd019c2640 1 -- 172.21.15.27:0/4139462308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc160a20 msgr2=0x7fdcfc0ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:52.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.042+0000 7fdd019c2640 1 --2- 172.21.15.27:0/4139462308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc160a20 0x7fdcfc0ca760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009850 tx=0x7fdcec02b840 comp rx=0 tx=0).stop 2023-08-11T15:52:52.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.042+0000 7fdd019c2640 1 -- 172.21.15.27:0/4139462308 shutdown_connections 2023-08-11T15:52:52.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.042+0000 7fdd019c2640 1 --2- 172.21.15.27:0/4139462308 >> v2:172.21.15.119:3300/0 conn(0x7fdcfc0c15c0 0x7fdcfc0caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:52.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.042+0000 7fdd019c2640 1 --2- 172.21.15.27:0/4139462308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc160a20 0x7fdcfc0ca760 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:52.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.042+0000 7fdd019c2640 1 -- 172.21.15.27:0/4139462308 >> 172.21.15.27:0/4139462308 conn(0x7fdcfc0afb70 msgr2=0x7fdcfc0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:52.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.042+0000 7fdd019c2640 1 -- 172.21.15.27:0/4139462308 shutdown_connections 2023-08-11T15:52:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.042+0000 7fdd019c2640 1 -- 172.21.15.27:0/4139462308 wait complete. 2023-08-11T15:52:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.044+0000 7fdd019c2640 1 Processor -- start 2023-08-11T15:52:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.044+0000 7fdd019c2640 1 -- start start 2023-08-11T15:52:52.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.045+0000 7fdd019c2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcfc0c1070 0x7fdcfc0c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:52.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.045+0000 7fdd019c2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc0c15c0 0x7fdcfc0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.045+0000 7fdd019c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcfc160a20 0x7fdcfc156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.045+0000 7fdd019c2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0b0fb0 con 0x7fdcfc0c1070 2023-08-11T15:52:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.045+0000 7fdd019c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc157300 con 0x7fdcfc160a20 2023-08-11T15:52:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.045+0000 7fdd019c2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0b11e0 con 0x7fdcfc0c15c0 2023-08-11T15:52:52.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.045+0000 7fdcfaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcfc0c1070 0x7fdcfc0c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:52.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.045+0000 7fdcfaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcfc0c1070 0x7fdcfc0c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47516/0 (socket says 172.21.15.27:47516) 2023-08-11T15:52:52.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.045+0000 7fdcfaffd640 1 -- 172.21.15.27:0/3519462257 learned_addr learned my addr 172.21.15.27:0/3519462257 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:52.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdcfb7fe640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcfc160a20 0x7fdcfc156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:52.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdcfa7fc640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc0c15c0 0x7fdcfc0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:52.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdcfaffd640 1 -- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc0c15c0 msgr2=0x7fdcfc0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:52.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdcfaffd640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc0c15c0 0x7fdcfc0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:52.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdcfaffd640 1 -- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcfc160a20 msgr2=0x7fdcfc156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:52.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdcfaffd640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcfc160a20 0x7fdcfc156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:52.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdcfaffd640 1 -- 172.21.15.27:0/3519462257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec009460 con 0x7fdcfc0c1070 2023-08-11T15:52:52.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdcfb7fe640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcfc160a20 0x7fdcfc156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdcfa7fc640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc0c15c0 0x7fdcfc0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdcfaffd640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcfc0c1070 0x7fdcfc0c7c90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdcec02b880 tx=0x7fdcec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.046+0000 7fdce3fff640 1 -- 172.21.15.27:0/3519462257 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0039e0 con 0x7fdcfc0c1070 2023-08-11T15:52:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.047+0000 7fdce3fff640 1 -- 172.21.15.27:0/3519462257 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec003b40 con 0x7fdcfc0c1070 2023-08-11T15:52:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.047+0000 7fdce3fff640 1 -- 172.21.15.27:0/3519462257 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec006040 con 0x7fdcfc0c1070 2023-08-11T15:52:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.047+0000 7fdd019c2640 1 -- 172.21.15.27:0/3519462257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc0b14c0 con 0x7fdcfc0c1070 2023-08-11T15:52:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.047+0000 7fdd019c2640 1 -- 172.21.15.27:0/3519462257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc0b1a80 con 0x7fdcfc0c1070 2023-08-11T15:52:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.048+0000 7fdd019c2640 1 -- 172.21.15.27:0/3519462257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc04e420 con 0x7fdcfc0c1070 2023-08-11T15:52:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.049+0000 7fdce3fff640 1 -- 172.21.15.27:0/3519462257 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdcec039070 con 0x7fdcfc0c1070 2023-08-11T15:52:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.050+0000 7fdce3fff640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdce806a050 0x7fdce806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.050+0000 7fdce3fff640 1 -- 172.21.15.27:0/3519462257 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6047+0+0 (secure 0 0 0) 0x7fdcec0a9d70 con 0x7fdcfc0c1070 2023-08-11T15:52:52.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.051+0000 7fdcfa7fc640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdce806a050 0x7fdce806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:52.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.052+0000 7fdcfa7fc640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdce806a050 0x7fdce806c510 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc15ac30 tx=0x7fdcf0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.054+0000 7fdce3fff640 1 -- 172.21.15.27:0/3519462257 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fdcec0753d0 con 0x7fdcfc0c1070 2023-08-11T15:52:52.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.251+0000 7fdd019c2640 1 -- 172.21.15.27:0/3519462257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdcfc070790 con 0x7fdcfc0c1070 2023-08-11T15:52:52.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.772+0000 7fdce3fff640 1 -- 172.21.15.27:0/3519462257 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v34) v1 ==== 178+0+0 (secure 0 0 0) 0x7fdcec037020 con 0x7fdcfc0c1070 2023-08-11T15:52:52.776 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' removed 2023-08-11T15:52:52.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.781+0000 7fdd019c2640 1 -- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdce806a050 msgr2=0x7fdce806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:52.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.781+0000 7fdd019c2640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdce806a050 0x7fdce806c510 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc15ac30 tx=0x7fdcf0008040 comp rx=0 tx=0).stop 2023-08-11T15:52:52.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.781+0000 7fdd019c2640 1 -- 172.21.15.27:0/3519462257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcfc0c1070 msgr2=0x7fdcfc0c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:52.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.781+0000 7fdd019c2640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcfc0c1070 0x7fdcfc0c7c90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdcec02b880 tx=0x7fdcec0046c0 comp rx=0 tx=0).stop 2023-08-11T15:52:52.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.782+0000 7fdd019c2640 1 -- 172.21.15.27:0/3519462257 shutdown_connections 2023-08-11T15:52:52.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.782+0000 7fdd019c2640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fdce806a050 0x7fdce806c510 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:52.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.782+0000 7fdd019c2640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcfc0c15c0 0x7fdcfc0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:52.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.782+0000 7fdd019c2640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcfc160a20 0x7fdcfc156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:52.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.782+0000 7fdd019c2640 1 --2- 172.21.15.27:0/3519462257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcfc0c1070 0x7fdcfc0c7c90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:52.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.782+0000 7fdd019c2640 1 -- 172.21.15.27:0/3519462257 >> 172.21.15.27:0/3519462257 conn(0x7fdcfc0afb70 msgr2=0x7fdcfc06f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:52.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.784+0000 7fdd019c2640 1 -- 172.21.15.27:0/3519462257 shutdown_connections 2023-08-11T15:52:52.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:52.784+0000 7fdd019c2640 1 -- 172.21.15.27:0/3519462257 wait complete. 2023-08-11T15:52:52.814 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-11T15:52:52.814 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-11T15:52:52.815 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-11T15:52:52.815 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:52:52.815 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-11T15:52:52.815 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-11T15:52:52.816 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-11T15:52:52.816 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:52:52.816 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-11T15:52:52.816 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-11T15:52:52.816 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-11T15:52:52.817 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:52:52.817 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-11T15:52:52.817 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-11T15:52:52.817 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-11T15:52:52.817 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:52:52.835 INFO:tasks.ceph.mds.c:Stopped 2023-08-11T15:52:52.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-11T15:52:52.843 INFO:tasks.ceph.mds.d:Stopped 2023-08-11T15:52:52.843 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-11T15:52:52.846 INFO:tasks.ceph.mds.b:Stopped 2023-08-11T15:52:52.847 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-11T15:52:52.884 INFO:tasks.ceph.mds.c:Started 2023-08-11T15:52:52.885 DEBUG:teuthology.parallel:result is None 2023-08-11T15:52:52.894 INFO:tasks.ceph.mds.d:Started 2023-08-11T15:52:52.894 DEBUG:teuthology.parallel:result is None 2023-08-11T15:52:52.902 INFO:tasks.ceph.mds.b:Started 2023-08-11T15:52:52.902 DEBUG:teuthology.parallel:result is None 2023-08-11T15:52:53.061 INFO:tasks.ceph.mds.d.smithi119.stdout:starting mds.d at 2023-08-11T15:52:53.061 INFO:tasks.ceph.mds.b.smithi119.stdout:starting mds.b at 2023-08-11T15:52:53.086 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-08-11T15:52:56.783 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-08-11T15:52:58.835 INFO:tasks.ceph.mds.a:Stopped 2023-08-11T15:52:58.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-11T15:52:58.845 INFO:tasks.ceph.mds.a:Started 2023-08-11T15:52:58.846 DEBUG:teuthology.parallel:result is None 2023-08-11T15:52:58.847 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-11T15:52:59.030 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-08-11T15:52:59.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.059+0000 7f2f37f42640 1 Processor -- start 2023-08-11T15:52:59.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.059+0000 7f2f37f42640 1 -- start start 2023-08-11T15:52:59.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f37f42640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2f300cb290 0x7f2f300ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:59.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f37f42640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2f300cc5e0 0x7f2f3015bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:59.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f37f42640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f301600e0 con 0x7f2f300ad770 2023-08-11T15:52:59.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f37f42640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f30160330 con 0x7f2f300cb290 2023-08-11T15:52:59.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f37f42640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f30160610 con 0x7f2f300cc5e0 2023-08-11T15:52:59.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f354dd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2f300ad770 0x7f2f300cbed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42734/0 (socket says 172.21.15.27:42734) 2023-08-11T15:52:59.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f354dd640 1 -- 172.21.15.27:0/3816212556 learned_addr learned my addr 172.21.15.27:0/3816212556 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:52:59.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f35cde640 1 --2- 172.21.15.27:0/3816212556 >> v2:172.21.15.27:3300/0 conn(0x7f2f300cb290 0x7f2f300ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:59.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f364df640 1 --2- 172.21.15.27:0/3816212556 >> v2:172.21.15.119:3300/0 conn(0x7f2f300cc5e0 0x7f2f3015bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:59.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f35cde640 1 -- 172.21.15.27:0/3816212556 >> v2:172.21.15.119:3300/0 conn(0x7f2f300cc5e0 msgr2=0x7f2f3015bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:59.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f35cde640 1 --2- 172.21.15.27:0/3816212556 >> v2:172.21.15.119:3300/0 conn(0x7f2f300cc5e0 0x7f2f3015bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:59.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.061+0000 7f2f35cde640 1 -- 172.21.15.27:0/3816212556 >> v1:172.21.15.27:6789/0 conn(0x7f2f300ad770 legacy=0x7f2f300cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:59.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.062+0000 7f2f35cde640 1 -- 172.21.15.27:0/3816212556 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f301608f0 con 0x7f2f300cb290 2023-08-11T15:52:59.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.063+0000 7f2f35cde640 1 --2- 172.21.15.27:0/3816212556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 0x7f2f300ad230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2f3006c830 tx=0x7f2f2c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04df25d599440b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:59.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.063+0000 7f2f34cdc640 1 -- 172.21.15.27:0/3816212556 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c030610 con 0x7f2f300cb290 2023-08-11T15:52:59.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.063+0000 7f2f34cdc640 1 -- 172.21.15.27:0/3816212556 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c030770 con 0x7f2f300cb290 2023-08-11T15:52:59.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.063+0000 7f2f34cdc640 1 -- 172.21.15.27:0/3816212556 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c030b00 con 0x7f2f300cb290 2023-08-11T15:52:59.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.063+0000 7f2f37f42640 1 -- 172.21.15.27:0/3816212556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 msgr2=0x7f2f300ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.063+0000 7f2f37f42640 1 --2- 172.21.15.27:0/3816212556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 0x7f2f300ad230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2f3006c830 tx=0x7f2f2c02b960 comp rx=0 tx=0).stop 2023-08-11T15:52:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.064+0000 7f2f37f42640 1 -- 172.21.15.27:0/3816212556 shutdown_connections 2023-08-11T15:52:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.064+0000 7f2f37f42640 1 --2- 172.21.15.27:0/3816212556 >> v2:172.21.15.119:3300/0 conn(0x7f2f300cc5e0 0x7f2f3015bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.064+0000 7f2f37f42640 1 --2- 172.21.15.27:0/3816212556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 0x7f2f300ad230 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.064+0000 7f2f37f42640 1 -- 172.21.15.27:0/3816212556 >> 172.21.15.27:0/3816212556 conn(0x7f2f30069e90 msgr2=0x7f2f300afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.064+0000 7f2f37f42640 1 -- 172.21.15.27:0/3816212556 shutdown_connections 2023-08-11T15:52:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.064+0000 7f2f37f42640 1 -- 172.21.15.27:0/3816212556 wait complete. 2023-08-11T15:52:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.065+0000 7f2f37f42640 1 Processor -- start 2023-08-11T15:52:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.066+0000 7f2f37f42640 1 -- start start 2023-08-11T15:52:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f37f42640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f300ad770 0x7f2f3014f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f37f42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 0x7f2f300c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f37f42640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f300cc5e0 0x7f2f300c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f37f42640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3015b180 con 0x7f2f300cb290 2023-08-11T15:52:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f37f42640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f300b0cd0 con 0x7f2f300ad770 2023-08-11T15:52:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f37f42640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f300b0fb0 con 0x7f2f300cc5e0 2023-08-11T15:52:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f354dd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 0x7f2f300c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f354dd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 0x7f2f300c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47558/0 (socket says 172.21.15.27:47558) 2023-08-11T15:52:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f354dd640 1 -- 172.21.15.27:0/1876573050 learned_addr learned my addr 172.21.15.27:0/1876573050 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:52:59.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f35cde640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f300ad770 0x7f2f3014f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:59.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f364df640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f300cc5e0 0x7f2f300c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:59.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f354dd640 1 -- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f300cc5e0 msgr2=0x7f2f300c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:59.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f354dd640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f300cc5e0 0x7f2f300c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:59.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f354dd640 1 -- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f300ad770 msgr2=0x7f2f3014f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:59.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f354dd640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f300ad770 0x7f2f3014f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:59.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f354dd640 1 -- 172.21.15.27:0/1876573050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c009400 con 0x7f2f300cb290 2023-08-11T15:52:59.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.067+0000 7f2f35cde640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f300ad770 0x7f2f3014f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:59.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.068+0000 7f2f354dd640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 0x7f2f300c62d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c4a0 tx=0x7f2f2000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:59.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.068+0000 7f2f267fc640 1 -- 172.21.15.27:0/1876573050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20005ce0 con 0x7f2f300cb290 2023-08-11T15:52:59.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.068+0000 7f2f267fc640 1 -- 172.21.15.27:0/1876573050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f20005e40 con 0x7f2f300cb290 2023-08-11T15:52:59.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.068+0000 7f2f267fc640 1 -- 172.21.15.27:0/1876573050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20007270 con 0x7f2f300cb290 2023-08-11T15:52:59.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.068+0000 7f2f37f42640 1 -- 172.21.15.27:0/1876573050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f300b1290 con 0x7f2f300cb290 2023-08-11T15:52:59.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.068+0000 7f2f37f42640 1 -- 172.21.15.27:0/1876573050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3014f950 con 0x7f2f300cb290 2023-08-11T15:52:59.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.069+0000 7f2f37f42640 1 -- 172.21.15.27:0/1876573050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f300b17b0 con 0x7f2f300cb290 2023-08-11T15:52:59.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.070+0000 7f2f267fc640 1 -- 172.21.15.27:0/1876573050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2f200073d0 con 0x7f2f300cb290 2023-08-11T15:52:59.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.071+0000 7f2f267fc640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2f1406a080 0x7f2f1406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:59.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.072+0000 7f2f267fc640 1 -- 172.21.15.27:0/1876573050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2f20087840 con 0x7f2f300cb290 2023-08-11T15:52:59.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.072+0000 7f2f35cde640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2f1406a080 0x7f2f1406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:59.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.073+0000 7f2f35cde640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2f1406a080 0x7f2f1406c540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c036000 tx=0x7f2f2c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:59.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.076+0000 7f2f267fc640 1 -- 172.21.15.27:0/1876573050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2f300b17b0 con 0x7f2f300cb290 2023-08-11T15:52:59.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:52:59.282+0000 7f2f37f42640 1 -- 172.21.15.27:0/1876573050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2f300b17b0 con 0x7f2f300cb290 2023-08-11T15:53:00.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.653+0000 7f2f267fc640 1 -- 172.21.15.27:0/1876573050 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2f300b17b0 con 0x7f2f300cb290 2023-08-11T15:53:00.655 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-08-11T15:53:00.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.656+0000 7f2f37f42640 1 -- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2f1406a080 msgr2=0x7f2f1406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.656+0000 7f2f37f42640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2f1406a080 0x7f2f1406c540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c036000 tx=0x7f2f2c03b040 comp rx=0 tx=0).stop 2023-08-11T15:53:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.656+0000 7f2f37f42640 1 -- 172.21.15.27:0/1876573050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 msgr2=0x7f2f300c62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.656+0000 7f2f37f42640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 0x7f2f300c62d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c4a0 tx=0x7f2f2000c960 comp rx=0 tx=0).stop 2023-08-11T15:53:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.657+0000 7f2f37f42640 1 -- 172.21.15.27:0/1876573050 shutdown_connections 2023-08-11T15:53:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.657+0000 7f2f37f42640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2f1406a080 0x7f2f1406c540 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.657+0000 7f2f37f42640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f300cc5e0 0x7f2f300c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.657+0000 7f2f37f42640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f300cb290 0x7f2f300c62d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.657+0000 7f2f37f42640 1 --2- 172.21.15.27:0/1876573050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f300ad770 0x7f2f3014f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.657+0000 7f2f37f42640 1 -- 172.21.15.27:0/1876573050 >> 172.21.15.27:0/1876573050 conn(0x7f2f30069e90 msgr2=0x7f2f300adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.657+0000 7f2f37f42640 1 -- 172.21.15.27:0/1876573050 shutdown_connections 2023-08-11T15:53:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.657+0000 7f2f37f42640 1 -- 172.21.15.27:0/1876573050 wait complete. 2023-08-11T15:53:00.680 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-11T15:53:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.916+0000 7f043565e640 1 Processor -- start 2023-08-11T15:53:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.916+0000 7f043565e640 1 -- start start 2023-08-11T15:53:00.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.917+0000 7f043565e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f043015c620 0x7f04300b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.917+0000 7f043565e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f04300b82f0 0x7f04300cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.917+0000 7f043565e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04300b55c0 0x7f04300cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.917+0000 7f043565e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04300ccc10 con 0x7f04300b82f0 2023-08-11T15:53:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.917+0000 7f043565e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04300cda30 con 0x7f043015c620 2023-08-11T15:53:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.917+0000 7f043565e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0430161110 con 0x7f04300b55c0 2023-08-11T15:53:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.918+0000 7f042e7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f04300b82f0 0x7f04300cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.918+0000 7f042e7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f04300b82f0 0x7f04300cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43410/0 (socket says 172.21.15.27:43410) 2023-08-11T15:53:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.918+0000 7f042e7fc640 1 -- 172.21.15.27:0/974927220 learned_addr learned my addr 172.21.15.27:0/974927220 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:00.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.918+0000 7f042effd640 1 --2- 172.21.15.27:0/974927220 >> v2:172.21.15.119:3300/0 conn(0x7f043015c620 0x7f04300b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.918+0000 7f042f7fe640 1 --2- 172.21.15.27:0/974927220 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04300b55c0 0x7f04300cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.918+0000 7f042e7fc640 1 -- 172.21.15.27:0/974927220 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04300b55c0 msgr2=0x7f04300cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.918+0000 7f042e7fc640 1 --2- 172.21.15.27:0/974927220 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04300b55c0 0x7f04300cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.918+0000 7f042e7fc640 1 -- 172.21.15.27:0/974927220 >> v2:172.21.15.119:3300/0 conn(0x7f043015c620 msgr2=0x7f04300b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.918+0000 7f042e7fc640 1 --2- 172.21.15.27:0/974927220 >> v2:172.21.15.119:3300/0 conn(0x7f043015c620 0x7f04300b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.918+0000 7f042e7fc640 1 -- 172.21.15.27:0/974927220 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04300cce20 con 0x7f04300b82f0 2023-08-11T15:53:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.919+0000 7f042e7fc640 1 --2- 172.21.15.27:0/974927220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300b82f0 0x7f04300cf3f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f04300cefa0 tx=0x7f041802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2070f810af98cad8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.920+0000 7f042dffb640 1 -- 172.21.15.27:0/974927220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418031870 con 0x7f04300b82f0 2023-08-11T15:53:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.920+0000 7f042dffb640 1 -- 172.21.15.27:0/974927220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04180319d0 con 0x7f04300b82f0 2023-08-11T15:53:00.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.920+0000 7f042dffb640 1 -- 172.21.15.27:0/974927220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418031d60 con 0x7f04300b82f0 2023-08-11T15:53:00.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.920+0000 7f043565e640 1 -- 172.21.15.27:0/974927220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300b82f0 msgr2=0x7f04300cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.920+0000 7f043565e640 1 --2- 172.21.15.27:0/974927220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300b82f0 0x7f04300cf3f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f04300cefa0 tx=0x7f041802b910 comp rx=0 tx=0).stop 2023-08-11T15:53:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.920+0000 7f043565e640 1 -- 172.21.15.27:0/974927220 shutdown_connections 2023-08-11T15:53:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.920+0000 7f043565e640 1 --2- 172.21.15.27:0/974927220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300b82f0 0x7f04300cf3f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.920+0000 7f043565e640 1 --2- 172.21.15.27:0/974927220 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04300b55c0 0x7f04300cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.920+0000 7f043565e640 1 --2- 172.21.15.27:0/974927220 >> v2:172.21.15.119:3300/0 conn(0x7f043015c620 0x7f04300b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.920+0000 7f043565e640 1 -- 172.21.15.27:0/974927220 >> 172.21.15.27:0/974927220 conn(0x7f0430071110 msgr2=0x7f04300714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.921+0000 7f043565e640 1 -- 172.21.15.27:0/974927220 shutdown_connections 2023-08-11T15:53:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.921+0000 7f043565e640 1 -- 172.21.15.27:0/974927220 wait complete. 2023-08-11T15:53:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.922+0000 7f043565e640 1 Processor -- start 2023-08-11T15:53:00.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.922+0000 7f043565e640 1 -- start start 2023-08-11T15:53:00.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.923+0000 7f043565e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04300b55c0 0x7f04300cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.923+0000 7f043565e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04300b82f0 0x7f04300ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.923+0000 7f043565e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f043015c620 0x7f04300caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.923+0000 7f043565e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430157350 con 0x7f043015c620 2023-08-11T15:53:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.923+0000 7f043565e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043015f7e0 con 0x7f04300b82f0 2023-08-11T15:53:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f043565e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043015ec10 con 0x7f04300b55c0 2023-08-11T15:53:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f042f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f043015c620 0x7f04300caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f042f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f043015c620 0x7f04300caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43412/0 (socket says 172.21.15.27:43412) 2023-08-11T15:53:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f042f7fe640 1 -- 172.21.15.27:0/2035718885 learned_addr learned my addr 172.21.15.27:0/2035718885 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f042effd640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04300b55c0 0x7f04300cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f042e7fc640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04300b82f0 0x7f04300ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f042f7fe640 1 -- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04300b55c0 msgr2=0x7f04300cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f042f7fe640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04300b55c0 0x7f04300cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f042f7fe640 1 -- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04300b82f0 msgr2=0x7f04300ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f042f7fe640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04300b82f0 0x7f04300ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.924+0000 7f042f7fe640 1 -- 172.21.15.27:0/2035718885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04180094d0 con 0x7f043015c620 2023-08-11T15:53:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.925+0000 7f042f7fe640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f043015c620 0x7f04300caa80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f042800d620 tx=0x7f042800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.925+0000 7f04277fe640 1 -- 172.21.15.27:0/2035718885 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428009600 con 0x7f043015c620 2023-08-11T15:53:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.925+0000 7f04277fe640 1 -- 172.21.15.27:0/2035718885 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0428009780 con 0x7f043015c620 2023-08-11T15:53:00.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.925+0000 7f04277fe640 1 -- 172.21.15.27:0/2035718885 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428009b30 con 0x7f043015c620 2023-08-11T15:53:00.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.925+0000 7f043565e640 1 -- 172.21.15.27:0/2035718885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043015eeb0 con 0x7f043015c620 2023-08-11T15:53:00.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.925+0000 7f043565e640 1 -- 172.21.15.27:0/2035718885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043015f470 con 0x7f043015c620 2023-08-11T15:53:00.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.926+0000 7f043565e640 1 -- 172.21.15.27:0/2035718885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043015f970 con 0x7f043015c620 2023-08-11T15:53:00.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.927+0000 7f04277fe640 1 -- 172.21.15.27:0/2035718885 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0428009c90 con 0x7f043015c620 2023-08-11T15:53:00.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.928+0000 7f04277fe640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f041006a050 0x7f041006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.928+0000 7f04277fe640 1 -- 172.21.15.27:0/2035718885 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f0428088870 con 0x7f043015c620 2023-08-11T15:53:00.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.929+0000 7f042effd640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f041006a050 0x7f041006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.930+0000 7f042effd640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f041006a050 0x7f041006c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f04300b4d80 tx=0x7f04200039b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:00.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:00.933+0000 7f04277fe640 1 -- 172.21.15.27:0/2035718885 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0428054160 con 0x7f043015c620 2023-08-11T15:53:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.194+0000 7f043565e640 1 -- 172.21.15.27:0/2035718885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0430073990 con 0x7f043015c620 2023-08-11T15:53:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.195+0000 7f04277fe640 1 -- 172.21.15.27:0/2035718885 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f0428057780 con 0x7f043015c620 2023-08-11T15:53:01.197 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:01.197 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:01.197 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-08-11T15:53:01.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.198 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-08-11T15:53:01.198 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCfWNZkJr0nCRAAt5jrDGXTdREF1aX+JJ6y1g==", 2023-08-11T15:53:01.198 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.198 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-08-11T15:53:01.198 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-08-11T15:53:01.198 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-08-11T15:53:01.198 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.198 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.199 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-08-11T15:53:01.199 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCfWNZklkmgLhAA5Of9CMgj/iq1k0xVRkHrrg==", 2023-08-11T15:53:01.199 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.199 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-08-11T15:53:01.199 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-08-11T15:53:01.200 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-08-11T15:53:01.200 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.200 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.200 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-08-11T15:53:01.200 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCfWNZk5UU/HBAAmBzoJNZcHdNXx1RCke7mkQ==", 2023-08-11T15:53:01.201 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.201 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-08-11T15:53:01.201 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-08-11T15:53:01.201 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-08-11T15:53:01.201 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.202 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-08-11T15:53:01.202 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCgWNZkYGufAxAALxZd1NcHLT2Y46Nfb+5wMA==", 2023-08-11T15:53:01.202 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.202 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-08-11T15:53:01.202 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-08-11T15:53:01.202 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-08-11T15:53:01.202 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.203 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.203 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-08-11T15:53:01.203 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQClWNZkD1sWERAAXKnWuWlUQdfesKGslW1yzA==", 2023-08-11T15:53:01.203 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.203 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-08-11T15:53:01.203 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-08-11T15:53:01.204 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.204 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-08-11T15:53:01.205 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCoWNZkCnrcLRAABS2uWXujbf29rznDr30fuA==", 2023-08-11T15:53:01.205 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.205 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-08-11T15:53:01.205 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-08-11T15:53:01.205 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.205 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.206 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-08-11T15:53:01.206 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCsWNZketPyDRAAY7pNdGg8whP6s3Wkjl+BaQ==", 2023-08-11T15:53:01.206 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.206 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-08-11T15:53:01.206 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-08-11T15:53:01.206 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.207 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-08-11T15:53:01.207 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCvWNZkvkVeGxAAdnmiTpGGjOzZ2X/aTtZR6A==", 2023-08-11T15:53:01.207 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.207 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-08-11T15:53:01.207 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-08-11T15:53:01.207 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.207 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.208 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-08-11T15:53:01.208 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQC2WNZkMdJ+KxAAmyGKek7HeVzxQbmtt+V/+Q==", 2023-08-11T15:53:01.208 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.208 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-08-11T15:53:01.208 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-08-11T15:53:01.209 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.209 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-08-11T15:53:01.209 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQC5WNZkSlW9NRAAp+mD8cyG363Kula78uB33A==", 2023-08-11T15:53:01.209 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.209 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-08-11T15:53:01.210 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-08-11T15:53:01.210 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.210 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.210 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-08-11T15:53:01.210 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQC9WNZkwxCbExAAelXzMnXkgUUHE2kgRKoVwA==", 2023-08-11T15:53:01.211 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.211 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-08-11T15:53:01.211 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-08-11T15:53:01.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.211 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-08-11T15:53:01.212 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDAWNZkZMPxHRAAEM/ndOZQ1wWBI3yhmVhqKA==", 2023-08-11T15:53:01.212 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.212 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-08-11T15:53:01.212 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-08-11T15:53:01.212 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.212 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.213 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-08-11T15:53:01.213 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCgWNZkD5/iEhAAZ4I6N46HjdmeGnn6p1vIvg==", 2023-08-11T15:53:01.213 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.213 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-08-11T15:53:01.213 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-08-11T15:53:01.213 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-08-11T15:53:01.213 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-08-11T15:53:01.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.214 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-08-11T15:53:01.214 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCgWNZk7aNKLBAAewcNiTQiIudZtZI3a8Lzng==", 2023-08-11T15:53:01.214 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.214 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-08-11T15:53:01.214 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-08-11T15:53:01.214 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-08-11T15:53:01.215 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-08-11T15:53:01.215 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.215 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-08-11T15:53:01.215 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCgWNZkaw5GIBAA2hBr1tFhgg8c8XwHQkYngg==", 2023-08-11T15:53:01.215 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.216 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-08-11T15:53:01.216 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-08-11T15:53:01.216 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-08-11T15:53:01.216 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-08-11T15:53:01.216 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.216 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-08-11T15:53:01.217 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCgWNZk7Rp+NxAAA2iTgfWpzexHHUd2PpVsig==", 2023-08-11T15:53:01.217 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.217 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-08-11T15:53:01.217 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-08-11T15:53:01.217 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-08-11T15:53:01.217 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-08-11T15:53:01.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.218 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-08-11T15:53:01.218 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCdWNZk8lfsLxAAMQtgW53cT3HJkqgqkuRKbg==", 2023-08-11T15:53:01.218 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.218 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-08-11T15:53:01.218 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-08-11T15:53:01.218 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-08-11T15:53:01.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.219 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.219 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-mds", 2023-08-11T15:53:01.219 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDJWNZkjIBSJRAA5cWOqGFrCAuumpo/FCPm6Q==", 2023-08-11T15:53:01.219 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.219 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-mds" 2023-08-11T15:53:01.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.220 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-mgr", 2023-08-11T15:53:01.220 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDJWNZkHZpSJRAAnEKI94YRrUwle3aMqC0olg==", 2023-08-11T15:53:01.220 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.220 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-mgr" 2023-08-11T15:53:01.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.221 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-osd", 2023-08-11T15:53:01.221 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDJWNZkB7BSJRAARzFGgorSvHe7PtONIKBGsA==", 2023-08-11T15:53:01.221 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.221 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-osd" 2023-08-11T15:53:01.221 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.222 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rbd", 2023-08-11T15:53:01.222 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDJWNZkAsNSJRAA4zEC5fa1yG6NXxJaNsCr8Q==", 2023-08-11T15:53:01.222 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.222 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rbd" 2023-08-11T15:53:01.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.223 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-08-11T15:53:01.223 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDJWNZkDNNSJRAAPueGYzH26zYtvesfJ1scag==", 2023-08-11T15:53:01.223 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.223 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-08-11T15:53:01.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.224 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rgw", 2023-08-11T15:53:01.224 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDJWNZkjt5SJRAA6hh1TOxGwuy9I8elZBbKcg==", 2023-08-11T15:53:01.224 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.224 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rgw" 2023-08-11T15:53:01.224 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.225 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-08-11T15:53:01.225 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCeWNZkEM6VNhAAJJYDJ0TrXp2W0ISyeGJ90w==", 2023-08-11T15:53:01.225 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.225 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-08-11T15:53:01.225 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-08-11T15:53:01.225 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.225 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:01.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:01.226 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-08-11T15:53:01.226 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCeWNZkx1hfKRAAJyIEwNyF3oZjsXNZJiVABQ==", 2023-08-11T15:53:01.226 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-08-11T15:53:01.226 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-08-11T15:53:01.226 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-08-11T15:53:01.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-08-11T15:53:01.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:01.227 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-08-11T15:53:01.227 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 -- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f041006a050 msgr2=0x7f041006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f041006a050 0x7f041006c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f04300b4d80 tx=0x7f04200039b0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 -- 172.21.15.27:0/2035718885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f043015c620 msgr2=0x7f04300caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f043015c620 0x7f04300caa80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f042800d620 tx=0x7f042800dae0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 -- 172.21.15.27:0/2035718885 shutdown_connections 2023-08-11T15:53:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f041006a050 0x7f041006c510 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f043015c620 0x7f04300caa80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04300b82f0 0x7f04300ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 --2- 172.21.15.27:0/2035718885 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04300b55c0 0x7f04300cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 -- 172.21.15.27:0/2035718885 >> 172.21.15.27:0/2035718885 conn(0x7f0430071110 msgr2=0x7f04300b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 -- 172.21.15.27:0/2035718885 shutdown_connections 2023-08-11T15:53:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.198+0000 7f043565e640 1 -- 172.21.15.27:0/2035718885 wait complete. 2023-08-11T15:53:01.230 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-08-11T15:53:01.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.469+0000 7ff41da98640 1 Processor -- start 2023-08-11T15:53:01.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.470+0000 7ff41da98640 1 -- start start 2023-08-11T15:53:01.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.471+0000 7ff41da98640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff41815c1f0 0x7ff4180b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:01.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.471+0000 7ff41da98640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4180b5170 con 0x7ff41815c5d0 2023-08-11T15:53:01.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.471+0000 7ff41da98640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41815d080 con 0x7ff41815ca30 2023-08-11T15:53:01.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.471+0000 7ff41da98640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41815ce70 con 0x7ff41815c1f0 2023-08-11T15:53:01.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.472+0000 7ff41ca96640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff41815c1f0 0x7ff4180b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:01.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.472+0000 7ff417fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff41815c5d0 0x7ff4180d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34242/0 (socket says 172.21.15.27:34242) 2023-08-11T15:53:01.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.472+0000 7ff417fff640 1 -- 172.21.15.27:0/1315735908 learned_addr learned my addr 172.21.15.27:0/1315735908 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:01.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.472+0000 7ff41ca96640 1 -- 172.21.15.27:0/1315735908 >> v1:172.21.15.119:6789/0 conn(0x7ff41815ca30 legacy=0x7ff418160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:01.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.472+0000 7ff41ca96640 1 -- 172.21.15.27:0/1315735908 >> v1:172.21.15.27:6789/0 conn(0x7ff41815c5d0 legacy=0x7ff4180d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.472+0000 7ff41ca96640 1 -- 172.21.15.27:0/1315735908 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41815d360 con 0x7ff41815c1f0 2023-08-11T15:53:01.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.473+0000 7ff41ca96640 1 --2- 172.21.15.27:0/1315735908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 0x7ff4180b7d40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff408009d90 tx=0x7ff40802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de53e9d734f1258 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:01.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.474+0000 7ff4177fe640 1 -- 172.21.15.27:0/1315735908 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40802ccf0 con 0x7ff41815c1f0 2023-08-11T15:53:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.474+0000 7ff4177fe640 1 -- 172.21.15.27:0/1315735908 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40802ce50 con 0x7ff41815c1f0 2023-08-11T15:53:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.474+0000 7ff4177fe640 1 -- 172.21.15.27:0/1315735908 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408034660 con 0x7ff41815c1f0 2023-08-11T15:53:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.474+0000 7ff41da98640 1 -- 172.21.15.27:0/1315735908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 msgr2=0x7ff4180b7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.474+0000 7ff41da98640 1 --2- 172.21.15.27:0/1315735908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 0x7ff4180b7d40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff408009d90 tx=0x7ff40802bf40 comp rx=0 tx=0).stop 2023-08-11T15:53:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.475+0000 7ff41da98640 1 -- 172.21.15.27:0/1315735908 shutdown_connections 2023-08-11T15:53:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.475+0000 7ff41da98640 1 --2- 172.21.15.27:0/1315735908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 0x7ff4180b7d40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.475+0000 7ff41da98640 1 -- 172.21.15.27:0/1315735908 >> 172.21.15.27:0/1315735908 conn(0x7ff41806a250 msgr2=0x7ff41806a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.475+0000 7ff41da98640 1 -- 172.21.15.27:0/1315735908 shutdown_connections 2023-08-11T15:53:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.475+0000 7ff41da98640 1 -- 172.21.15.27:0/1315735908 wait complete. 2023-08-11T15:53:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.477+0000 7ff41da98640 1 Processor -- start 2023-08-11T15:53:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.477+0000 7ff41da98640 1 -- start start 2023-08-11T15:53:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.478+0000 7ff41da98640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 0x7ff4181526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.478+0000 7ff41da98640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff41815c5d0 0x7ff4181f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.478+0000 7ff41da98640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff41815ca30 0x7ff4181551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.478+0000 7ff41da98640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180b5e80 con 0x7ff41815c1f0 2023-08-11T15:53:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.478+0000 7ff41da98640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180b6720 con 0x7ff41815ca30 2023-08-11T15:53:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.478+0000 7ff41da98640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff418152be0 con 0x7ff41815c5d0 2023-08-11T15:53:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.478+0000 7ff41ca96640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 0x7ff4181526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.478+0000 7ff41ca96640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 0x7ff4181526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43424/0 (socket says 172.21.15.27:43424) 2023-08-11T15:53:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.478+0000 7ff41ca96640 1 -- 172.21.15.27:0/2619711904 learned_addr learned my addr 172.21.15.27:0/2619711904 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.479+0000 7ff41d297640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff41815ca30 0x7ff4181551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.479+0000 7ff417fff640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff41815c5d0 0x7ff4181f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.479+0000 7ff41ca96640 1 -- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff41815c5d0 msgr2=0x7ff4181f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.479+0000 7ff41ca96640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff41815c5d0 0x7ff4181f2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.479+0000 7ff41ca96640 1 -- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff41815ca30 msgr2=0x7ff4181551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.479+0000 7ff41ca96640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff41815ca30 0x7ff4181551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.479+0000 7ff41ca96640 1 -- 172.21.15.27:0/2619711904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408009a70 con 0x7ff41815c1f0 2023-08-11T15:53:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.479+0000 7ff41d297640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff41815ca30 0x7ff4181551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.479+0000 7ff417fff640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff41815c5d0 0x7ff4181f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.479+0000 7ff41ca96640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 0x7ff4181526a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff408009d90 tx=0x7ff408004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.480+0000 7ff4157fa640 1 -- 172.21.15.27:0/2619711904 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40803d600 con 0x7ff41815c1f0 2023-08-11T15:53:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.480+0000 7ff4157fa640 1 -- 172.21.15.27:0/2619711904 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40803d780 con 0x7ff41815c1f0 2023-08-11T15:53:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.480+0000 7ff4157fa640 1 -- 172.21.15.27:0/2619711904 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40803db30 con 0x7ff41815c1f0 2023-08-11T15:53:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.480+0000 7ff41da98640 1 -- 172.21.15.27:0/2619711904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4181f2e60 con 0x7ff41815c1f0 2023-08-11T15:53:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.480+0000 7ff41da98640 1 -- 172.21.15.27:0/2619711904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4180b60d0 con 0x7ff41815c1f0 2023-08-11T15:53:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.481+0000 7ff41da98640 1 -- 172.21.15.27:0/2619711904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4180671d0 con 0x7ff41815c1f0 2023-08-11T15:53:01.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.482+0000 7ff4157fa640 1 -- 172.21.15.27:0/2619711904 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff40803b040 con 0x7ff41815c1f0 2023-08-11T15:53:01.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.483+0000 7ff4157fa640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff40406a050 0x7ff40406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:01.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.483+0000 7ff4157fa640 1 -- 172.21.15.27:0/2619711904 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ff4080aa3b0 con 0x7ff41815c1f0 2023-08-11T15:53:01.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.484+0000 7ff417fff640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff40406a050 0x7ff40406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.485+0000 7ff417fff640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff40406a050 0x7ff40406c510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff4180d00e0 tx=0x7ff40c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.488+0000 7ff4157fa640 1 -- 172.21.15.27:0/2619711904 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ff408075ca0 con 0x7ff41815c1f0 2023-08-11T15:53:01.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.751+0000 7ff41da98640 1 -- 172.21.15.27:0/2619711904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7ff41804e420 con 0x7ff41815c1f0 2023-08-11T15:53:01.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.759+0000 7ff4157fa640 1 -- 172.21.15.27:0/2619711904 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff4080792c0 con 0x7ff41815c1f0 2023-08-11T15:53:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.761+0000 7ff41da98640 1 -- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff40406a050 msgr2=0x7ff40406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.761+0000 7ff41da98640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff40406a050 0x7ff40406c510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff4180d00e0 tx=0x7ff40c009040 comp rx=0 tx=0).stop 2023-08-11T15:53:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.761+0000 7ff41da98640 1 -- 172.21.15.27:0/2619711904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 msgr2=0x7ff4181526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.761+0000 7ff41da98640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 0x7ff4181526a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff408009d90 tx=0x7ff408004a40 comp rx=0 tx=0).stop 2023-08-11T15:53:01.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.762+0000 7ff41da98640 1 -- 172.21.15.27:0/2619711904 shutdown_connections 2023-08-11T15:53:01.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.762+0000 7ff41da98640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff40406a050 0x7ff40406c510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.762+0000 7ff41da98640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff41815c5d0 0x7ff4181f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.762+0000 7ff41da98640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff41815ca30 0x7ff4181551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.762+0000 7ff41da98640 1 --2- 172.21.15.27:0/2619711904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff41815c1f0 0x7ff4181526a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.762+0000 7ff41da98640 1 -- 172.21.15.27:0/2619711904 >> 172.21.15.27:0/2619711904 conn(0x7ff41806a250 msgr2=0x7ff4181568d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:01.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.762+0000 7ff41da98640 1 -- 172.21.15.27:0/2619711904 shutdown_connections 2023-08-11T15:53:01.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:01.762+0000 7ff41da98640 1 -- 172.21.15.27:0/2619711904 wait complete. 2023-08-11T15:53:01.782 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-08-11T15:53:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.018+0000 7f4246fd2640 1 Processor -- start 2023-08-11T15:53:02.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.018+0000 7f4246fd2640 1 -- start start 2023-08-11T15:53:02.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.020+0000 7f4246fd2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f424015a020 0x7f424015e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.020+0000 7f4246fd2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42400c85f0 con 0x7f424006f5d0 2023-08-11T15:53:02.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.020+0000 7f4246fd2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42400c8840 con 0x7f42400c9600 2023-08-11T15:53:02.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.020+0000 7f4246fd2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424015eae0 con 0x7f424015a020 2023-08-11T15:53:02.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.020+0000 7f42467d1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f424006f5d0 0x7f42400ca120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34248/0 (socket says 172.21.15.27:34248) 2023-08-11T15:53:02.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.020+0000 7f42467d1640 1 -- 172.21.15.27:0/1349627345 learned_addr learned my addr 172.21.15.27:0/1349627345 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:02.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4245fd0640 1 --2- 172.21.15.27:0/1349627345 >> v2:172.21.15.119:3300/0 conn(0x7f424015a020 0x7f424015e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580951760 0 0) 0x7f42400c85f0 con 0x7f424006f5d0 2023-08-11T15:53:02.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f422c003520 con 0x7f424006f5d0 2023-08-11T15:53:02.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1513928308 0 0) 0x7f422c003520 con 0x7f424006f5d0 2023-08-11T15:53:02.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 >> v2:172.21.15.119:3300/0 conn(0x7f424015a020 msgr2=0x7f424015e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:02.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4244fce640 1 --2- 172.21.15.27:0/1349627345 >> v2:172.21.15.119:3300/0 conn(0x7f424015a020 0x7f424015e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 >> v1:172.21.15.119:6789/0 conn(0x7f42400c9600 legacy=0x7f4240071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400c8b20 con 0x7f424006f5d0 2023-08-11T15:53:02.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f42380019f0 con 0x7f424006f5d0 2023-08-11T15:53:02.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 >> v1:172.21.15.27:6789/0 conn(0x7f424006f5d0 legacy=0x7f42400ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.021+0000 7f4244fce640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f422c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.022+0000 7f4244fce640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c0047c0 0x7f422c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.022+0000 7f4244fce640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f422c005180 0x7f422c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.022+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c00c0f0 con 0x7f422c003970 2023-08-11T15:53:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.022+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c00c260 con 0x7f422c0047c0 2023-08-11T15:53:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.022+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c00c420 con 0x7f422c005180 2023-08-11T15:53:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.022+0000 7f4245fd0640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f422c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.022+0000 7f42467d1640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f422c005180 0x7f422c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.022+0000 7f42457cf640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c0047c0 0x7f422c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.023+0000 7f4245fd0640 1 -- 172.21.15.27:0/1349627345 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f422c005180 msgr2=0x7f422c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:02.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.023+0000 7f4245fd0640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f422c005180 0x7f422c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.023+0000 7f4245fd0640 1 -- 172.21.15.27:0/1349627345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c0047c0 msgr2=0x7f422c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.023+0000 7f4245fd0640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c0047c0 0x7f422c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.023+0000 7f42467d1640 1 -- 172.21.15.27:0/1349627345 reap_dead start 2023-08-11T15:53:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.023+0000 7f4245fd0640 1 -- 172.21.15.27:0/1349627345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c003520 con 0x7f422c003970 2023-08-11T15:53:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.024+0000 7f4245fd0640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f422c003d90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f423c009ac0 tx=0x7f422c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0e8f290f79c878 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:02.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.024+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c02c6b0 con 0x7f422c003970 2023-08-11T15:53:02.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.024+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c02c810 con 0x7f422c003970 2023-08-11T15:53:02.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.024+0000 7f4244fce640 1 -- 172.21.15.27:0/1349627345 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c02cba0 con 0x7f422c003970 2023-08-11T15:53:02.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.025+0000 7f4246fd2640 1 -- 172.21.15.27:0/1349627345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 msgr2=0x7f422c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.025+0000 7f4246fd2640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f422c003d90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f423c009ac0 tx=0x7f422c0057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.025+0000 7f4246fd2640 1 -- 172.21.15.27:0/1349627345 shutdown_connections 2023-08-11T15:53:02.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.025+0000 7f4246fd2640 1 --2- 172.21.15.27:0/1349627345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f422c003d90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.025+0000 7f4246fd2640 1 -- 172.21.15.27:0/1349627345 >> 172.21.15.27:0/1349627345 conn(0x7f42400afb70 msgr2=0x7f42400aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:02.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.025+0000 7f4246fd2640 1 -- 172.21.15.27:0/1349627345 shutdown_connections 2023-08-11T15:53:02.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.025+0000 7f4246fd2640 1 -- 172.21.15.27:0/1349627345 wait complete. 2023-08-11T15:53:02.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.027+0000 7f4246fd2640 1 Processor -- start 2023-08-11T15:53:02.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.027+0000 7f4246fd2640 1 -- start start 2023-08-11T15:53:02.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.028+0000 7f4246fd2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f4240156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.028+0000 7f4246fd2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42400ab8e0 0x7f42400abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.028+0000 7f4246fd2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42400c7240 0x7f42400c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.028+0000 7f4246fd2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42400ac2a0 con 0x7f422c003970 2023-08-11T15:53:02.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.028+0000 7f4246fd2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42400ad210 con 0x7f42400ab8e0 2023-08-11T15:53:02.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.028+0000 7f4246fd2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42400cbb70 con 0x7f42400c7240 2023-08-11T15:53:02.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.028+0000 7f4245fd0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f4240156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.028+0000 7f4245fd0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f4240156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43444/0 (socket says 172.21.15.27:43444) 2023-08-11T15:53:02.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.028+0000 7f4245fd0640 1 -- 172.21.15.27:0/2231413920 learned_addr learned my addr 172.21.15.27:0/2231413920 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:02.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.029+0000 7f4245fd0640 1 -- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42400c7240 msgr2=0x7f42400c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.029+0000 7f42467d1640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42400c7240 0x7f42400c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.029+0000 7f42457cf640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42400ab8e0 0x7f42400abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.029+0000 7f4245fd0640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42400c7240 0x7f42400c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.029+0000 7f4245fd0640 1 -- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42400ab8e0 msgr2=0x7f42400abd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.029+0000 7f4245fd0640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42400ab8e0 0x7f42400abd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.029+0000 7f4245fd0640 1 -- 172.21.15.27:0/2231413920 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400c7c10 con 0x7f422c003970 2023-08-11T15:53:02.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.029+0000 7f42467d1640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42400c7240 0x7f42400c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:02.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.029+0000 7f4245fd0640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f4240156da0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f423c009ac0 tx=0x7f423c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:02.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.030+0000 7f42367fc640 1 -- 172.21.15.27:0/2231413920 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c003810 con 0x7f422c003970 2023-08-11T15:53:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.030+0000 7f42367fc640 1 -- 172.21.15.27:0/2231413920 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c003970 con 0x7f422c003970 2023-08-11T15:53:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.030+0000 7f42367fc640 1 -- 172.21.15.27:0/2231413920 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c003d00 con 0x7f422c003970 2023-08-11T15:53:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.030+0000 7f4246fd2640 1 -- 172.21.15.27:0/2231413920 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42400ac4c0 con 0x7f422c003970 2023-08-11T15:53:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.030+0000 7f4246fd2640 1 -- 172.21.15.27:0/2231413920 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424015ba70 con 0x7f422c003970 2023-08-11T15:53:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.031+0000 7f4246fd2640 1 -- 172.21.15.27:0/2231413920 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424004e420 con 0x7f422c003970 2023-08-11T15:53:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.032+0000 7f42367fc640 1 -- 172.21.15.27:0/2231413920 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f423c040450 con 0x7f422c003970 2023-08-11T15:53:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.033+0000 7f42367fc640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f422406a050 0x7f422406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.033+0000 7f42367fc640 1 -- 172.21.15.27:0/2231413920 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f423c0aa300 con 0x7f422c003970 2023-08-11T15:53:02.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.034+0000 7f42457cf640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f422406a050 0x7f422406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.035+0000 7f42457cf640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f422406a050 0x7f422406c510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4230007fd0 tx=0x7f4230009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:02.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.037+0000 7f42367fc640 1 -- 172.21.15.27:0/2231413920 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f423c075b60 con 0x7f422c003970 2023-08-11T15:53:02.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.320+0000 7f4246fd2640 1 -- 172.21.15.27:0/2231413920 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f42400cb720 con 0x7f422c003970 2023-08-11T15:53:02.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.326+0000 7f42367fc640 1 -- 172.21.15.27:0/2231413920 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f423c003e60 con 0x7f422c003970 2023-08-11T15:53:02.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.328+0000 7f4246fd2640 1 -- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f422406a050 msgr2=0x7f422406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.328+0000 7f4246fd2640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f422406a050 0x7f422406c510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4230007fd0 tx=0x7f4230009040 comp rx=0 tx=0).stop 2023-08-11T15:53:02.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.328+0000 7f4246fd2640 1 -- 172.21.15.27:0/2231413920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 msgr2=0x7f4240156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.328+0000 7f4246fd2640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f4240156da0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f423c009ac0 tx=0x7f423c004730 comp rx=0 tx=0).stop 2023-08-11T15:53:02.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.329+0000 7f4246fd2640 1 -- 172.21.15.27:0/2231413920 shutdown_connections 2023-08-11T15:53:02.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.329+0000 7f4246fd2640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f422406a050 0x7f422406c510 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.329+0000 7f4246fd2640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42400c7240 0x7f42400c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.329+0000 7f4246fd2640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42400ab8e0 0x7f42400abd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.329+0000 7f4246fd2640 1 --2- 172.21.15.27:0/2231413920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422c003970 0x7f4240156da0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.329+0000 7f4246fd2640 1 -- 172.21.15.27:0/2231413920 >> 172.21.15.27:0/2231413920 conn(0x7f42400afb70 msgr2=0x7f424006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:02.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.329+0000 7f4246fd2640 1 -- 172.21.15.27:0/2231413920 shutdown_connections 2023-08-11T15:53:02.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.329+0000 7f4246fd2640 1 -- 172.21.15.27:0/2231413920 wait complete. 2023-08-11T15:53:02.351 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-08-11T15:53:02.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.581+0000 7f92af1b5640 1 Processor -- start 2023-08-11T15:53:02.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.582+0000 7f92af1b5640 1 -- start start 2023-08-11T15:53:02.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.583+0000 7f92af1b5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f92a815c620 0x7f92a80b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.583+0000 7f92af1b5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92a80b82f0 0x7f92a80cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.583+0000 7f92af1b5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a80b55c0 0x7f92a80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.583+0000 7f92af1b5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a80ccc10 con 0x7f92a815c620 2023-08-11T15:53:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.583+0000 7f92af1b5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a80cda30 con 0x7f92a80b82f0 2023-08-11T15:53:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.583+0000 7f92af1b5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a8161110 con 0x7f92a80b55c0 2023-08-11T15:53:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.584+0000 7f92acf51640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f92a815c620 0x7f92a80b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.584+0000 7f92acf51640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f92a815c620 0x7f92a80b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43458/0 (socket says 172.21.15.27:43458) 2023-08-11T15:53:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.584+0000 7f92acf51640 1 -- 172.21.15.27:0/4094589927 learned_addr learned my addr 172.21.15.27:0/4094589927 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:02.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.584+0000 7f92a7fff640 1 --2- 172.21.15.27:0/4094589927 >> v2:172.21.15.119:3300/0 conn(0x7f92a80b82f0 0x7f92a80cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.584+0000 7f92ad752640 1 --2- 172.21.15.27:0/4094589927 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a80b55c0 0x7f92a80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.584+0000 7f92acf51640 1 -- 172.21.15.27:0/4094589927 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a80b55c0 msgr2=0x7f92a80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.584+0000 7f92acf51640 1 --2- 172.21.15.27:0/4094589927 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a80b55c0 0x7f92a80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.584+0000 7f92acf51640 1 -- 172.21.15.27:0/4094589927 >> v2:172.21.15.119:3300/0 conn(0x7f92a80b82f0 msgr2=0x7f92a80cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.584+0000 7f92acf51640 1 --2- 172.21.15.27:0/4094589927 >> v2:172.21.15.119:3300/0 conn(0x7f92a80b82f0 0x7f92a80cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.584+0000 7f92acf51640 1 -- 172.21.15.27:0/4094589927 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a80cce20 con 0x7f92a815c620 2023-08-11T15:53:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.586+0000 7f92acf51640 1 --2- 172.21.15.27:0/4094589927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a815c620 0x7f92a80b7db0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f92a80b50e0 tx=0x7f929c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fa11c42963d687 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.586+0000 7f92a77fe640 1 -- 172.21.15.27:0/4094589927 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c031870 con 0x7f92a815c620 2023-08-11T15:53:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.586+0000 7f92a77fe640 1 -- 172.21.15.27:0/4094589927 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c0319d0 con 0x7f92a815c620 2023-08-11T15:53:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.586+0000 7f92a77fe640 1 -- 172.21.15.27:0/4094589927 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c031d60 con 0x7f92a815c620 2023-08-11T15:53:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.587+0000 7f92af1b5640 1 -- 172.21.15.27:0/4094589927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a815c620 msgr2=0x7f92a80b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.587+0000 7f92af1b5640 1 --2- 172.21.15.27:0/4094589927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a815c620 0x7f92a80b7db0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f92a80b50e0 tx=0x7f929c02b910 comp rx=0 tx=0).stop 2023-08-11T15:53:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.587+0000 7f92af1b5640 1 -- 172.21.15.27:0/4094589927 shutdown_connections 2023-08-11T15:53:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.587+0000 7f92af1b5640 1 --2- 172.21.15.27:0/4094589927 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a80b55c0 0x7f92a80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.587+0000 7f92af1b5640 1 --2- 172.21.15.27:0/4094589927 >> v2:172.21.15.119:3300/0 conn(0x7f92a80b82f0 0x7f92a80cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.587+0000 7f92af1b5640 1 --2- 172.21.15.27:0/4094589927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a815c620 0x7f92a80b7db0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.587+0000 7f92af1b5640 1 -- 172.21.15.27:0/4094589927 >> 172.21.15.27:0/4094589927 conn(0x7f92a8071110 msgr2=0x7f92a80714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.587+0000 7f92af1b5640 1 -- 172.21.15.27:0/4094589927 shutdown_connections 2023-08-11T15:53:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.587+0000 7f92af1b5640 1 -- 172.21.15.27:0/4094589927 wait complete. 2023-08-11T15:53:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.589+0000 7f92af1b5640 1 Processor -- start 2023-08-11T15:53:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.589+0000 7f92af1b5640 1 -- start start 2023-08-11T15:53:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.590+0000 7f92af1b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a80b55c0 0x7f92a8156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.590+0000 7f92af1b5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a80b82f0 0x7f92a815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.590+0000 7f92af1b5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a815c620 0x7f92a80ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.590+0000 7f92af1b5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80b3090 con 0x7f92a80b82f0 2023-08-11T15:53:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.590+0000 7f92af1b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80b3350 con 0x7f92a80b55c0 2023-08-11T15:53:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.590+0000 7f92af1b5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80b3630 con 0x7f92a815c620 2023-08-11T15:53:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.590+0000 7f92a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a80b82f0 0x7f92a815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.590+0000 7f92a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a80b82f0 0x7f92a815d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43466/0 (socket says 172.21.15.27:43466) 2023-08-11T15:53:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.590+0000 7f92a7fff640 1 -- 172.21.15.27:0/3823716228 learned_addr learned my addr 172.21.15.27:0/3823716228 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92ad752640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a815c620 0x7f92a80ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92a7fff640 1 -- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a815c620 msgr2=0x7f92a80ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92a7fff640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a815c620 0x7f92a80ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92a7fff640 1 -- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a80b55c0 msgr2=0x7f92a8156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92acf51640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a80b55c0 0x7f92a8156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92a7fff640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a80b55c0 0x7f92a8156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92a7fff640 1 -- 172.21.15.27:0/3823716228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c0094d0 con 0x7f92a80b82f0 2023-08-11T15:53:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92ad752640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a815c620 0x7f92a80ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92acf51640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a80b55c0 0x7f92a8156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92a7fff640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a80b82f0 0x7f92a815d8d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f929400d7d0 tx=0x7f929400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92a57fa640 1 -- 172.21.15.27:0/3823716228 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294018070 con 0x7f92a80b82f0 2023-08-11T15:53:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92a57fa640 1 -- 172.21.15.27:0/3823716228 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9294004020 con 0x7f92a80b82f0 2023-08-11T15:53:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92a57fa640 1 -- 172.21.15.27:0/3823716228 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92940041b0 con 0x7f92a80b82f0 2023-08-11T15:53:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.591+0000 7f92af1b5640 1 -- 172.21.15.27:0/3823716228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a80b3910 con 0x7f92a80b82f0 2023-08-11T15:53:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.592+0000 7f92af1b5640 1 -- 172.21.15.27:0/3823716228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a8156fe0 con 0x7f92a80b82f0 2023-08-11T15:53:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.593+0000 7f92af1b5640 1 -- 172.21.15.27:0/3823716228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a8157170 con 0x7f92a80b82f0 2023-08-11T15:53:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.593+0000 7f92a57fa640 1 -- 172.21.15.27:0/3823716228 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9294004310 con 0x7f92a80b82f0 2023-08-11T15:53:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.594+0000 7f92a57fa640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f928c06a050 0x7f928c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.595+0000 7f92a57fa640 1 -- 172.21.15.27:0/3823716228 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9294088310 con 0x7f92a80b82f0 2023-08-11T15:53:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.595+0000 7f92acf51640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f928c06a050 0x7f928c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.596+0000 7f92acf51640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f928c06a050 0x7f928c06c510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f929c036000 tx=0x7f929c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.599+0000 7f92a57fa640 1 -- 172.21.15.27:0/3823716228 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9294057600 con 0x7f92a80b82f0 2023-08-11T15:53:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.860+0000 7f92af1b5640 1 -- 172.21.15.27:0/3823716228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f92a8073990 con 0x7f92a80b82f0 2023-08-11T15:53:02.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.868+0000 7f92a57fa640 1 -- 172.21.15.27:0/3823716228 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9294057190 con 0x7f92a80b82f0 2023-08-11T15:53:02.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 -- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f928c06a050 msgr2=0x7f928c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f928c06a050 0x7f928c06c510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f929c036000 tx=0x7f929c03c040 comp rx=0 tx=0).stop 2023-08-11T15:53:02.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 -- 172.21.15.27:0/3823716228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a80b82f0 msgr2=0x7f92a815d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:02.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a80b82f0 0x7f92a815d8d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f929400d7d0 tx=0x7f929400dc90 comp rx=0 tx=0).stop 2023-08-11T15:53:02.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 -- 172.21.15.27:0/3823716228 shutdown_connections 2023-08-11T15:53:02.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f928c06a050 0x7f928c06c510 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92a815c620 0x7f92a80ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92a80b82f0 0x7f92a815d8d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 --2- 172.21.15.27:0/3823716228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a80b55c0 0x7f92a8156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:02.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 -- 172.21.15.27:0/3823716228 >> 172.21.15.27:0/3823716228 conn(0x7f92a8071110 msgr2=0x7f92a80d25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:02.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 -- 172.21.15.27:0/3823716228 shutdown_connections 2023-08-11T15:53:02.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:02.871+0000 7f92af1b5640 1 -- 172.21.15.27:0/3823716228 wait complete. 2023-08-11T15:53:02.893 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-08-11T15:53:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.117+0000 7faf3759e640 1 Processor -- start 2023-08-11T15:53:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.117+0000 7faf3759e640 1 -- start start 2023-08-11T15:53:03.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.119+0000 7faf3759e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faf380c94d0 0x7faf3815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.119+0000 7faf3759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faf3815d740 0x7faf380cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.119+0000 7faf3759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3815ca90 con 0x7faf381590a0 2023-08-11T15:53:03.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.119+0000 7faf3759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf380ca200 con 0x7faf3815d740 2023-08-11T15:53:03.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.119+0000 7faf3759e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380c8db0 con 0x7faf380c94d0 2023-08-11T15:53:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.119+0000 7faf3659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faf381590a0 0x7faf380c86a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34252/0 (socket says 172.21.15.27:34252) 2023-08-11T15:53:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.119+0000 7faf3659c640 1 -- 172.21.15.27:0/4134942117 learned_addr learned my addr 172.21.15.27:0/4134942117 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:03.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.120+0000 7faf36d9d640 1 --2- 172.21.15.27:0/4134942117 >> v2:172.21.15.27:3300/0 conn(0x7faf3815d740 0x7faf380cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.120+0000 7faf35d9b640 1 -- 172.21.15.27:0/4134942117 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279954010 0 0) 0x7faf3815ca90 con 0x7faf381590a0 2023-08-11T15:53:03.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.120+0000 7faf35d9b640 1 -- 172.21.15.27:0/4134942117 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf24003520 con 0x7faf381590a0 2023-08-11T15:53:03.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.120+0000 7faf2ffff640 1 --2- 172.21.15.27:0/4134942117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faf380c94d0 0x7faf3815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.120+0000 7faf36d9d640 1 -- 172.21.15.27:0/4134942117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faf380c94d0 msgr2=0x7faf3815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:03.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.120+0000 7faf36d9d640 1 --2- 172.21.15.27:0/4134942117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faf380c94d0 0x7faf3815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.120+0000 7faf36d9d640 1 -- 172.21.15.27:0/4134942117 >> v1:172.21.15.27:6789/0 conn(0x7faf381590a0 legacy=0x7faf380c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.120+0000 7faf35d9b640 1 -- 172.21.15.27:0/4134942117 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4142673996 0 0) 0x7faf24003520 con 0x7faf381590a0 2023-08-11T15:53:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.120+0000 7faf36d9d640 1 -- 172.21.15.27:0/4134942117 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380c9960 con 0x7faf3815d740 2023-08-11T15:53:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.121+0000 7faf36d9d640 1 --2- 172.21.15.27:0/4134942117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf3815d740 0x7faf380cd2a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7faf30009dc0 tx=0x7faf3002bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1965ffd2899efaa server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.121+0000 7faf35d9b640 1 -- 172.21.15.27:0/4134942117 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28001670 con 0x7faf3815d740 2023-08-11T15:53:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.122+0000 7faf35d9b640 1 -- 172.21.15.27:0/4134942117 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3002c880 con 0x7faf3815d740 2023-08-11T15:53:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.122+0000 7faf35d9b640 1 -- 172.21.15.27:0/4134942117 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3002cbf0 con 0x7faf3815d740 2023-08-11T15:53:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.122+0000 7faf3759e640 1 -- 172.21.15.27:0/4134942117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf3815d740 msgr2=0x7faf380cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.122+0000 7faf3759e640 1 --2- 172.21.15.27:0/4134942117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf3815d740 0x7faf380cd2a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7faf30009dc0 tx=0x7faf3002bd50 comp rx=0 tx=0).stop 2023-08-11T15:53:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.122+0000 7faf3759e640 1 -- 172.21.15.27:0/4134942117 shutdown_connections 2023-08-11T15:53:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.122+0000 7faf3759e640 1 --2- 172.21.15.27:0/4134942117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf3815d740 0x7faf380cd2a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.122+0000 7faf3759e640 1 --2- 172.21.15.27:0/4134942117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faf380c94d0 0x7faf3815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.122+0000 7faf3759e640 1 -- 172.21.15.27:0/4134942117 >> 172.21.15.27:0/4134942117 conn(0x7faf380afb70 msgr2=0x7faf380aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.122+0000 7faf3759e640 1 -- 172.21.15.27:0/4134942117 shutdown_connections 2023-08-11T15:53:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.122+0000 7faf3759e640 1 -- 172.21.15.27:0/4134942117 wait complete. 2023-08-11T15:53:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.124+0000 7faf3759e640 1 Processor -- start 2023-08-11T15:53:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.124+0000 7faf3759e640 1 -- start start 2023-08-11T15:53:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.125+0000 7faf3759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf381590a0 0x7faf3814cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.125+0000 7faf3759e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faf3815d740 0x7faf380b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.125+0000 7faf3759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf38164e30 0x7faf38149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.125+0000 7faf3759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf380b3cd0 con 0x7faf381590a0 2023-08-11T15:53:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.125+0000 7faf3759e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf381528d0 con 0x7faf38164e30 2023-08-11T15:53:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.125+0000 7faf3759e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf38152bb0 con 0x7faf3815d740 2023-08-11T15:53:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.125+0000 7faf3659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf381590a0 0x7faf3814cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.125+0000 7faf3659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf381590a0 0x7faf3814cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43484/0 (socket says 172.21.15.27:43484) 2023-08-11T15:53:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.125+0000 7faf3659c640 1 -- 172.21.15.27:0/1183642515 learned_addr learned my addr 172.21.15.27:0/1183642515 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.126+0000 7faf36d9d640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf38164e30 0x7faf38149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.126+0000 7faf2ffff640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faf3815d740 0x7faf380b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.126+0000 7faf3659c640 1 -- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faf3815d740 msgr2=0x7faf380b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.126+0000 7faf3659c640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faf3815d740 0x7faf380b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.126+0000 7faf3659c640 1 -- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf38164e30 msgr2=0x7faf38149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.126+0000 7faf3659c640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf38164e30 0x7faf38149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.126+0000 7faf3659c640 1 -- 172.21.15.27:0/1183642515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf30009a70 con 0x7faf381590a0 2023-08-11T15:53:03.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.126+0000 7faf36d9d640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf38164e30 0x7faf38149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:03.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.126+0000 7faf3659c640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf381590a0 0x7faf3814cd40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faf2800c750 tx=0x7faf2800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:03.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.127+0000 7faf2effd640 1 -- 172.21.15.27:0/1183642515 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28005ad0 con 0x7faf381590a0 2023-08-11T15:53:03.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.127+0000 7faf2effd640 1 -- 172.21.15.27:0/1183642515 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf28005c50 con 0x7faf381590a0 2023-08-11T15:53:03.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.127+0000 7faf2effd640 1 -- 172.21.15.27:0/1183642515 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28007070 con 0x7faf381590a0 2023-08-11T15:53:03.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.127+0000 7faf3759e640 1 -- 172.21.15.27:0/1183642515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf38152e90 con 0x7faf381590a0 2023-08-11T15:53:03.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.127+0000 7faf3759e640 1 -- 172.21.15.27:0/1183642515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3814da00 con 0x7faf381590a0 2023-08-11T15:53:03.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.128+0000 7faf3759e640 1 -- 172.21.15.27:0/1183642515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3804e420 con 0x7faf381590a0 2023-08-11T15:53:03.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.129+0000 7faf2effd640 1 -- 172.21.15.27:0/1183642515 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faf280071d0 con 0x7faf381590a0 2023-08-11T15:53:03.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.130+0000 7faf2effd640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf1806a080 0x7faf1806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.130+0000 7faf2effd640 1 -- 172.21.15.27:0/1183642515 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7faf28088130 con 0x7faf381590a0 2023-08-11T15:53:03.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.131+0000 7faf2ffff640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf1806a080 0x7faf1806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.132+0000 7faf2ffff640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf1806a080 0x7faf1806c540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faf38158b80 tx=0x7faf20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:03.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.134+0000 7faf2effd640 1 -- 172.21.15.27:0/1183642515 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7faf28053a30 con 0x7faf381590a0 2023-08-11T15:53:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.388+0000 7faf3759e640 1 -- 172.21.15.27:0/1183642515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7faf380c86d0 con 0x7faf381590a0 2023-08-11T15:53:03.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.395+0000 7faf2effd640 1 -- 172.21.15.27:0/1183642515 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7faf28057050 con 0x7faf381590a0 2023-08-11T15:53:03.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.397+0000 7faf3759e640 1 -- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf1806a080 msgr2=0x7faf1806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.397+0000 7faf3759e640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf1806a080 0x7faf1806c540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faf38158b80 tx=0x7faf20008040 comp rx=0 tx=0).stop 2023-08-11T15:53:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.397+0000 7faf3759e640 1 -- 172.21.15.27:0/1183642515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf381590a0 msgr2=0x7faf3814cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.397+0000 7faf3759e640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf381590a0 0x7faf3814cd40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faf2800c750 tx=0x7faf2800cc10 comp rx=0 tx=0).stop 2023-08-11T15:53:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.398+0000 7faf3759e640 1 -- 172.21.15.27:0/1183642515 shutdown_connections 2023-08-11T15:53:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.398+0000 7faf3759e640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faf1806a080 0x7faf1806c540 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.398+0000 7faf3759e640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faf3815d740 0x7faf380b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.398+0000 7faf3759e640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf38164e30 0x7faf38149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.398+0000 7faf3759e640 1 --2- 172.21.15.27:0/1183642515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf381590a0 0x7faf3814cd40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.398+0000 7faf3759e640 1 -- 172.21.15.27:0/1183642515 >> 172.21.15.27:0/1183642515 conn(0x7faf380afb70 msgr2=0x7faf380cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.398+0000 7faf3759e640 1 -- 172.21.15.27:0/1183642515 shutdown_connections 2023-08-11T15:53:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.398+0000 7faf3759e640 1 -- 172.21.15.27:0/1183642515 wait complete. 2023-08-11T15:53:03.418 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-08-11T15:53:03.497 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-08-11T15:53:03.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.645+0000 7f1a695f3640 1 Processor -- start 2023-08-11T15:53:03.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.645+0000 7f1a695f3640 1 -- start start 2023-08-11T15:53:03.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.646+0000 7f1a695f3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1a6415a4a0 0x7f1a6406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.646+0000 7f1a695f3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a640b3070 0x7f1a640b3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.646+0000 7f1a695f3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6415e390 con 0x7f1a640cc600 2023-08-11T15:53:03.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.646+0000 7f1a695f3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6406ff30 con 0x7f1a640b3070 2023-08-11T15:53:03.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.646+0000 7f1a695f3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a640cca60 con 0x7f1a6415a4a0 2023-08-11T15:53:03.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.647+0000 7f1a637fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a640b3070 0x7f1a640b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.647+0000 7f1a637fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a640b3070 0x7f1a640b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34916/0 (socket says 172.21.15.27:34916) 2023-08-11T15:53:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.647+0000 7f1a637fe640 1 -- 172.21.15.27:0/771217580 learned_addr learned my addr 172.21.15.27:0/771217580 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.647+0000 7f1a62ffd640 1 --2- 172.21.15.27:0/771217580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1a6415a4a0 0x7f1a6406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.648+0000 7f1a637fe640 1 -- 172.21.15.27:0/771217580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1a6415a4a0 msgr2=0x7f1a6406f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.648+0000 7f1a637fe640 1 --2- 172.21.15.27:0/771217580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1a6415a4a0 0x7f1a6406f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.648+0000 7f1a637fe640 1 -- 172.21.15.27:0/771217580 >> v1:172.21.15.119:6789/0 conn(0x7f1a640cc600 legacy=0x7f1a6415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.648+0000 7f1a637fe640 1 -- 172.21.15.27:0/771217580 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6406f690 con 0x7f1a640b3070 2023-08-11T15:53:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.649+0000 7f1a637fe640 1 --2- 172.21.15.27:0/771217580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a640b3070 0x7f1a640b3500 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0098a0 tx=0x7f1a5c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71bcb85772e3ccb4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.650+0000 7f1a627fc640 1 -- 172.21.15.27:0/771217580 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c030840 con 0x7f1a640b3070 2023-08-11T15:53:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.650+0000 7f1a627fc640 1 -- 172.21.15.27:0/771217580 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c0309a0 con 0x7f1a640b3070 2023-08-11T15:53:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.650+0000 7f1a627fc640 1 -- 172.21.15.27:0/771217580 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c030d30 con 0x7f1a640b3070 2023-08-11T15:53:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.650+0000 7f1a695f3640 1 -- 172.21.15.27:0/771217580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a640b3070 msgr2=0x7f1a640b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.650+0000 7f1a695f3640 1 --2- 172.21.15.27:0/771217580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a640b3070 0x7f1a640b3500 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0098a0 tx=0x7f1a5c02b980 comp rx=0 tx=0).stop 2023-08-11T15:53:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.651+0000 7f1a695f3640 1 -- 172.21.15.27:0/771217580 shutdown_connections 2023-08-11T15:53:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.651+0000 7f1a695f3640 1 --2- 172.21.15.27:0/771217580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a640b3070 0x7f1a640b3500 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.651+0000 7f1a695f3640 1 --2- 172.21.15.27:0/771217580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1a6415a4a0 0x7f1a6406f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.651+0000 7f1a695f3640 1 -- 172.21.15.27:0/771217580 >> 172.21.15.27:0/771217580 conn(0x7f1a640d0220 msgr2=0x7f1a640d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.651+0000 7f1a695f3640 1 -- 172.21.15.27:0/771217580 shutdown_connections 2023-08-11T15:53:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.651+0000 7f1a695f3640 1 -- 172.21.15.27:0/771217580 wait complete. 2023-08-11T15:53:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.653+0000 7f1a695f3640 1 Processor -- start 2023-08-11T15:53:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.653+0000 7f1a695f3640 1 -- start start 2023-08-11T15:53:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.654+0000 7f1a695f3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1a640b3070 0x7f1a64159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.654+0000 7f1a695f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a640cc600 0x7f1a64158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.654+0000 7f1a695f3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a6415a4a0 0x7f1a640c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.654+0000 7f1a695f3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a641e4980 con 0x7f1a6415a4a0 2023-08-11T15:53:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.654+0000 7f1a695f3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a640c8990 con 0x7f1a640cc600 2023-08-11T15:53:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.654+0000 7f1a695f3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64158e80 con 0x7f1a640b3070 2023-08-11T15:53:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.654+0000 7f1a637fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a6415a4a0 0x7f1a640c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.654+0000 7f1a637fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a6415a4a0 0x7f1a640c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43492/0 (socket says 172.21.15.27:43492) 2023-08-11T15:53:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.654+0000 7f1a637fe640 1 -- 172.21.15.27:0/2163657159 learned_addr learned my addr 172.21.15.27:0/2163657159 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:03.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.654+0000 7f1a5a7fc640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a640cc600 0x7f1a64158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a62ffd640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1a640b3070 0x7f1a64159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a637fe640 1 -- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1a640b3070 msgr2=0x7f1a64159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a637fe640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1a640b3070 0x7f1a64159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a637fe640 1 -- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a640cc600 msgr2=0x7f1a64158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a637fe640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a640cc600 0x7f1a64158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a637fe640 1 -- 172.21.15.27:0/2163657159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0094d0 con 0x7f1a6415a4a0 2023-08-11T15:53:03.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a637fe640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a6415a4a0 0x7f1a640c8ce0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c02b9c0 tx=0x7f1a5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:03.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a5bfff640 1 -- 172.21.15.27:0/2163657159 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c039c90 con 0x7f1a6415a4a0 2023-08-11T15:53:03.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a5bfff640 1 -- 172.21.15.27:0/2163657159 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c039df0 con 0x7f1a6415a4a0 2023-08-11T15:53:03.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a5bfff640 1 -- 172.21.15.27:0/2163657159 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c043640 con 0x7f1a6415a4a0 2023-08-11T15:53:03.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.655+0000 7f1a695f3640 1 -- 172.21.15.27:0/2163657159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a641e3fe0 con 0x7f1a6415a4a0 2023-08-11T15:53:03.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.656+0000 7f1a695f3640 1 -- 172.21.15.27:0/2163657159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a641e45a0 con 0x7f1a6415a4a0 2023-08-11T15:53:03.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.657+0000 7f1a695f3640 1 -- 172.21.15.27:0/2163657159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a641e4ac0 con 0x7f1a6415a4a0 2023-08-11T15:53:03.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.658+0000 7f1a5bfff640 1 -- 172.21.15.27:0/2163657159 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1a5c0437a0 con 0x7f1a6415a4a0 2023-08-11T15:53:03.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.659+0000 7f1a5bfff640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1a4406a050 0x7f1a4406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:03.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.659+0000 7f1a5bfff640 1 -- 172.21.15.27:0/2163657159 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1a5c03e070 con 0x7f1a6415a4a0 2023-08-11T15:53:03.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.659+0000 7f1a62ffd640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1a4406a050 0x7f1a4406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:03.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.660+0000 7f1a62ffd640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1a4406a050 0x7f1a4406c510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1a640aed50 tx=0x7f1a5400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:03.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.663+0000 7f1a5bfff640 1 -- 172.21.15.27:0/2163657159 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f1a5c076cf0 con 0x7f1a6415a4a0 2023-08-11T15:53:03.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.927+0000 7f1a695f3640 1 -- 172.21.15.27:0/2163657159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f1a6406f970 con 0x7f1a6415a4a0 2023-08-11T15:53:03.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.937+0000 7f1a5bfff640 1 -- 172.21.15.27:0/2163657159 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1a5c07a310 con 0x7f1a6415a4a0 2023-08-11T15:53:03.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.939+0000 7f1a695f3640 1 -- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1a4406a050 msgr2=0x7f1a4406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1a4406a050 0x7f1a4406c510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1a640aed50 tx=0x7f1a5400b040 comp rx=0 tx=0).stop 2023-08-11T15:53:03.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 -- 172.21.15.27:0/2163657159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a6415a4a0 msgr2=0x7f1a640c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:03.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a6415a4a0 0x7f1a640c8ce0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c02b9c0 tx=0x7f1a5c004730 comp rx=0 tx=0).stop 2023-08-11T15:53:03.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 -- 172.21.15.27:0/2163657159 shutdown_connections 2023-08-11T15:53:03.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1a4406a050 0x7f1a4406c510 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a6415a4a0 0x7f1a640c8ce0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a640cc600 0x7f1a64158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 --2- 172.21.15.27:0/2163657159 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1a640b3070 0x7f1a64159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:03.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 -- 172.21.15.27:0/2163657159 >> 172.21.15.27:0/2163657159 conn(0x7f1a640d0220 msgr2=0x7f1a640afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:03.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 -- 172.21.15.27:0/2163657159 shutdown_connections 2023-08-11T15:53:03.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:03.940+0000 7f1a695f3640 1 -- 172.21.15.27:0/2163657159 wait complete. 2023-08-11T15:53:03.960 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-08-11T15:53:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.194+0000 7f1334d64640 1 Processor -- start 2023-08-11T15:53:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.194+0000 7f1334d64640 1 -- start start 2023-08-11T15:53:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.196+0000 7f1334d64640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cb0c0 0x7f133015a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.196+0000 7f1334d64640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13300cbb70 0x7f13300708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.196+0000 7f1334d64640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13300cc020 con 0x7f1330156e50 2023-08-11T15:53:04.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.196+0000 7f1334d64640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13300ca460 con 0x7f13300cbb70 2023-08-11T15:53:04.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.196+0000 7f1334d64640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1330070470 con 0x7f13300cb0c0 2023-08-11T15:53:04.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.196+0000 7f132ed9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13300cbb70 0x7f13300708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.197+0000 7f132e59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1330156e50 0x7f13300ca9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:54260/0 (socket says 172.21.15.27:54260) 2023-08-11T15:53:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.197+0000 7f132e59c640 1 -- 172.21.15.27:0/2336501020 learned_addr learned my addr 172.21.15.27:0/2336501020 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.197+0000 7f132ed9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13300cbb70 0x7f13300708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34928/0 (socket says 172.21.15.27:34928) 2023-08-11T15:53:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.197+0000 7f132dd9b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cb0c0 0x7f133015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.197+0000 7f132dd9b640 1 -- 172.21.15.27:0/2336501020 >> v2:172.21.15.119:3300/0 conn(0x7f13300cbb70 msgr2=0x7f13300708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.197+0000 7f132dd9b640 1 --2- 172.21.15.27:0/2336501020 >> v2:172.21.15.119:3300/0 conn(0x7f13300cbb70 0x7f13300708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.197+0000 7f132dd9b640 1 -- 172.21.15.27:0/2336501020 >> v1:172.21.15.119:6789/0 conn(0x7f1330156e50 legacy=0x7f13300ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.197+0000 7f132dd9b640 1 -- 172.21.15.27:0/2336501020 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13300cc270 con 0x7f13300cb0c0 2023-08-11T15:53:04.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.197+0000 7f132ed9d640 1 --2- 172.21.15.27:0/2336501020 >> v2:172.21.15.119:3300/0 conn(0x7f13300cbb70 0x7f13300708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.199+0000 7f132dd9b640 1 --2- 172.21.15.27:0/2336501020 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cb0c0 0x7f133015a180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13180097a0 tx=0x7f131802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c89feb89d3d0de0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.200+0000 7f132d59a640 1 -- 172.21.15.27:0/2336501020 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131802d040 con 0x7f13300cb0c0 2023-08-11T15:53:04.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.200+0000 7f132d59a640 1 -- 172.21.15.27:0/2336501020 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1318031810 con 0x7f13300cb0c0 2023-08-11T15:53:04.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.200+0000 7f132d59a640 1 -- 172.21.15.27:0/2336501020 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318031ba0 con 0x7f13300cb0c0 2023-08-11T15:53:04.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.200+0000 7f1334d64640 1 -- 172.21.15.27:0/2336501020 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cb0c0 msgr2=0x7f133015a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.200+0000 7f1334d64640 1 --2- 172.21.15.27:0/2336501020 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cb0c0 0x7f133015a180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13180097a0 tx=0x7f131802b8b0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.201+0000 7f1334d64640 1 -- 172.21.15.27:0/2336501020 shutdown_connections 2023-08-11T15:53:04.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.201+0000 7f1334d64640 1 --2- 172.21.15.27:0/2336501020 >> v2:172.21.15.119:3300/0 conn(0x7f13300cbb70 0x7f13300708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.201+0000 7f1334d64640 1 --2- 172.21.15.27:0/2336501020 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cb0c0 0x7f133015a180 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.201+0000 7f1334d64640 1 -- 172.21.15.27:0/2336501020 >> 172.21.15.27:0/2336501020 conn(0x7f13300afb70 msgr2=0x7f13300aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.201+0000 7f1334d64640 1 -- 172.21.15.27:0/2336501020 shutdown_connections 2023-08-11T15:53:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.201+0000 7f1334d64640 1 -- 172.21.15.27:0/2336501020 wait complete. 2023-08-11T15:53:04.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.203+0000 7f1334d64640 1 Processor -- start 2023-08-11T15:53:04.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.203+0000 7f1334d64640 1 -- start start 2023-08-11T15:53:04.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f1334d64640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13300cb0c0 0x7f13300c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f1334d64640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cbb70 0x7f13300ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f1334d64640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1330156e50 0x7f13300c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f1334d64640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13300aedf0 con 0x7f1330156e50 2023-08-11T15:53:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f1334d64640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13300c4a30 con 0x7f13300cb0c0 2023-08-11T15:53:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f1334d64640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13300ae170 con 0x7f13300cbb70 2023-08-11T15:53:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f132ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1330156e50 0x7f13300c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f132ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1330156e50 0x7f13300c1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43504/0 (socket says 172.21.15.27:43504) 2023-08-11T15:53:04.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f132ed9d640 1 -- 172.21.15.27:0/1057618189 learned_addr learned my addr 172.21.15.27:0/1057618189 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:04.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f132e59c640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13300cb0c0 0x7f13300c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.205+0000 7f132dd9b640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cbb70 0x7f13300ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.206+0000 7f132ed9d640 1 -- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cbb70 msgr2=0x7f13300ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.206+0000 7f132ed9d640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cbb70 0x7f13300ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.206+0000 7f132ed9d640 1 -- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13300cb0c0 msgr2=0x7f13300c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.206+0000 7f132ed9d640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13300cb0c0 0x7f13300c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.206+0000 7f132ed9d640 1 -- 172.21.15.27:0/1057618189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1328009400 con 0x7f1330156e50 2023-08-11T15:53:04.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.206+0000 7f132e59c640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13300cb0c0 0x7f13300c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:04.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.206+0000 7f132ed9d640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1330156e50 0x7f13300c1c90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f132800e930 tx=0x7f132800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:04.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.206+0000 7f1326ffd640 1 -- 172.21.15.27:0/1057618189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328009700 con 0x7f1330156e50 2023-08-11T15:53:04.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.206+0000 7f1326ffd640 1 -- 172.21.15.27:0/1057618189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132800ca00 con 0x7f1330156e50 2023-08-11T15:53:04.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.206+0000 7f1334d64640 1 -- 172.21.15.27:0/1057618189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1318009400 con 0x7f1330156e50 2023-08-11T15:53:04.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.207+0000 7f1326ffd640 1 -- 172.21.15.27:0/1057618189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132800cd90 con 0x7f1330156e50 2023-08-11T15:53:04.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.207+0000 7f1334d64640 1 -- 172.21.15.27:0/1057618189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13300ae730 con 0x7f1330156e50 2023-08-11T15:53:04.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.208+0000 7f1334d64640 1 -- 172.21.15.27:0/1057618189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13300aef30 con 0x7f1330156e50 2023-08-11T15:53:04.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.209+0000 7f1326ffd640 1 -- 172.21.15.27:0/1057618189 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1328013470 con 0x7f1330156e50 2023-08-11T15:53:04.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.210+0000 7f1326ffd640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f131006a080 0x7f131006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.210+0000 7f1326ffd640 1 -- 172.21.15.27:0/1057618189 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1328087f90 con 0x7f1330156e50 2023-08-11T15:53:04.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.210+0000 7f132e59c640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f131006a080 0x7f131006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.212+0000 7f132e59c640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f131006a080 0x7f131006c540 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1320005c50 tx=0x7f1320009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:04.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.216+0000 7f1326ffd640 1 -- 172.21.15.27:0/1057618189 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f13300aef30 con 0x7f1330156e50 2023-08-11T15:53:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.481+0000 7f1334d64640 1 -- 172.21.15.27:0/1057618189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f13300aef30 con 0x7f1330156e50 2023-08-11T15:53:04.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.490+0000 7f1326ffd640 1 -- 172.21.15.27:0/1057618189 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f13300aef30 con 0x7f1330156e50 2023-08-11T15:53:04.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.492+0000 7f1334d64640 1 -- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f131006a080 msgr2=0x7f131006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.492+0000 7f1334d64640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f131006a080 0x7f131006c540 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1320005c50 tx=0x7f1320009040 comp rx=0 tx=0).stop 2023-08-11T15:53:04.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.492+0000 7f1334d64640 1 -- 172.21.15.27:0/1057618189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1330156e50 msgr2=0x7f13300c1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.492+0000 7f1334d64640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1330156e50 0x7f13300c1c90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f132800e930 tx=0x7f132800edf0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.493+0000 7f1334d64640 1 -- 172.21.15.27:0/1057618189 shutdown_connections 2023-08-11T15:53:04.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.493+0000 7f1334d64640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f131006a080 0x7f131006c540 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.493+0000 7f1334d64640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13300cbb70 0x7f13300ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.493+0000 7f1334d64640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1330156e50 0x7f13300c1c90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.493+0000 7f1334d64640 1 --2- 172.21.15.27:0/1057618189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13300cb0c0 0x7f13300c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.493+0000 7f1334d64640 1 -- 172.21.15.27:0/1057618189 >> 172.21.15.27:0/1057618189 conn(0x7f13300afb70 msgr2=0x7f13300aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:04.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.493+0000 7f1334d64640 1 -- 172.21.15.27:0/1057618189 shutdown_connections 2023-08-11T15:53:04.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.493+0000 7f1334d64640 1 -- 172.21.15.27:0/1057618189 wait complete. 2023-08-11T15:53:04.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:53:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.748+0000 7fe2059cf640 1 Processor -- start 2023-08-11T15:53:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.748+0000 7fe2059cf640 1 -- start start 2023-08-11T15:53:04.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.750+0000 7fe2059cf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe2000cc520 0x7fe20015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.750+0000 7fe2059cf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 0x7fe2000c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.750+0000 7fe2059cf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20015da30 con 0x7fe200160d20 2023-08-11T15:53:04.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.750+0000 7fe2059cf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe200156fa0 con 0x7fe2000cc520 2023-08-11T15:53:04.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.750+0000 7fe2059cf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000ca570 con 0x7fe20015d1e0 2023-08-11T15:53:04.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.751+0000 7fe1ff7fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 0x7fe2000c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.751+0000 7fe1fe7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe2000cc520 0x7fe20015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.751+0000 7fe1ff7fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 0x7fe2000c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:49788/0 (socket says 172.21.15.27:49788) 2023-08-11T15:53:04.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.751+0000 7fe1feffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe200160d20 0x7fe2000cbe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:54264/0 (socket says 172.21.15.27:54264) 2023-08-11T15:53:04.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.751+0000 7fe1ff7fe640 1 -- 172.21.15.27:0/1820594716 learned_addr learned my addr 172.21.15.27:0/1820594716 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:04.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.752+0000 7fe1fe7fc640 1 -- 172.21.15.27:0/1820594716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 msgr2=0x7fe2000c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:04.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.752+0000 7fe1fe7fc640 1 --2- 172.21.15.27:0/1820594716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 0x7fe2000c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.752+0000 7fe1fe7fc640 1 -- 172.21.15.27:0/1820594716 >> v1:172.21.15.119:6789/0 conn(0x7fe200160d20 legacy=0x7fe2000cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.752+0000 7fe1fdffb640 1 -- 172.21.15.27:0/1820594716 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700103476 0 0) 0x7fe20015da30 con 0x7fe200160d20 2023-08-11T15:53:04.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.752+0000 7fe1fe7fc640 1 -- 172.21.15.27:0/1820594716 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000ca850 con 0x7fe2000cc520 2023-08-11T15:53:04.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.753+0000 7fe1fe7fc640 1 --2- 172.21.15.27:0/1820594716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000cc520 0x7fe20015cca0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40097a0 tx=0x7fe1f402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd993716b590bd0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:04.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.754+0000 7fe1fdffb640 1 -- 172.21.15.27:0/1820594716 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4031830 con 0x7fe2000cc520 2023-08-11T15:53:04.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.754+0000 7fe1fdffb640 1 -- 172.21.15.27:0/1820594716 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4031990 con 0x7fe2000cc520 2023-08-11T15:53:04.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.754+0000 7fe1fdffb640 1 -- 172.21.15.27:0/1820594716 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4031d20 con 0x7fe2000cc520 2023-08-11T15:53:04.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.754+0000 7fe2059cf640 1 -- 172.21.15.27:0/1820594716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000cc520 msgr2=0x7fe20015cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.754+0000 7fe2059cf640 1 --2- 172.21.15.27:0/1820594716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000cc520 0x7fe20015cca0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40097a0 tx=0x7fe1f402b8d0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.755+0000 7fe2059cf640 1 -- 172.21.15.27:0/1820594716 shutdown_connections 2023-08-11T15:53:04.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.755+0000 7fe2059cf640 1 --2- 172.21.15.27:0/1820594716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 0x7fe2000c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.755+0000 7fe2059cf640 1 --2- 172.21.15.27:0/1820594716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000cc520 0x7fe20015cca0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.755+0000 7fe2059cf640 1 -- 172.21.15.27:0/1820594716 >> 172.21.15.27:0/1820594716 conn(0x7fe2000afb70 msgr2=0x7fe2000aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:04.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.755+0000 7fe2059cf640 1 -- 172.21.15.27:0/1820594716 shutdown_connections 2023-08-11T15:53:04.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.755+0000 7fe2059cf640 1 -- 172.21.15.27:0/1820594716 wait complete. 2023-08-11T15:53:04.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.757+0000 7fe2059cf640 1 Processor -- start 2023-08-11T15:53:04.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.757+0000 7fe2059cf640 1 -- start start 2023-08-11T15:53:04.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.759+0000 7fe2059cf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2000cc520 0x7fe2001570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.759+0000 7fe2059cf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 0x7fe2000c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.759+0000 7fe2059cf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe200160d20 0x7fe2000c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.759+0000 7fe2059cf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c5330 con 0x7fe2000cc520 2023-08-11T15:53:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.759+0000 7fe2059cf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200157620 con 0x7fe200160d20 2023-08-11T15:53:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.759+0000 7fe2059cf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c28f0 con 0x7fe20015d1e0 2023-08-11T15:53:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.759+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2000cc520 0x7fe2001570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.759+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2000cc520 0x7fe2001570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43516/0 (socket says 172.21.15.27:43516) 2023-08-11T15:53:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.759+0000 7fe1feffd640 1 -- 172.21.15.27:0/1413251412 learned_addr learned my addr 172.21.15.27:0/1413251412 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.760+0000 7fe1ff7fe640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe200160d20 0x7fe2000c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.760+0000 7fe1fe7fc640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 0x7fe2000c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.760+0000 7fe1feffd640 1 -- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 msgr2=0x7fe2000c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.760+0000 7fe1feffd640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 0x7fe2000c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.760+0000 7fe1feffd640 1 -- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe200160d20 msgr2=0x7fe2000c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.760+0000 7fe1feffd640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe200160d20 0x7fe2000c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.760+0000 7fe1feffd640 1 -- 172.21.15.27:0/1413251412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f4009400 con 0x7fe2000cc520 2023-08-11T15:53:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.760+0000 7fe1feffd640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2000cc520 0x7fe2001570e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000c450 tx=0x7fe1f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.761+0000 7fe1e7fff640 1 -- 172.21.15.27:0/1413251412 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0005c70 con 0x7fe2000cc520 2023-08-11T15:53:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.761+0000 7fe1e7fff640 1 -- 172.21.15.27:0/1413251412 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f0005dd0 con 0x7fe2000cc520 2023-08-11T15:53:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.761+0000 7fe1e7fff640 1 -- 172.21.15.27:0/1413251412 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0007270 con 0x7fe2000cc520 2023-08-11T15:53:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.761+0000 7fe2059cf640 1 -- 172.21.15.27:0/1413251412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2000c2bd0 con 0x7fe2000cc520 2023-08-11T15:53:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.761+0000 7fe2059cf640 1 -- 172.21.15.27:0/1413251412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2000c3190 con 0x7fe2000cc520 2023-08-11T15:53:04.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.762+0000 7fe2059cf640 1 -- 172.21.15.27:0/1413251412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe200073990 con 0x7fe2000cc520 2023-08-11T15:53:04.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.763+0000 7fe1e7fff640 1 -- 172.21.15.27:0/1413251412 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe1f00073d0 con 0x7fe2000cc520 2023-08-11T15:53:04.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.764+0000 7fe1e7fff640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe1ec06a050 0x7fe1ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:04.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.764+0000 7fe1e7fff640 1 -- 172.21.15.27:0/1413251412 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe1f0087840 con 0x7fe2000cc520 2023-08-11T15:53:04.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.765+0000 7fe1fe7fc640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe1ec06a050 0x7fe1ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:04.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.766+0000 7fe1fe7fc640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe1ec06a050 0x7fe1ec06c510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4036000 tx=0x7fe1f4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:04.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.768+0000 7fe1e7fff640 1 -- 172.21.15.27:0/1413251412 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe1f0053150 con 0x7fe2000cc520 2023-08-11T15:53:04.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.965+0000 7fe2059cf640 1 -- 172.21.15.27:0/1413251412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2000cb7f0 con 0x7fe2000cc520 2023-08-11T15:53:04.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.967+0000 7fe1e7fff640 1 -- 172.21.15.27:0/1413251412 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+14844 (secure 0 0 0) 0x7fe1f0056770 con 0x7fe2000cc520 2023-08-11T15:53:04.970 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:04.970 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:53:00.644709+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:53:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 -- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe1ec06a050 msgr2=0x7fe1ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe1ec06a050 0x7fe1ec06c510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4036000 tx=0x7fe1f4008030 comp rx=0 tx=0).stop 2023-08-11T15:53:04.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 -- 172.21.15.27:0/1413251412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2000cc520 msgr2=0x7fe2001570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:04.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2000cc520 0x7fe2001570e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000c450 tx=0x7fe1f000c910 comp rx=0 tx=0).stop 2023-08-11T15:53:04.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 -- 172.21.15.27:0/1413251412 shutdown_connections 2023-08-11T15:53:04.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe1ec06a050 0x7fe1ec06c510 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20015d1e0 0x7fe2000c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe200160d20 0x7fe2000c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 --2- 172.21.15.27:0/1413251412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2000cc520 0x7fe2001570e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:04.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 -- 172.21.15.27:0/1413251412 >> 172.21.15.27:0/1413251412 conn(0x7fe2000afb70 msgr2=0x7fe2000cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:04.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 -- 172.21.15.27:0/1413251412 shutdown_connections 2023-08-11T15:53:04.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:04.970+0000 7fe2059cf640 1 -- 172.21.15.27:0/1413251412 wait complete. 2023-08-11T15:53:04.993 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T15:53:04.993 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:53:05.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.231+0000 7fd6ff379640 1 Processor -- start 2023-08-11T15:53:05.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.231+0000 7fd6ff379640 1 -- start start 2023-08-11T15:53:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.233+0000 7fd6ff379640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6f80ca080 0x7fd6f815a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.233+0000 7fd6ff379640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd6f815ceb0 0x7fd6f80caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.233+0000 7fd6ff379640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f80cd040 con 0x7fd6f80cb530 2023-08-11T15:53:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.233+0000 7fd6ff379640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f80cb980 con 0x7fd6f80ca080 2023-08-11T15:53:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.233+0000 7fd6ff379640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f8070600 con 0x7fd6f815ceb0 2023-08-11T15:53:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.234+0000 7fd6fd115640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6f80ca080 0x7fd6f815a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.234+0000 7fd6fd115640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6f80ca080 0x7fd6f815a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43518/0 (socket says 172.21.15.27:43518) 2023-08-11T15:53:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.234+0000 7fd6fd115640 1 -- 172.21.15.27:0/3470128283 learned_addr learned my addr 172.21.15.27:0/3470128283 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:05.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.234+0000 7fd6fc914640 1 --2- 172.21.15.27:0/3470128283 >> v2:172.21.15.119:3300/0 conn(0x7fd6f815ceb0 0x7fd6f80caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.234+0000 7fd6fd115640 1 -- 172.21.15.27:0/3470128283 >> v2:172.21.15.119:3300/0 conn(0x7fd6f815ceb0 msgr2=0x7fd6f80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:05.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.234+0000 7fd6fd115640 1 --2- 172.21.15.27:0/3470128283 >> v2:172.21.15.119:3300/0 conn(0x7fd6f815ceb0 0x7fd6f80caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.234+0000 7fd6fd115640 1 -- 172.21.15.27:0/3470128283 >> v1:172.21.15.119:6789/0 conn(0x7fd6f80cb530 legacy=0x7fd6f806b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:05.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.234+0000 7fd6fd115640 1 -- 172.21.15.27:0/3470128283 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f815a830 con 0x7fd6f80ca080 2023-08-11T15:53:05.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.236+0000 7fd6fd115640 1 --2- 172.21.15.27:0/3470128283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f80ca080 0x7fd6f815a2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80ccaa0 tx=0x7fd6f402da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b118c0c0dbcd3da3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:05.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.236+0000 7fd6effff640 1 -- 172.21.15.27:0/3470128283 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f402f850 con 0x7fd6f80ca080 2023-08-11T15:53:05.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.236+0000 7fd6effff640 1 -- 172.21.15.27:0/3470128283 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f402f9b0 con 0x7fd6f80ca080 2023-08-11T15:53:05.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.236+0000 7fd6effff640 1 -- 172.21.15.27:0/3470128283 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f402fd40 con 0x7fd6f80ca080 2023-08-11T15:53:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.237+0000 7fd6ff379640 1 -- 172.21.15.27:0/3470128283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f80ca080 msgr2=0x7fd6f815a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.237+0000 7fd6ff379640 1 --2- 172.21.15.27:0/3470128283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f80ca080 0x7fd6f815a2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80ccaa0 tx=0x7fd6f402da90 comp rx=0 tx=0).stop 2023-08-11T15:53:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.237+0000 7fd6ff379640 1 -- 172.21.15.27:0/3470128283 shutdown_connections 2023-08-11T15:53:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.237+0000 7fd6ff379640 1 --2- 172.21.15.27:0/3470128283 >> v2:172.21.15.119:3300/0 conn(0x7fd6f815ceb0 0x7fd6f80caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.237+0000 7fd6ff379640 1 --2- 172.21.15.27:0/3470128283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f80ca080 0x7fd6f815a2f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.237+0000 7fd6ff379640 1 -- 172.21.15.27:0/3470128283 >> 172.21.15.27:0/3470128283 conn(0x7fd6f8069bf0 msgr2=0x7fd6f806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.237+0000 7fd6ff379640 1 -- 172.21.15.27:0/3470128283 shutdown_connections 2023-08-11T15:53:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.237+0000 7fd6ff379640 1 -- 172.21.15.27:0/3470128283 wait complete. 2023-08-11T15:53:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.239+0000 7fd6ff379640 1 Processor -- start 2023-08-11T15:53:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.239+0000 7fd6ff379640 1 -- start start 2023-08-11T15:53:05.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.240+0000 7fd6ff379640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6f80ca080 0x7fd6f80c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.240+0000 7fd6ff379640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6f80cb530 0x7fd6f8155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.240+0000 7fd6ff379640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f815ceb0 0x7fd6f8155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.240+0000 7fd6ff379640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80c3af0 con 0x7fd6f815ceb0 2023-08-11T15:53:05.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.240+0000 7fd6ff379640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80c5330 con 0x7fd6f80ca080 2023-08-11T15:53:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.240+0000 7fd6ff379640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80c2f70 con 0x7fd6f80cb530 2023-08-11T15:53:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fd916640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f815ceb0 0x7fd6f8155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fd916640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f815ceb0 0x7fd6f8155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43528/0 (socket says 172.21.15.27:43528) 2023-08-11T15:53:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fd916640 1 -- 172.21.15.27:0/499529 learned_addr learned my addr 172.21.15.27:0/499529 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fd115640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6f80ca080 0x7fd6f80c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fc914640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6f80cb530 0x7fd6f8155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fd916640 1 -- 172.21.15.27:0/499529 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6f80cb530 msgr2=0x7fd6f8155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fd916640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6f80cb530 0x7fd6f8155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fd916640 1 -- 172.21.15.27:0/499529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6f80ca080 msgr2=0x7fd6f80c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fd916640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6f80ca080 0x7fd6f80c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fd916640 1 -- 172.21.15.27:0/499529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f40094d0 con 0x7fd6f815ceb0 2023-08-11T15:53:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.241+0000 7fd6fc914640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6f80cb530 0x7fd6f8155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.242+0000 7fd6fd916640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f815ceb0 0x7fd6f8155580 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c780 tx=0x7fd6f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.242+0000 7fd6edffb640 1 -- 172.21.15.27:0/499529 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0003d80 con 0x7fd6f815ceb0 2023-08-11T15:53:05.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.242+0000 7fd6edffb640 1 -- 172.21.15.27:0/499529 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f000ce90 con 0x7fd6f815ceb0 2023-08-11T15:53:05.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.242+0000 7fd6ff379640 1 -- 172.21.15.27:0/499529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f80c3250 con 0x7fd6f815ceb0 2023-08-11T15:53:05.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.242+0000 7fd6edffb640 1 -- 172.21.15.27:0/499529 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00052b0 con 0x7fd6f815ceb0 2023-08-11T15:53:05.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.242+0000 7fd6ff379640 1 -- 172.21.15.27:0/499529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f80c3810 con 0x7fd6f815ceb0 2023-08-11T15:53:05.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.243+0000 7fd6ff379640 1 -- 172.21.15.27:0/499529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f804e420 con 0x7fd6f815ceb0 2023-08-11T15:53:05.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.244+0000 7fd6edffb640 1 -- 172.21.15.27:0/499529 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd6f0005410 con 0x7fd6f815ceb0 2023-08-11T15:53:05.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.245+0000 7fd6edffb640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6dc06a080 0x7fd6dc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.245+0000 7fd6edffb640 1 -- 172.21.15.27:0/499529 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd6f0087c40 con 0x7fd6f815ceb0 2023-08-11T15:53:05.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.246+0000 7fd6fd115640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6dc06a080 0x7fd6dc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.247+0000 7fd6fd115640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6dc06a080 0x7fd6dc06c540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4038000 tx=0x7fd6f403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:05.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.249+0000 7fd6edffb640 1 -- 172.21.15.27:0/499529 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fd6f00534a0 con 0x7fd6f815ceb0 2023-08-11T15:53:05.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.442+0000 7fd6ff379640 1 -- 172.21.15.27:0/499529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6f8153920 con 0x7fd6f815ceb0 2023-08-11T15:53:05.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.443+0000 7fd6edffb640 1 -- 172.21.15.27:0/499529 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6f0017070 con 0x7fd6f815ceb0 2023-08-11T15:53:05.448 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:53:05.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 -- 172.21.15.27:0/499529 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6dc06a080 msgr2=0x7fd6dc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6dc06a080 0x7fd6dc06c540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4038000 tx=0x7fd6f403b040 comp rx=0 tx=0).stop 2023-08-11T15:53:05.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 -- 172.21.15.27:0/499529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f815ceb0 msgr2=0x7fd6f8155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f815ceb0 0x7fd6f8155580 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c780 tx=0x7fd6f000cc40 comp rx=0 tx=0).stop 2023-08-11T15:53:05.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 -- 172.21.15.27:0/499529 shutdown_connections 2023-08-11T15:53:05.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd6dc06a080 0x7fd6dc06c540 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd6f80cb530 0x7fd6f8155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f815ceb0 0x7fd6f8155580 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 --2- 172.21.15.27:0/499529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6f80ca080 0x7fd6f80c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 -- 172.21.15.27:0/499529 >> 172.21.15.27:0/499529 conn(0x7fd6f8069bf0 msgr2=0x7fd6f80aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:05.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 -- 172.21.15.27:0/499529 shutdown_connections 2023-08-11T15:53:05.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.446+0000 7fd6ff379640 1 -- 172.21.15.27:0/499529 wait complete. 2023-08-11T15:53:05.469 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:53:05.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.705+0000 7f797b408640 1 Processor -- start 2023-08-11T15:53:05.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.705+0000 7f797b408640 1 -- start start 2023-08-11T15:53:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.707+0000 7f797b408640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740cf280 0x7f79740b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.707+0000 7f797b408640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79740cd890 con 0x7f79740b1580 2023-08-11T15:53:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.707+0000 7f797b408640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79740b4b10 con 0x7f797415c2a0 2023-08-11T15:53:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.707+0000 7f797b408640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79740b4140 con 0x7f79740cf280 2023-08-11T15:53:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.707+0000 7f7979c05640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f79740b1580 0x7f79740ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34268/0 (socket says 172.21.15.27:34268) 2023-08-11T15:53:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.707+0000 7f7979c05640 1 -- 172.21.15.27:0/3729346956 learned_addr learned my addr 172.21.15.27:0/3729346956 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.708+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570747942 0 0) 0x7f7964001110 con 0x7f79740b1580 2023-08-11T15:53:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.708+0000 7f797ac07640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740cf280 0x7f79740b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.708+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7960003520 con 0x7f79740b1580 2023-08-11T15:53:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.708+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2253383878 0 0) 0x7f7960003520 con 0x7f79740b1580 2023-08-11T15:53:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.708+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740cf280 msgr2=0x7f79740b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.708+0000 7f7979404640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740cf280 0x7f79740b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.708+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 >> v1:172.21.15.119:6789/0 conn(0x7f797415c2a0 legacy=0x7f79740ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.708+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79740cd2d0 con 0x7f79740b1580 2023-08-11T15:53:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f7964001800 con 0x7f79740b1580 2023-08-11T15:53:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 >> v1:172.21.15.27:6789/0 conn(0x7f79740b1580 legacy=0x7f79740ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f7979404640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740cf280 0x7f79600039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f7979404640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7960003f20 0x7f79600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f7979404640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 0x7f7960005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79740b4140 con 0x7f7960004d00 2023-08-11T15:53:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796000bb10 con 0x7f7960003f20 2023-08-11T15:53:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796000bcd0 con 0x7f79740cf280 2023-08-11T15:53:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f797ac07640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 0x7f7960005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f7979c05640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7960003f20 0x7f79600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.709+0000 7f797a406640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740cf280 0x7f79600039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.710+0000 7f797ac07640 1 -- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740cf280 msgr2=0x7f79600039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.710+0000 7f797ac07640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740cf280 0x7f79600039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.710+0000 7f797ac07640 1 -- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7960003f20 msgr2=0x7f79600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.710+0000 7f797ac07640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7960003f20 0x7f79600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.710+0000 7f797ac07640 1 -- 172.21.15.27:0/3729346956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7960003520 con 0x7f7960004d00 2023-08-11T15:53:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.711+0000 7f797ac07640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 0x7f7960005180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f796c009f90 tx=0x7f79600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08b6124db02c5fa server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.711+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c02cbd0 con 0x7f7960004d00 2023-08-11T15:53:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.711+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c00f040 con 0x7f7960004d00 2023-08-11T15:53:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.711+0000 7f7979404640 1 -- 172.21.15.27:0/3729346956 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c034440 con 0x7f7960004d00 2023-08-11T15:53:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.712+0000 7f797b408640 1 -- 172.21.15.27:0/3729346956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 msgr2=0x7f7960005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.712+0000 7f797b408640 1 --2- 172.21.15.27:0/3729346956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 0x7f7960005180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f796c009f90 tx=0x7f79600057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.712+0000 7f797ac07640 1 -- 172.21.15.27:0/3729346956 reap_dead start 2023-08-11T15:53:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.712+0000 7f797b408640 1 -- 172.21.15.27:0/3729346956 shutdown_connections 2023-08-11T15:53:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.712+0000 7f797b408640 1 -- 172.21.15.27:0/3729346956 >> 172.21.15.27:0/3729346956 conn(0x7f797406a250 msgr2=0x7f797406a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.712+0000 7f797b408640 1 -- 172.21.15.27:0/3729346956 shutdown_connections 2023-08-11T15:53:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.712+0000 7f797b408640 1 -- 172.21.15.27:0/3729346956 wait complete. 2023-08-11T15:53:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.714+0000 7f797b408640 1 Processor -- start 2023-08-11T15:53:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.714+0000 7f797b408640 1 -- start start 2023-08-11T15:53:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.715+0000 7f797b408640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 0x7f79741f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.715+0000 7f797b408640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740aadf0 0x7f797415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.715+0000 7f797b408640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79740c7660 0x7f79740cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.715+0000 7f797b408640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741f5b10 con 0x7f7960004d00 2023-08-11T15:53:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.715+0000 7f797b408640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79740ab390 con 0x7f79740c7660 2023-08-11T15:53:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.715+0000 7f797b408640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79740ab670 con 0x7f79740aadf0 2023-08-11T15:53:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.715+0000 7f797a406640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 0x7f79741f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.715+0000 7f797a406640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 0x7f79741f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43544/0 (socket says 172.21.15.27:43544) 2023-08-11T15:53:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.715+0000 7f797a406640 1 -- 172.21.15.27:0/4195727491 learned_addr learned my addr 172.21.15.27:0/4195727491 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f7979c05640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740aadf0 0x7f797415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f797ac07640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79740c7660 0x7f79740cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f797a406640 1 -- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740aadf0 msgr2=0x7f797415a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f797a406640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740aadf0 0x7f797415a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f797a406640 1 -- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79740c7660 msgr2=0x7f79740cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f797a406640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79740c7660 0x7f79740cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f797a406640 1 -- 172.21.15.27:0/4195727491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79740c6e60 con 0x7f7960004d00 2023-08-11T15:53:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f797a406640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 0x7f79741f55d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f797000e550 tx=0x7f797000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f796affd640 1 -- 172.21.15.27:0/4195727491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79700052e0 con 0x7f7960004d00 2023-08-11T15:53:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f796affd640 1 -- 172.21.15.27:0/4195727491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970005440 con 0x7f7960004d00 2023-08-11T15:53:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f796affd640 1 -- 172.21.15.27:0/4195727491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970003810 con 0x7f7960004d00 2023-08-11T15:53:05.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.716+0000 7f797b408640 1 -- 172.21.15.27:0/4195727491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79740c7140 con 0x7f7960004d00 2023-08-11T15:53:05.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.717+0000 7f797b408640 1 -- 172.21.15.27:0/4195727491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797415aec0 con 0x7f7960004d00 2023-08-11T15:53:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.718+0000 7f797b408640 1 -- 172.21.15.27:0/4195727491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79740cdae0 con 0x7f7960004d00 2023-08-11T15:53:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.719+0000 7f796affd640 1 -- 172.21.15.27:0/4195727491 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7970003970 con 0x7f7960004d00 2023-08-11T15:53:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.719+0000 7f796affd640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f795806a080 0x7f795806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.720+0000 7f796affd640 1 -- 172.21.15.27:0/4195727491 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f7970087fb0 con 0x7f7960004d00 2023-08-11T15:53:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.720+0000 7f7979c05640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f795806a080 0x7f795806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.721+0000 7f7979c05640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f795806a080 0x7f795806c540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7964001e10 tx=0x7f7964001110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.724+0000 7f796affd640 1 -- 172.21.15.27:0/4195727491 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f79700538a0 con 0x7f7960004d00 2023-08-11T15:53:05.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.915+0000 7f797b408640 1 -- 172.21.15.27:0/4195727491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f79740b4b10 con 0x7f7960004d00 2023-08-11T15:53:05.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.916+0000 7f796affd640 1 -- 172.21.15.27:0/4195727491 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7f79740b4b10 con 0x7f7960004d00 2023-08-11T15:53:05.918 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:53:05.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 -- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f795806a080 msgr2=0x7f795806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f795806a080 0x7f795806c540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7964001e10 tx=0x7f7964001110 comp rx=0 tx=0).stop 2023-08-11T15:53:05.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 -- 172.21.15.27:0/4195727491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 msgr2=0x7f79741f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:05.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 0x7f79741f55d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f797000e550 tx=0x7f797000ea10 comp rx=0 tx=0).stop 2023-08-11T15:53:05.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 -- 172.21.15.27:0/4195727491 shutdown_connections 2023-08-11T15:53:05.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f795806a080 0x7f795806c540 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79740aadf0 0x7f797415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79740c7660 0x7f79740cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 --2- 172.21.15.27:0/4195727491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7960004d00 0x7f79741f55d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:05.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 -- 172.21.15.27:0/4195727491 >> 172.21.15.27:0/4195727491 conn(0x7f797406a250 msgr2=0x7f79740cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:05.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.918+0000 7f797b408640 1 -- 172.21.15.27:0/4195727491 shutdown_connections 2023-08-11T15:53:05.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:05.919+0000 7f797b408640 1 -- 172.21.15.27:0/4195727491 wait complete. 2023-08-11T15:53:05.939 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-11T15:53:06.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.138+0000 7fe6fa7f5640 1 Processor -- start 2023-08-11T15:53:06.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.138+0000 7fe6fa7f5640 1 -- start start 2023-08-11T15:53:06.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.140+0000 7fe6fa7f5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 0x7fe6f40c8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.140+0000 7fe6fa7f5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe6f406bd30 0x7fe6f406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.140+0000 7fe6fa7f5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f415b760 con 0x7fe6f413d310 2023-08-11T15:53:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.140+0000 7fe6fa7f5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f40c8cd0 con 0x7fe6f406bd30 2023-08-11T15:53:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.140+0000 7fe6fa7f5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f41407b0 con 0x7fe6f40c8300 2023-08-11T15:53:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.140+0000 7fe6f97f3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe6f413d310 0x7fe6f415a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34276/0 (socket says 172.21.15.27:34276) 2023-08-11T15:53:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.140+0000 7fe6f97f3640 1 -- 172.21.15.27:0/3882841381 learned_addr learned my addr 172.21.15.27:0/3882841381 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:06.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.140+0000 7fe6f9ff4640 1 --2- 172.21.15.27:0/3882841381 >> v2:172.21.15.27:3300/0 conn(0x7fe6f406bd30 0x7fe6f406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.141+0000 7fe6f8ff2640 1 --2- 172.21.15.27:0/3882841381 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 0x7fe6f40c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.141+0000 7fe6ebfff640 1 -- 172.21.15.27:0/3882841381 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668266701 0 0) 0x7fe6f415b760 con 0x7fe6f413d310 2023-08-11T15:53:06.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.141+0000 7fe6ebfff640 1 -- 172.21.15.27:0/3882841381 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e0003520 con 0x7fe6f413d310 2023-08-11T15:53:06.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.141+0000 7fe6f9ff4640 1 -- 172.21.15.27:0/3882841381 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 msgr2=0x7fe6f40c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:06.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.141+0000 7fe6f9ff4640 1 --2- 172.21.15.27:0/3882841381 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 0x7fe6f40c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.141+0000 7fe6f9ff4640 1 -- 172.21.15.27:0/3882841381 >> v1:172.21.15.27:6789/0 conn(0x7fe6f413d310 legacy=0x7fe6f415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.141+0000 7fe6f9ff4640 1 -- 172.21.15.27:0/3882841381 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f415b1a0 con 0x7fe6f406bd30 2023-08-11T15:53:06.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.142+0000 7fe6f9ff4640 1 --2- 172.21.15.27:0/3882841381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 0x7fe6f406c1c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009d90 tx=0x7fe6ec02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5f0e888bebc66a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:06.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.143+0000 7fe6ebfff640 1 -- 172.21.15.27:0/3882841381 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec02cb40 con 0x7fe6f406bd30 2023-08-11T15:53:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.143+0000 7fe6ebfff640 1 -- 172.21.15.27:0/3882841381 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec02cca0 con 0x7fe6f406bd30 2023-08-11T15:53:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.143+0000 7fe6ebfff640 1 -- 172.21.15.27:0/3882841381 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec034640 con 0x7fe6f406bd30 2023-08-11T15:53:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.143+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/3882841381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 msgr2=0x7fe6f406c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.143+0000 7fe6fa7f5640 1 --2- 172.21.15.27:0/3882841381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 0x7fe6f406c1c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009d90 tx=0x7fe6ec02bd90 comp rx=0 tx=0).stop 2023-08-11T15:53:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.143+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/3882841381 shutdown_connections 2023-08-11T15:53:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.143+0000 7fe6fa7f5640 1 --2- 172.21.15.27:0/3882841381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 0x7fe6f406c1c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.143+0000 7fe6fa7f5640 1 --2- 172.21.15.27:0/3882841381 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 0x7fe6f40c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.143+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/3882841381 >> 172.21.15.27:0/3882841381 conn(0x7fe6f4069800 msgr2=0x7fe6f4069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.144+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/3882841381 shutdown_connections 2023-08-11T15:53:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.144+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/3882841381 wait complete. 2023-08-11T15:53:06.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.145+0000 7fe6fa7f5640 1 Processor -- start 2023-08-11T15:53:06.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.145+0000 7fe6fa7f5640 1 -- start start 2023-08-11T15:53:06.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.146+0000 7fe6fa7f5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 0x7fe6f41509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.146+0000 7fe6fa7f5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 0x7fe6f40c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.146+0000 7fe6fa7f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f413d310 0x7fe6f40c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.146+0000 7fe6fa7f5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40c7a20 con 0x7fe6f406bd30 2023-08-11T15:53:06.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.146+0000 7fe6fa7f5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4071280 con 0x7fe6f413d310 2023-08-11T15:53:06.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.146+0000 7fe6fa7f5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40c6e10 con 0x7fe6f40c8300 2023-08-11T15:53:06.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f97f3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 0x7fe6f41509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f97f3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 0x7fe6f41509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43572/0 (socket says 172.21.15.27:43572) 2023-08-11T15:53:06.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f97f3640 1 -- 172.21.15.27:0/1875855845 learned_addr learned my addr 172.21.15.27:0/1875855845 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:06.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f8ff2640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 0x7fe6f40c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f9ff4640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f413d310 0x7fe6f40c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f97f3640 1 -- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 msgr2=0x7fe6f40c3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f97f3640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 0x7fe6f40c3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f97f3640 1 -- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f413d310 msgr2=0x7fe6f40c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f97f3640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f413d310 0x7fe6f40c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f97f3640 1 -- 172.21.15.27:0/1875855845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec009a70 con 0x7fe6f406bd30 2023-08-11T15:53:06.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.147+0000 7fe6f8ff2640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 0x7fe6f40c3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:06.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.148+0000 7fe6f97f3640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 0x7fe6f41509b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000c430 tx=0x7fe6f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:06.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.148+0000 7fe6e9ffb640 1 -- 172.21.15.27:0/1875855845 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0005ce0 con 0x7fe6f406bd30 2023-08-11T15:53:06.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.148+0000 7fe6e9ffb640 1 -- 172.21.15.27:0/1875855845 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f0005e40 con 0x7fe6f406bd30 2023-08-11T15:53:06.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.148+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/1875855845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f40c70f0 con 0x7fe6f406bd30 2023-08-11T15:53:06.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.148+0000 7fe6e9ffb640 1 -- 172.21.15.27:0/1875855845 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00072f0 con 0x7fe6f406bd30 2023-08-11T15:53:06.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.148+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/1875855845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f40c76b0 con 0x7fe6f406bd30 2023-08-11T15:53:06.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.149+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/1875855845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f40728d0 con 0x7fe6f406bd30 2023-08-11T15:53:06.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.150+0000 7fe6e9ffb640 1 -- 172.21.15.27:0/1875855845 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe6f0007450 con 0x7fe6f406bd30 2023-08-11T15:53:06.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.152+0000 7fe6e9ffb640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe6d806a020 0x7fe6d806c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.152+0000 7fe6e9ffb640 1 -- 172.21.15.27:0/1875855845 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe6f0087a20 con 0x7fe6f406bd30 2023-08-11T15:53:06.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.153+0000 7fe6f8ff2640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe6d806a020 0x7fe6d806c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.154+0000 7fe6f8ff2640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe6d806a020 0x7fe6d806c4e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4006fd0 tx=0x7fe6e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:06.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.156+0000 7fe6e9ffb640 1 -- 172.21.15.27:0/1875855845 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe6f0053320 con 0x7fe6f406bd30 2023-08-11T15:53:06.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.353+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/1875855845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe6f4066e80 con 0x7fe6f406bd30 2023-08-11T15:53:06.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.353+0000 7fe6e9ffb640 1 -- 172.21.15.27:0/1875855845 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe6f0017090 con 0x7fe6f406bd30 2023-08-11T15:53:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:06.357 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-08-11T15:53:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-08-11T15:53:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-08-11T15:53:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-08-11T15:53:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 2, 2023-08-11T15:53:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": "rbd" 2023-08-11T15:53:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:06.359 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-08-11T15:53:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.356+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe6d806a020 msgr2=0x7fe6d806c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.356+0000 7fe6fa7f5640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe6d806a020 0x7fe6d806c4e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4006fd0 tx=0x7fe6e4008040 comp rx=0 tx=0).stop 2023-08-11T15:53:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.356+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/1875855845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 msgr2=0x7fe6f41509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.356+0000 7fe6fa7f5640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 0x7fe6f41509b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000c430 tx=0x7fe6f000c8f0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.356+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/1875855845 shutdown_connections 2023-08-11T15:53:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.357+0000 7fe6fa7f5640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe6d806a020 0x7fe6d806c4e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.357+0000 7fe6fa7f5640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6f40c8300 0x7fe6f40c3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.357+0000 7fe6fa7f5640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f413d310 0x7fe6f40c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.357+0000 7fe6fa7f5640 1 --2- 172.21.15.27:0/1875855845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6f406bd30 0x7fe6f41509b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.357+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/1875855845 >> 172.21.15.27:0/1875855845 conn(0x7fe6f4069800 msgr2=0x7fe6f415a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.357+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/1875855845 shutdown_connections 2023-08-11T15:53:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.357+0000 7fe6fa7f5640 1 -- 172.21.15.27:0/1875855845 wait complete. 2023-08-11T15:53:06.381 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-11T15:53:06.381 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-11T15:53:06.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.619+0000 7fa555a2d640 1 Processor -- start 2023-08-11T15:53:06.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.620+0000 7fa555a2d640 1 -- start start 2023-08-11T15:53:06.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.621+0000 7fa555a2d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa55015a180 0x7fa5500cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.621+0000 7fa555a2d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55015a610 con 0x7fa5500cb0c0 2023-08-11T15:53:06.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.621+0000 7fa555a2d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5500cc270 con 0x7fa550156e50 2023-08-11T15:53:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.621+0000 7fa555a2d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5500ca460 con 0x7fa55015a180 2023-08-11T15:53:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.622+0000 7fa54e7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa5500cb0c0 0x7fa5500708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34284/0 (socket says 172.21.15.27:34284) 2023-08-11T15:53:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.622+0000 7fa54e7fc640 1 -- 172.21.15.27:0/1362068296 learned_addr learned my addr 172.21.15.27:0/1362068296 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.622+0000 7fa54f7fe640 1 --2- 172.21.15.27:0/1362068296 >> v2:172.21.15.119:3300/0 conn(0x7fa55015a180 0x7fa5500cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.622+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066030343 0 0) 0x7fa544001110 con 0x7fa5500cb0c0 2023-08-11T15:53:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.622+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa538003520 con 0x7fa5500cb0c0 2023-08-11T15:53:06.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2142053139 0 0) 0x7fa538003520 con 0x7fa5500cb0c0 2023-08-11T15:53:06.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 >> v2:172.21.15.119:3300/0 conn(0x7fa55015a180 msgr2=0x7fa5500cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:06.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 --2- 172.21.15.27:0/1362068296 >> v2:172.21.15.119:3300/0 conn(0x7fa55015a180 0x7fa5500cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 >> v1:172.21.15.119:6789/0 conn(0x7fa550156e50 legacy=0x7fa5500ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5500cc550 con 0x7fa5500cb0c0 2023-08-11T15:53:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fa544001800 con 0x7fa5500cb0c0 2023-08-11T15:53:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 >> v1:172.21.15.27:6789/0 conn(0x7fa5500cb0c0 legacy=0x7fa5500708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa538003970 0x7fa538003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5380047c0 0x7fa538004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa538005180 0x7fa53800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5500ca460 con 0x7fa538003970 2023-08-11T15:53:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53800c020 con 0x7fa5380047c0 2023-08-11T15:53:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53800c1e0 con 0x7fa538005180 2023-08-11T15:53:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.623+0000 7fa54effd640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa538003970 0x7fa538003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.624+0000 7fa54e7fc640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5380047c0 0x7fa538004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.624+0000 7fa54f7fe640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa538005180 0x7fa53800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.624+0000 7fa54effd640 1 -- 172.21.15.27:0/1362068296 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa538005180 msgr2=0x7fa53800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.624+0000 7fa54effd640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa538005180 0x7fa53800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.624+0000 7fa54effd640 1 -- 172.21.15.27:0/1362068296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5380047c0 msgr2=0x7fa538004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:06.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.624+0000 7fa54effd640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5380047c0 0x7fa538004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.624+0000 7fa54effd640 1 -- 172.21.15.27:0/1362068296 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa538003520 con 0x7fa538003970 2023-08-11T15:53:06.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.624+0000 7fa54f7fe640 1 -- 172.21.15.27:0/1362068296 reap_dead start 2023-08-11T15:53:06.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.625+0000 7fa54effd640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa538003970 0x7fa538003d90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa540001e70 tx=0x7fa5380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a822f88e80f2c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:06.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.625+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540003bc0 con 0x7fa538003970 2023-08-11T15:53:06.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.625+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa540003d20 con 0x7fa538003970 2023-08-11T15:53:06.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.625+0000 7fa54dffb640 1 -- 172.21.15.27:0/1362068296 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540005310 con 0x7fa538003970 2023-08-11T15:53:06.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.626+0000 7fa555a2d640 1 -- 172.21.15.27:0/1362068296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa538003970 msgr2=0x7fa538003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.626+0000 7fa555a2d640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa538003970 0x7fa538003d90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa540001e70 tx=0x7fa5380057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.626+0000 7fa555a2d640 1 -- 172.21.15.27:0/1362068296 shutdown_connections 2023-08-11T15:53:06.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.626+0000 7fa555a2d640 1 --2- 172.21.15.27:0/1362068296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa538003970 0x7fa538003d90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.626+0000 7fa555a2d640 1 -- 172.21.15.27:0/1362068296 >> 172.21.15.27:0/1362068296 conn(0x7fa5500afb70 msgr2=0x7fa5500aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:06.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.626+0000 7fa555a2d640 1 -- 172.21.15.27:0/1362068296 shutdown_connections 2023-08-11T15:53:06.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.626+0000 7fa555a2d640 1 -- 172.21.15.27:0/1362068296 wait complete. 2023-08-11T15:53:06.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.628+0000 7fa555a2d640 1 Processor -- start 2023-08-11T15:53:06.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.628+0000 7fa555a2d640 1 -- start start 2023-08-11T15:53:06.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.629+0000 7fa555a2d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa538003970 0x7fa55014a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.629+0000 7fa555a2d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5500c3c60 0x7fa5500c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.629+0000 7fa555a2d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500c5100 0x7fa5500c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.629+0000 7fa555a2d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55014aa40 con 0x7fa5500c5100 2023-08-11T15:53:06.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.629+0000 7fa555a2d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5500c4620 con 0x7fa538003970 2023-08-11T15:53:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.629+0000 7fa555a2d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5501efe90 con 0x7fa5500c3c60 2023-08-11T15:53:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.629+0000 7fa54f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500c5100 0x7fa5500c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.630+0000 7fa54f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500c5100 0x7fa5500c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43580/0 (socket says 172.21.15.27:43580) 2023-08-11T15:53:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.630+0000 7fa54f7fe640 1 -- 172.21.15.27:0/203295129 learned_addr learned my addr 172.21.15.27:0/203295129 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.630+0000 7fa54effd640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa538003970 0x7fa55014a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.630+0000 7fa54e7fc640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5500c3c60 0x7fa5500c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.630+0000 7fa54f7fe640 1 -- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5500c3c60 msgr2=0x7fa5500c40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.630+0000 7fa54f7fe640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5500c3c60 0x7fa5500c40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.630+0000 7fa54f7fe640 1 -- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa538003970 msgr2=0x7fa55014a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.630+0000 7fa54f7fe640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa538003970 0x7fa55014a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.630+0000 7fa54f7fe640 1 -- 172.21.15.27:0/203295129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5500c5ac0 con 0x7fa5500c5100 2023-08-11T15:53:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.630+0000 7fa54f7fe640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500c5100 0x7fa5500c5580 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa5500cb8a0 tx=0x7fa548010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.631+0000 7fa537fff640 1 -- 172.21.15.27:0/203295129 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548014800 con 0x7fa5500c5100 2023-08-11T15:53:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.631+0000 7fa537fff640 1 -- 172.21.15.27:0/203295129 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa548014960 con 0x7fa5500c5100 2023-08-11T15:53:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.631+0000 7fa537fff640 1 -- 172.21.15.27:0/203295129 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548014d10 con 0x7fa5500c5100 2023-08-11T15:53:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.631+0000 7fa555a2d640 1 -- 172.21.15.27:0/203295129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5501ef110 con 0x7fa5500c5100 2023-08-11T15:53:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.631+0000 7fa555a2d640 1 -- 172.21.15.27:0/203295129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5501ef690 con 0x7fa5500c5100 2023-08-11T15:53:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.632+0000 7fa555a2d640 1 -- 172.21.15.27:0/203295129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5500c8100 con 0x7fa5500c5100 2023-08-11T15:53:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.633+0000 7fa537fff640 1 -- 172.21.15.27:0/203295129 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa548020410 con 0x7fa5500c5100 2023-08-11T15:53:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.634+0000 7fa537fff640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa53c06a080 0x7fa53c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.634+0000 7fa537fff640 1 -- 172.21.15.27:0/203295129 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fa548087b20 con 0x7fa5500c5100 2023-08-11T15:53:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.634+0000 7fa54effd640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa53c06a080 0x7fa53c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:06.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.636+0000 7fa54effd640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa53c06a080 0x7fa53c06c540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa540034000 tx=0x7fa540009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:06.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.638+0000 7fa537fff640 1 -- 172.21.15.27:0/203295129 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fa5500c8100 con 0x7fa5500c5100 2023-08-11T15:53:06.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:06.836+0000 7fa555a2d640 1 -- 172.21.15.27:0/203295129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa5500c8100 con 0x7fa5500c5100 2023-08-11T15:53:07.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.708+0000 7fa537fff640 1 -- 172.21.15.27:0/203295129 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa5500c8100 con 0x7fa5500c5100 2023-08-11T15:53:07.713 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-08-11T15:53:07.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.715+0000 7fa555a2d640 1 -- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa53c06a080 msgr2=0x7fa53c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:07.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.715+0000 7fa555a2d640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa53c06a080 0x7fa53c06c540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa540034000 tx=0x7fa540009030 comp rx=0 tx=0).stop 2023-08-11T15:53:07.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.715+0000 7fa555a2d640 1 -- 172.21.15.27:0/203295129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500c5100 msgr2=0x7fa5500c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:07.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.715+0000 7fa555a2d640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500c5100 0x7fa5500c5580 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa5500cb8a0 tx=0x7fa548010a10 comp rx=0 tx=0).stop 2023-08-11T15:53:07.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.719+0000 7fa555a2d640 1 -- 172.21.15.27:0/203295129 shutdown_connections 2023-08-11T15:53:07.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.719+0000 7fa555a2d640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa53c06a080 0x7fa53c06c540 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:07.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.719+0000 7fa555a2d640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5500c3c60 0x7fa5500c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:07.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.719+0000 7fa555a2d640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500c5100 0x7fa5500c5580 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:07.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.719+0000 7fa555a2d640 1 --2- 172.21.15.27:0/203295129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa538003970 0x7fa55014a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:07.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.720+0000 7fa555a2d640 1 -- 172.21.15.27:0/203295129 >> 172.21.15.27:0/203295129 conn(0x7fa5500afb70 msgr2=0x7fa550157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:07.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.720+0000 7fa555a2d640 1 -- 172.21.15.27:0/203295129 shutdown_connections 2023-08-11T15:53:07.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.720+0000 7fa555a2d640 1 -- 172.21.15.27:0/203295129 wait complete. 2023-08-11T15:53:07.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-11T15:53:07.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.973+0000 7f79f24bf640 1 Processor -- start 2023-08-11T15:53:07.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.973+0000 7f79f24bf640 1 -- start start 2023-08-11T15:53:07.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.975+0000 7f79f24bf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79ec0c8640 0x7f79ec0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:07.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.975+0000 7f79f24bf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79ec0c8a20 0x7f79ec1646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:07.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.975+0000 7f79f24bf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ec165460 con 0x7f79ec164c10 2023-08-11T15:53:07.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.975+0000 7f79f24bf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0c1980 con 0x7f79ec0c8640 2023-08-11T15:53:07.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.975+0000 7f79f24bf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0cc940 con 0x7f79ec0c8a20 2023-08-11T15:53:07.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.975+0000 7f79ebfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79ec0c8640 0x7f79ec0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:07.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.976+0000 7f79ebfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79ec0c8640 0x7f79ec0ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43590/0 (socket says 172.21.15.27:43590) 2023-08-11T15:53:07.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.976+0000 7f79ebfff640 1 -- 172.21.15.27:0/3576749371 learned_addr learned my addr 172.21.15.27:0/3576749371 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:07.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.976+0000 7f79eb7fe640 1 --2- 172.21.15.27:0/3576749371 >> v2:172.21.15.119:3300/0 conn(0x7f79ec0c8a20 0x7f79ec1646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:07.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.976+0000 7f79ebfff640 1 -- 172.21.15.27:0/3576749371 >> v2:172.21.15.119:3300/0 conn(0x7f79ec0c8a20 msgr2=0x7f79ec1646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:07.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.976+0000 7f79ebfff640 1 --2- 172.21.15.27:0/3576749371 >> v2:172.21.15.119:3300/0 conn(0x7f79ec0c8a20 0x7f79ec1646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:07.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.976+0000 7f79ebfff640 1 -- 172.21.15.27:0/3576749371 >> v1:172.21.15.119:6789/0 conn(0x7f79ec164c10 legacy=0x7f79ec06f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:07.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.976+0000 7f79ebfff640 1 -- 172.21.15.27:0/3576749371 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0cc010 con 0x7f79ec0c8640 2023-08-11T15:53:07.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.977+0000 7f79ebfff640 1 --2- 172.21.15.27:0/3576749371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec0c8640 0x7f79ec0ccdd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f79ec06d290 tx=0x7f79dc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44463bbe2ce0cb8d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:07.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.978+0000 7f79eaffd640 1 -- 172.21.15.27:0/3576749371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0317e0 con 0x7f79ec0c8640 2023-08-11T15:53:07.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.978+0000 7f79eaffd640 1 -- 172.21.15.27:0/3576749371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc031940 con 0x7f79ec0c8640 2023-08-11T15:53:07.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.978+0000 7f79eaffd640 1 -- 172.21.15.27:0/3576749371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc031cd0 con 0x7f79ec0c8640 2023-08-11T15:53:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.978+0000 7f79f24bf640 1 -- 172.21.15.27:0/3576749371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec0c8640 msgr2=0x7f79ec0ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.978+0000 7f79f24bf640 1 --2- 172.21.15.27:0/3576749371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec0c8640 0x7f79ec0ccdd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f79ec06d290 tx=0x7f79dc02b880 comp rx=0 tx=0).stop 2023-08-11T15:53:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.979+0000 7f79f24bf640 1 -- 172.21.15.27:0/3576749371 shutdown_connections 2023-08-11T15:53:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.979+0000 7f79f24bf640 1 --2- 172.21.15.27:0/3576749371 >> v2:172.21.15.119:3300/0 conn(0x7f79ec0c8a20 0x7f79ec1646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.979+0000 7f79f24bf640 1 --2- 172.21.15.27:0/3576749371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec0c8640 0x7f79ec0ccdd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.979+0000 7f79f24bf640 1 -- 172.21.15.27:0/3576749371 >> 172.21.15.27:0/3576749371 conn(0x7f79ec0afbe0 msgr2=0x7f79ec0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.979+0000 7f79f24bf640 1 -- 172.21.15.27:0/3576749371 shutdown_connections 2023-08-11T15:53:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.979+0000 7f79f24bf640 1 -- 172.21.15.27:0/3576749371 wait complete. 2023-08-11T15:53:07.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.981+0000 7f79f24bf640 1 Processor -- start 2023-08-11T15:53:07.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.981+0000 7f79f24bf640 1 -- start start 2023-08-11T15:53:07.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.982+0000 7f79f24bf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79ec0c8640 0x7f79ec1f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:07.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.982+0000 7f79f24bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0c8a20 0x7f79ec0c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:07.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.982+0000 7f79f24bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec164c10 0x7f79ec0c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:07.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.982+0000 7f79f24bf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1f67f0 con 0x7f79ec164c10 2023-08-11T15:53:07.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.982+0000 7f79f24bf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1f5ee0 con 0x7f79ec0c8a20 2023-08-11T15:53:07.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.982+0000 7f79f24bf640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0c3a10 con 0x7f79ec0c8640 2023-08-11T15:53:07.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.982+0000 7f79f0a5c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec164c10 0x7f79ec0c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:07.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.982+0000 7f79f0a5c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec164c10 0x7f79ec0c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43598/0 (socket says 172.21.15.27:43598) 2023-08-11T15:53:07.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.983+0000 7f79f0a5c640 1 -- 172.21.15.27:0/3331300339 learned_addr learned my addr 172.21.15.27:0/3331300339 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:07.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.983+0000 7f79ebfff640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79ec0c8640 0x7f79ec1f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:07.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.983+0000 7f79eb7fe640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0c8a20 0x7f79ec0c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:07.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.983+0000 7f79f0a5c640 1 -- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79ec0c8640 msgr2=0x7f79ec1f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:07.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.983+0000 7f79f0a5c640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79ec0c8640 0x7f79ec1f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:07.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.983+0000 7f79f0a5c640 1 -- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0c8a20 msgr2=0x7f79ec0c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:07.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.983+0000 7f79f0a5c640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0c8a20 0x7f79ec0c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:07.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.983+0000 7f79f0a5c640 1 -- 172.21.15.27:0/3331300339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc0094d0 con 0x7f79ec164c10 2023-08-11T15:53:07.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.983+0000 7f79f0a5c640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec164c10 0x7f79ec0c3e00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f79e400c780 tx=0x7f79e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:07.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.984+0000 7f79e8ff9640 1 -- 172.21.15.27:0/3331300339 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4004b60 con 0x7f79ec164c10 2023-08-11T15:53:07.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.984+0000 7f79e8ff9640 1 -- 172.21.15.27:0/3331300339 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e400ce90 con 0x7f79ec164c10 2023-08-11T15:53:07.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.984+0000 7f79f24bf640 1 -- 172.21.15.27:0/3331300339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec1f61a0 con 0x7f79ec164c10 2023-08-11T15:53:07.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.984+0000 7f79e8ff9640 1 -- 172.21.15.27:0/3331300339 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400a850 con 0x7f79ec164c10 2023-08-11T15:53:07.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.984+0000 7f79f24bf640 1 -- 172.21.15.27:0/3331300339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec15faa0 con 0x7f79ec164c10 2023-08-11T15:53:07.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.985+0000 7f79f24bf640 1 -- 172.21.15.27:0/3331300339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec1f6960 con 0x7f79ec164c10 2023-08-11T15:53:07.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.986+0000 7f79e8ff9640 1 -- 172.21.15.27:0/3331300339 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f79e400a9b0 con 0x7f79ec164c10 2023-08-11T15:53:07.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.987+0000 7f79e8ff9640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79d806a080 0x7f79d806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:07.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.987+0000 7f79e8ff9640 1 -- 172.21.15.27:0/3331300339 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f79e4088560 con 0x7f79ec164c10 2023-08-11T15:53:07.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.988+0000 7f79ebfff640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79d806a080 0x7f79d806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:07.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.989+0000 7f79ebfff640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79d806a080 0x7f79d806c540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f79dc036000 tx=0x7f79dc03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:07.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:07.992+0000 7f79e8ff9640 1 -- 172.21.15.27:0/3331300339 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f79e4053d00 con 0x7f79ec164c10 2023-08-11T15:53:08.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.188+0000 7f79f24bf640 1 -- 172.21.15.27:0/3331300339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f79ec04e420 con 0x7f79ec164c10 2023-08-11T15:53:08.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.715+0000 7f79e8ff9640 1 -- 172.21.15.27:0/3331300339 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) v1 ==== 170+0+0 (secure 0 0 0) 0x7f79e4057320 con 0x7f79ec164c10 2023-08-11T15:53:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-08-11T15:53:08.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.725+0000 7f79f24bf640 1 -- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79d806a080 msgr2=0x7f79d806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:08.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.725+0000 7f79f24bf640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79d806a080 0x7f79d806c540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f79dc036000 tx=0x7f79dc03d040 comp rx=0 tx=0).stop 2023-08-11T15:53:08.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.725+0000 7f79f24bf640 1 -- 172.21.15.27:0/3331300339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec164c10 msgr2=0x7f79ec0c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:08.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.725+0000 7f79f24bf640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec164c10 0x7f79ec0c3e00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f79e400c780 tx=0x7f79e400cc40 comp rx=0 tx=0).stop 2023-08-11T15:53:08.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.726+0000 7f79f24bf640 1 -- 172.21.15.27:0/3331300339 shutdown_connections 2023-08-11T15:53:08.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.727+0000 7f79f24bf640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f79d806a080 0x7f79d806c540 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:08.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.727+0000 7f79f24bf640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79ec164c10 0x7f79ec0c3e00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:08.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.727+0000 7f79f24bf640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79ec0c8a20 0x7f79ec0c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:08.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.727+0000 7f79f24bf640 1 --2- 172.21.15.27:0/3331300339 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79ec0c8640 0x7f79ec1f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:08.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.727+0000 7f79f24bf640 1 -- 172.21.15.27:0/3331300339 >> 172.21.15.27:0/3331300339 conn(0x7f79ec0afbe0 msgr2=0x7f79ec0722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:08.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.730+0000 7f79f24bf640 1 -- 172.21.15.27:0/3331300339 shutdown_connections 2023-08-11T15:53:08.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.731+0000 7f79f24bf640 1 -- 172.21.15.27:0/3331300339 wait complete. 2023-08-11T15:53:08.761 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-11T15:53:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.963+0000 7fcbf7ed9640 1 Processor -- start 2023-08-11T15:53:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.963+0000 7fcbf7ed9640 1 -- start start 2023-08-11T15:53:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.964+0000 7fcbf7ed9640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0161210 0x7fcbf00d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.964+0000 7fcbf7ed9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf00af250 con 0x7fcbf00b3090 2023-08-11T15:53:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.964+0000 7fcbf7ed9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf00af510 con 0x7fcbf00b38f0 2023-08-11T15:53:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.964+0000 7fcbf7ed9640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00b0190 con 0x7fcbf0161210 2023-08-11T15:53:08.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.965+0000 7fcbf5474640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcbf00b3090 0x7fcbf00b05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34292/0 (socket says 172.21.15.27:34292) 2023-08-11T15:53:08.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.965+0000 7fcbf5474640 1 -- 172.21.15.27:0/4237274749 learned_addr learned my addr 172.21.15.27:0/4237274749 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:08.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.965+0000 7fcbf6476640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0161210 0x7fcbf00d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:08.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.965+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875245987 0 0) 0x7fcbf00af250 con 0x7fcbf00b3090 2023-08-11T15:53:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.965+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbdc003520 con 0x7fcbf00b3090 2023-08-11T15:53:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3419781238 0 0) 0x7fcbdc003520 con 0x7fcbf00b3090 2023-08-11T15:53:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0161210 msgr2=0x7fcbf00d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0161210 0x7fcbf00d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 >> v1:172.21.15.119:6789/0 conn(0x7fcbf00b38f0 legacy=0x7fcbf0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00af7f0 con 0x7fcbf00b3090 2023-08-11T15:53:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fcbe0001670 con 0x7fcbf00b3090 2023-08-11T15:53:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 >> v1:172.21.15.27:6789/0 conn(0x7fcbf00b3090 legacy=0x7fcbf00b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbdc0039a0 0x7fcbdc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbdc0047c0 0x7fcbdc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0161210 0x7fcbdc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00b0190 con 0x7fcbdc0047c0 2023-08-11T15:53:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc00ba30 con 0x7fcbdc0039a0 2023-08-11T15:53:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc00bbf0 con 0x7fcbf0161210 2023-08-11T15:53:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf5474640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbdc0047c0 0x7fcbdc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.966+0000 7fcbf5c75640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbdc0039a0 0x7fcbdc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.967+0000 7fcbf6476640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0161210 0x7fcbdc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.967+0000 7fcbf5474640 1 -- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0161210 msgr2=0x7fcbdc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.967+0000 7fcbf5474640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0161210 0x7fcbdc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.967+0000 7fcbf5474640 1 -- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbdc0039a0 msgr2=0x7fcbdc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.967+0000 7fcbf5474640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbdc0039a0 0x7fcbdc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.967+0000 7fcbf5474640 1 -- 172.21.15.27:0/4237274749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc003520 con 0x7fcbdc0047c0 2023-08-11T15:53:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.967+0000 7fcbf6476640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0161210 0x7fcbdc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:08.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.968+0000 7fcbf5474640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbdc0047c0 0x7fcbdc004c40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c210 tx=0x7fcbdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63ed2a673f77378 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:08.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.968+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0005ad0 con 0x7fcbdc0047c0 2023-08-11T15:53:08.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.968+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe0005c30 con 0x7fcbdc0047c0 2023-08-11T15:53:08.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.968+0000 7fcbf4c73640 1 -- 172.21.15.27:0/4237274749 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe002c870 con 0x7fcbdc0047c0 2023-08-11T15:53:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.969+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/4237274749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbdc0047c0 msgr2=0x7fcbdc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.969+0000 7fcbf7ed9640 1 --2- 172.21.15.27:0/4237274749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbdc0047c0 0x7fcbdc004c40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c210 tx=0x7fcbdc0057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.969+0000 7fcbf6476640 1 -- 172.21.15.27:0/4237274749 reap_dead start 2023-08-11T15:53:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.969+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/4237274749 shutdown_connections 2023-08-11T15:53:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.969+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/4237274749 >> 172.21.15.27:0/4237274749 conn(0x7fcbf009b960 msgr2=0x7fcbf00d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.969+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/4237274749 shutdown_connections 2023-08-11T15:53:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.969+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/4237274749 wait complete. 2023-08-11T15:53:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.971+0000 7fcbf7ed9640 1 Processor -- start 2023-08-11T15:53:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.971+0000 7fcbf7ed9640 1 -- start start 2023-08-11T15:53:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf7ed9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbdc0047c0 0x7fcbf00c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf7ed9640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0157ea0 0x7fcbf00cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf7ed9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbf0158bd0 0x7fcbf0159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf7ed9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00ce200 con 0x7fcbf0158bd0 2023-08-11T15:53:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf7ed9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00c9e70 con 0x7fcbdc0047c0 2023-08-11T15:53:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf7ed9640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00ca150 con 0x7fcbf0157ea0 2023-08-11T15:53:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf6476640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbf0158bd0 0x7fcbf0159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf6476640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbf0158bd0 0x7fcbf0159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43606/0 (socket says 172.21.15.27:43606) 2023-08-11T15:53:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf6476640 1 -- 172.21.15.27:0/1364435967 learned_addr learned my addr 172.21.15.27:0/1364435967 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf5474640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0157ea0 0x7fcbf00cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.972+0000 7fcbf5c75640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbdc0047c0 0x7fcbf00c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.973+0000 7fcbf6476640 1 -- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0157ea0 msgr2=0x7fcbf00cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.973+0000 7fcbf6476640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0157ea0 0x7fcbf00cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.973+0000 7fcbf6476640 1 -- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbdc0047c0 msgr2=0x7fcbf00c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.973+0000 7fcbf6476640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbdc0047c0 0x7fcbf00c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.973+0000 7fcbf6476640 1 -- 172.21.15.27:0/1364435967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00ca430 con 0x7fcbf0158bd0 2023-08-11T15:53:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.973+0000 7fcbf6476640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbf0158bd0 0x7fcbf0159050 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcbf008c710 tx=0x7fcbe800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.973+0000 7fcbe67fc640 1 -- 172.21.15.27:0/1364435967 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8017400 con 0x7fcbf0158bd0 2023-08-11T15:53:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.973+0000 7fcbe67fc640 1 -- 172.21.15.27:0/1364435967 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8011040 con 0x7fcbf0158bd0 2023-08-11T15:53:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.973+0000 7fcbe67fc640 1 -- 172.21.15.27:0/1364435967 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8017780 con 0x7fcbf0158bd0 2023-08-11T15:53:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.974+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/1364435967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf00ca710 con 0x7fcbf0158bd0 2023-08-11T15:53:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.974+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/1364435967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf0158300 con 0x7fcbf0158bd0 2023-08-11T15:53:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.975+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/1364435967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf00cac30 con 0x7fcbf0158bd0 2023-08-11T15:53:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.976+0000 7fcbe67fc640 1 -- 172.21.15.27:0/1364435967 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcbe80203b0 con 0x7fcbf0158bd0 2023-08-11T15:53:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.977+0000 7fcbe67fc640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcbd406a080 0x7fcbd406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.977+0000 7fcbe67fc640 1 -- 172.21.15.27:0/1364435967 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fcbe8088a40 con 0x7fcbf0158bd0 2023-08-11T15:53:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.978+0000 7fcbf5c75640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcbd406a080 0x7fcbd406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.979+0000 7fcbf5c75640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcbd406a080 0x7fcbd406c540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcbec001680 tx=0x7fcbec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:08.982+0000 7fcbe67fc640 1 -- 172.21.15.27:0/1364435967 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fcbf00cac30 con 0x7fcbf0158bd0 2023-08-11T15:53:09.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.217+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/1364435967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcbf00cac30 con 0x7fcbf0158bd0 2023-08-11T15:53:09.784 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 6 and data pool 7 2023-08-11T15:53:09.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.781+0000 7fcbe67fc640 1 -- 172.21.15.27:0/1364435967 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fcbf00cac30 con 0x7fcbf0158bd0 2023-08-11T15:53:09.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.787+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcbd406a080 msgr2=0x7fcbd406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:09.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.788+0000 7fcbf7ed9640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcbd406a080 0x7fcbd406c540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcbec001680 tx=0x7fcbec009040 comp rx=0 tx=0).stop 2023-08-11T15:53:09.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.788+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/1364435967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbf0158bd0 msgr2=0x7fcbf0159050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:09.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.788+0000 7fcbf7ed9640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbf0158bd0 0x7fcbf0159050 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcbf008c710 tx=0x7fcbe800f4a0 comp rx=0 tx=0).stop 2023-08-11T15:53:09.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.789+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/1364435967 shutdown_connections 2023-08-11T15:53:09.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.789+0000 7fcbf7ed9640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcbd406a080 0x7fcbd406c540 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:09.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.789+0000 7fcbf7ed9640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbf0157ea0 0x7fcbf00cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:09.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.789+0000 7fcbf7ed9640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbf0158bd0 0x7fcbf0159050 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:09.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.789+0000 7fcbf7ed9640 1 --2- 172.21.15.27:0/1364435967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbdc0047c0 0x7fcbf00c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:09.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.789+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/1364435967 >> 172.21.15.27:0/1364435967 conn(0x7fcbf009b960 msgr2=0x7fcbf015e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:09.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.789+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/1364435967 shutdown_connections 2023-08-11T15:53:09.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:09.789+0000 7fcbf7ed9640 1 -- 172.21.15.27:0/1364435967 wait complete. 2023-08-11T15:53:09.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:53:10.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.009+0000 7f9d83c1f640 1 Processor -- start 2023-08-11T15:53:10.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.010+0000 7f9d83c1f640 1 -- start start 2023-08-11T15:53:10.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.011+0000 7f9d83c1f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9d7c15c620 0x7f9d7c0b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.011+0000 7f9d83c1f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d7c0b82f0 0x7f9d7c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.011+0000 7f9d83c1f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7c0cda30 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.011+0000 7f9d83c1f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7c0ccc10 con 0x7f9d7c15c620 2023-08-11T15:53:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.011+0000 7f9d83c1f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7c161110 con 0x7f9d7c0b82f0 2023-08-11T15:53:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.011+0000 7f9d819bb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9d7c15c620 0x7f9d7c0b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.011+0000 7f9d819bb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9d7c15c620 0x7f9d7c0b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40172/0 (socket says 172.21.15.27:40172) 2023-08-11T15:53:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.011+0000 7f9d819bb640 1 -- 172.21.15.27:0/222075564 learned_addr learned my addr 172.21.15.27:0/222075564 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.012+0000 7f9d819bb640 1 -- 172.21.15.27:0/222075564 >> v2:172.21.15.119:3300/0 conn(0x7f9d7c0b82f0 msgr2=0x7f9d7c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.012+0000 7f9d819bb640 1 --2- 172.21.15.27:0/222075564 >> v2:172.21.15.119:3300/0 conn(0x7f9d7c0b82f0 0x7f9d7c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.012+0000 7f9d819bb640 1 -- 172.21.15.27:0/222075564 >> v1:172.21.15.119:6789/0 conn(0x7f9d7c0b55c0 legacy=0x7f9d7c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.012+0000 7f9d819bb640 1 -- 172.21.15.27:0/222075564 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c0cce20 con 0x7f9d7c15c620 2023-08-11T15:53:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.013+0000 7f9d819bb640 1 --2- 172.21.15.27:0/222075564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c15c620 0x7f9d7c0b7db0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0b50e0 tx=0x7f9d7802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43744e29839f2049 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:10.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.013+0000 7f9d809b9640 1 -- 172.21.15.27:0/222075564 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7802ca80 con 0x7f9d7c15c620 2023-08-11T15:53:10.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.013+0000 7f9d809b9640 1 -- 172.21.15.27:0/222075564 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7802cbe0 con 0x7f9d7c15c620 2023-08-11T15:53:10.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.013+0000 7f9d809b9640 1 -- 172.21.15.27:0/222075564 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d780343f0 con 0x7f9d7c15c620 2023-08-11T15:53:10.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.014+0000 7f9d83c1f640 1 -- 172.21.15.27:0/222075564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c15c620 msgr2=0x7f9d7c0b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.014+0000 7f9d83c1f640 1 --2- 172.21.15.27:0/222075564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c15c620 0x7f9d7c0b7db0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0b50e0 tx=0x7f9d7802bee0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.014+0000 7f9d83c1f640 1 -- 172.21.15.27:0/222075564 shutdown_connections 2023-08-11T15:53:10.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.014+0000 7f9d83c1f640 1 --2- 172.21.15.27:0/222075564 >> v2:172.21.15.119:3300/0 conn(0x7f9d7c0b82f0 0x7f9d7c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.014+0000 7f9d83c1f640 1 --2- 172.21.15.27:0/222075564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c15c620 0x7f9d7c0b7db0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.014+0000 7f9d83c1f640 1 -- 172.21.15.27:0/222075564 >> 172.21.15.27:0/222075564 conn(0x7f9d7c071110 msgr2=0x7f9d7c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:10.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.014+0000 7f9d83c1f640 1 -- 172.21.15.27:0/222075564 shutdown_connections 2023-08-11T15:53:10.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.014+0000 7f9d83c1f640 1 -- 172.21.15.27:0/222075564 wait complete. 2023-08-11T15:53:10.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.016+0000 7f9d83c1f640 1 Processor -- start 2023-08-11T15:53:10.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.016+0000 7f9d83c1f640 1 -- start start 2023-08-11T15:53:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.017+0000 7f9d83c1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c0b55c0 0x7f9d7c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.017+0000 7f9d83c1f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d7c0b82f0 0x7f9d7c0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.017+0000 7f9d83c1f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7c15c620 0x7f9d7c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.017+0000 7f9d83c1f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c15f830 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.017+0000 7f9d83c1f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c15fb10 con 0x7f9d7c15c620 2023-08-11T15:53:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.017+0000 7f9d83c1f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0c9730 con 0x7f9d7c0b82f0 2023-08-11T15:53:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.017+0000 7f9d819bb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c0b55c0 0x7f9d7c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.017+0000 7f9d819bb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c0b55c0 0x7f9d7c150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40178/0 (socket says 172.21.15.27:40178) 2023-08-11T15:53:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.017+0000 7f9d819bb640 1 -- 172.21.15.27:0/3241388659 learned_addr learned my addr 172.21.15.27:0/3241388659 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.018+0000 7f9d819bb640 1 -- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d7c0b82f0 msgr2=0x7f9d7c0b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.018+0000 7f9d819bb640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d7c0b82f0 0x7f9d7c0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.018+0000 7f9d819bb640 1 -- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7c15c620 msgr2=0x7f9d7c0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.018+0000 7f9d819bb640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7c15c620 0x7f9d7c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.018+0000 7f9d819bb640 1 -- 172.21.15.27:0/3241388659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d78009b30 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.018+0000 7f9d819bb640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c0b55c0 0x7f9d7c150ab0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9d78036040 tx=0x7f9d78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.018+0000 7f9d727fc640 1 -- 172.21.15.27:0/3241388659 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78009510 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.018+0000 7f9d727fc640 1 -- 172.21.15.27:0/3241388659 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7803e450 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.019+0000 7f9d727fc640 1 -- 172.21.15.27:0/3241388659 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7803e800 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.019+0000 7f9d83c1f640 1 -- 172.21.15.27:0/3241388659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c0c9a10 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.019+0000 7f9d83c1f640 1 -- 172.21.15.27:0/3241388659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c150ff0 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.020+0000 7f9d83c1f640 1 -- 172.21.15.27:0/3241388659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c073990 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.021+0000 7f9d727fc640 1 -- 172.21.15.27:0/3241388659 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9d7802cd50 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.022+0000 7f9d727fc640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9d6006a050 0x7f9d6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.022+0000 7f9d727fc640 1 -- 172.21.15.27:0/3241388659 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9d780ab400 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.023+0000 7f9d811ba640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9d6006a050 0x7f9d6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.024+0000 7f9d811ba640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9d6006a050 0x7f9d6006c510 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0b4ff0 tx=0x7f9d6c006c90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.026+0000 7f9d727fc640 1 -- 172.21.15.27:0/3241388659 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9d78076ae0 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.224+0000 7f9d83c1f640 1 -- 172.21.15.27:0/3241388659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d7c06b6f0 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.226+0000 7f9d727fc640 1 -- 172.21.15.27:0/3241388659 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18513 (secure 0 0 0) 0x7f9d7807a100 con 0x7f9d7c0b55c0 2023-08-11T15:53:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:10.229 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":42,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:53:09.220031+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:53:06.898951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-11T15:53:08.258996+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:53:10.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 -- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9d6006a050 msgr2=0x7f9d6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9d6006a050 0x7f9d6006c510 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0b4ff0 tx=0x7f9d6c006c90 comp rx=0 tx=0).stop 2023-08-11T15:53:10.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 -- 172.21.15.27:0/3241388659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c0b55c0 msgr2=0x7f9d7c150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c0b55c0 0x7f9d7c150ab0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9d78036040 tx=0x7f9d78004a40 comp rx=0 tx=0).stop 2023-08-11T15:53:10.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 -- 172.21.15.27:0/3241388659 shutdown_connections 2023-08-11T15:53:10.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f9d6006a050 0x7f9d6006c510 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d7c0b82f0 0x7f9d7c0b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d7c15c620 0x7f9d7c0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 --2- 172.21.15.27:0/3241388659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d7c0b55c0 0x7f9d7c150ab0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 -- 172.21.15.27:0/3241388659 >> 172.21.15.27:0/3241388659 conn(0x7f9d7c071110 msgr2=0x7f9d7c0d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:10.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 -- 172.21.15.27:0/3241388659 shutdown_connections 2023-08-11T15:53:10.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.229+0000 7f9d83c1f640 1 -- 172.21.15.27:0/3241388659 wait complete. 2023-08-11T15:53:10.249 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:53:10.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.486+0000 7fd99a512640 1 Processor -- start 2023-08-11T15:53:10.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.486+0000 7fd99a512640 1 -- start start 2023-08-11T15:53:10.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.488+0000 7fd99a512640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd99415a140 0x7fd99406b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.488+0000 7fd99a512640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9940cb430 0x7fd9940b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.488+0000 7fd99a512640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9940c8a30 con 0x7fd99406f5d0 2023-08-11T15:53:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.488+0000 7fd99a512640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99406c0c0 con 0x7fd99415a140 2023-08-11T15:53:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.488+0000 7fd99a512640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99406c3a0 con 0x7fd9940cb430 2023-08-11T15:53:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.488+0000 7fd993fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd99415a140 0x7fd99406b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.488+0000 7fd998aaf640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd99406f5d0 0x7fd9940c8ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48992/0 (socket says 172.21.15.27:48992) 2023-08-11T15:53:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.488+0000 7fd998aaf640 1 -- 172.21.15.27:0/1246593947 learned_addr learned my addr 172.21.15.27:0/1246593947 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.489+0000 7fd9937fe640 1 --2- 172.21.15.27:0/1246593947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9940cb430 0x7fd9940b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.489+0000 7fd992ffd640 1 -- 172.21.15.27:0/1246593947 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809505504 0 0) 0x7fd9940c8a30 con 0x7fd99406f5d0 2023-08-11T15:53:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.489+0000 7fd992ffd640 1 -- 172.21.15.27:0/1246593947 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd97c003520 con 0x7fd99406f5d0 2023-08-11T15:53:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.489+0000 7fd993fff640 1 -- 172.21.15.27:0/1246593947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9940cb430 msgr2=0x7fd9940b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:10.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.489+0000 7fd993fff640 1 --2- 172.21.15.27:0/1246593947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9940cb430 0x7fd9940b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.489+0000 7fd993fff640 1 -- 172.21.15.27:0/1246593947 >> v1:172.21.15.27:6789/0 conn(0x7fd99406f5d0 legacy=0x7fd9940c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.489+0000 7fd993fff640 1 -- 172.21.15.27:0/1246593947 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99406c680 con 0x7fd99415a140 2023-08-11T15:53:10.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.490+0000 7fd993fff640 1 --2- 172.21.15.27:0/1246593947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd99415a140 0x7fd99406b870 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd984009790 tx=0x7fd98402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60419b52657386ae server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:10.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd992ffd640 1 -- 172.21.15.27:0/1246593947 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984031800 con 0x7fd99415a140 2023-08-11T15:53:10.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd992ffd640 1 -- 172.21.15.27:0/1246593947 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd984031960 con 0x7fd99415a140 2023-08-11T15:53:10.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd992ffd640 1 -- 172.21.15.27:0/1246593947 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984031cd0 con 0x7fd99415a140 2023-08-11T15:53:10.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd99a512640 1 -- 172.21.15.27:0/1246593947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd99415a140 msgr2=0x7fd99406b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd99a512640 1 --2- 172.21.15.27:0/1246593947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd99415a140 0x7fd99406b870 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd984009790 tx=0x7fd98402b8a0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd99a512640 1 -- 172.21.15.27:0/1246593947 shutdown_connections 2023-08-11T15:53:10.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd99a512640 1 --2- 172.21.15.27:0/1246593947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9940cb430 0x7fd9940b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd99a512640 1 --2- 172.21.15.27:0/1246593947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd99415a140 0x7fd99406b870 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd99a512640 1 -- 172.21.15.27:0/1246593947 >> 172.21.15.27:0/1246593947 conn(0x7fd9940afb70 msgr2=0x7fd9940aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:10.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd99a512640 1 -- 172.21.15.27:0/1246593947 shutdown_connections 2023-08-11T15:53:10.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.491+0000 7fd99a512640 1 -- 172.21.15.27:0/1246593947 wait complete. 2023-08-11T15:53:10.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.493+0000 7fd99a512640 1 Processor -- start 2023-08-11T15:53:10.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.494+0000 7fd99a512640 1 -- start start 2023-08-11T15:53:10.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.494+0000 7fd99a512640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd99406f5d0 0x7fd99415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd99a512640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9940cb430 0x7fd9940ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd99a512640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd99415a140 0x7fd99414b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd99a512640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd994148c90 con 0x7fd9940cb430 2023-08-11T15:53:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd99a512640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940cd530 con 0x7fd99406f5d0 2023-08-11T15:53:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd99a512640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99414bf00 con 0x7fd99415a140 2023-08-11T15:53:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9940cb430 0x7fd9940ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9940cb430 0x7fd9940ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40184/0 (socket says 172.21.15.27:40184) 2023-08-11T15:53:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd9937fe640 1 -- 172.21.15.27:0/983767702 learned_addr learned my addr 172.21.15.27:0/983767702 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:10.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd993fff640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd99406f5d0 0x7fd99415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd998aaf640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd99415a140 0x7fd99414b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd9937fe640 1 -- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd99415a140 msgr2=0x7fd99414b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd9937fe640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd99415a140 0x7fd99414b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd9937fe640 1 -- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd99406f5d0 msgr2=0x7fd99415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd9937fe640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd99406f5d0 0x7fd99415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd9937fe640 1 -- 172.21.15.27:0/983767702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9840093a0 con 0x7fd9940cb430 2023-08-11T15:53:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.495+0000 7fd998aaf640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd99415a140 0x7fd99414b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.496+0000 7fd9937fe640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9940cb430 0x7fd9940ccff0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd98800d800 tx=0x7fd98800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.496+0000 7fd990ff9640 1 -- 172.21.15.27:0/983767702 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98800b6c0 con 0x7fd9940cb430 2023-08-11T15:53:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.496+0000 7fd990ff9640 1 -- 172.21.15.27:0/983767702 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98800b820 con 0x7fd9940cb430 2023-08-11T15:53:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.496+0000 7fd990ff9640 1 -- 172.21.15.27:0/983767702 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98800bbb0 con 0x7fd9940cb430 2023-08-11T15:53:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.496+0000 7fd99a512640 1 -- 172.21.15.27:0/983767702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9940cbf50 con 0x7fd9940cb430 2023-08-11T15:53:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.496+0000 7fd99a512640 1 -- 172.21.15.27:0/983767702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9940cc510 con 0x7fd9940cb430 2023-08-11T15:53:10.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.497+0000 7fd99a512640 1 -- 172.21.15.27:0/983767702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd994073990 con 0x7fd9940cb430 2023-08-11T15:53:10.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.498+0000 7fd990ff9640 1 -- 172.21.15.27:0/983767702 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd98800bd10 con 0x7fd9940cb430 2023-08-11T15:53:10.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.499+0000 7fd990ff9640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd98006a050 0x7fd98006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.499+0000 7fd990ff9640 1 -- 172.21.15.27:0/983767702 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd988088d00 con 0x7fd9940cb430 2023-08-11T15:53:10.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.500+0000 7fd993fff640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd98006a050 0x7fd98006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.501+0000 7fd993fff640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd98006a050 0x7fd98006c510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd984036000 tx=0x7fd984008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:10.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.503+0000 7fd990ff9640 1 -- 172.21.15.27:0/983767702 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fd988054210 con 0x7fd9940cb430 2023-08-11T15:53:10.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.707+0000 7fd99a512640 1 -- 172.21.15.27:0/983767702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9940c75d0 con 0x7fd9940cb430 2023-08-11T15:53:10.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.710+0000 7fd990ff9640 1 -- 172.21.15.27:0/983767702 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18513 (secure 0 0 0) 0x7fd988057830 con 0x7fd9940cb430 2023-08-11T15:53:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:10.713 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":42,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:53:09.220031+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:53:06.898951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-11T15:53:08.258996+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:53:10.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 -- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd98006a050 msgr2=0x7fd98006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd98006a050 0x7fd98006c510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd984036000 tx=0x7fd984008030 comp rx=0 tx=0).stop 2023-08-11T15:53:10.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 -- 172.21.15.27:0/983767702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9940cb430 msgr2=0x7fd9940ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9940cb430 0x7fd9940ccff0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd98800d800 tx=0x7fd98800dcc0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 -- 172.21.15.27:0/983767702 shutdown_connections 2023-08-11T15:53:10.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd98006a050 0x7fd98006c510 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd99415a140 0x7fd99414b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9940cb430 0x7fd9940ccff0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 --2- 172.21.15.27:0/983767702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd99406f5d0 0x7fd99415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 -- 172.21.15.27:0/983767702 >> 172.21.15.27:0/983767702 conn(0x7fd9940afb70 msgr2=0x7fd9940caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 -- 172.21.15.27:0/983767702 shutdown_connections 2023-08-11T15:53:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.713+0000 7fd99a512640 1 -- 172.21.15.27:0/983767702 wait complete. 2023-08-11T15:53:10.733 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-11T15:53:10.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.959+0000 7f83ad10f640 1 Processor -- start 2023-08-11T15:53:10.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.959+0000 7f83ad10f640 1 -- start start 2023-08-11T15:53:10.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.961+0000 7f83ad10f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83a806b670 0x7f83a80d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.961+0000 7f83ad10f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 0x7f83a80af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.961+0000 7f83ad10f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a806bc10 con 0x7f83a815c760 2023-08-11T15:53:10.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.961+0000 7f83ad10f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a806c790 con 0x7f83a806b670 2023-08-11T15:53:10.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.961+0000 7f83ad10f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80d13f0 con 0x7f83a80d2850 2023-08-11T15:53:10.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.961+0000 7f83a6d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f83a815c760 0x7f83a80d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49004/0 (socket says 172.21.15.27:49004) 2023-08-11T15:53:10.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.961+0000 7f83a6d9d640 1 -- 172.21.15.27:0/4135619587 learned_addr learned my addr 172.21.15.27:0/4135619587 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:10.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.961+0000 7f83a659c640 1 --2- 172.21.15.27:0/4135619587 >> v2:172.21.15.27:3300/0 conn(0x7f83a806b670 0x7f83a80d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.961+0000 7f83a759e640 1 --2- 172.21.15.27:0/4135619587 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 0x7f83a80af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.962+0000 7f83a659c640 1 -- 172.21.15.27:0/4135619587 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 msgr2=0x7f83a80af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:10.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.962+0000 7f83a659c640 1 --2- 172.21.15.27:0/4135619587 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 0x7f83a80af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.962+0000 7f83a659c640 1 -- 172.21.15.27:0/4135619587 >> v1:172.21.15.27:6789/0 conn(0x7f83a815c760 legacy=0x7f83a80d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.962+0000 7f83a659c640 1 -- 172.21.15.27:0/4135619587 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a806bef0 con 0x7f83a806b670 2023-08-11T15:53:10.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.963+0000 7f83a659c640 1 --2- 172.21.15.27:0/4135619587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a806b670 0x7f83a80d2310 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f83900097a0 tx=0x7f839002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f071b339afbe7b3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:10.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.963+0000 7f83a5d9b640 1 -- 172.21.15.27:0/4135619587 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839002d040 con 0x7f83a806b670 2023-08-11T15:53:10.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.963+0000 7f83a5d9b640 1 -- 172.21.15.27:0/4135619587 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8390031810 con 0x7f83a806b670 2023-08-11T15:53:10.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.963+0000 7f83a5d9b640 1 -- 172.21.15.27:0/4135619587 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390031ba0 con 0x7f83a806b670 2023-08-11T15:53:10.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.964+0000 7f83ad10f640 1 -- 172.21.15.27:0/4135619587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a806b670 msgr2=0x7f83a80d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.964+0000 7f83ad10f640 1 --2- 172.21.15.27:0/4135619587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a806b670 0x7f83a80d2310 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f83900097a0 tx=0x7f839002b8b0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.964+0000 7f83ad10f640 1 -- 172.21.15.27:0/4135619587 shutdown_connections 2023-08-11T15:53:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.964+0000 7f83ad10f640 1 --2- 172.21.15.27:0/4135619587 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 0x7f83a80af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.964+0000 7f83ad10f640 1 --2- 172.21.15.27:0/4135619587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a806b670 0x7f83a80d2310 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.964+0000 7f83ad10f640 1 -- 172.21.15.27:0/4135619587 >> 172.21.15.27:0/4135619587 conn(0x7f83a8089ff0 msgr2=0x7f83a809b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.964+0000 7f83ad10f640 1 -- 172.21.15.27:0/4135619587 shutdown_connections 2023-08-11T15:53:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.964+0000 7f83ad10f640 1 -- 172.21.15.27:0/4135619587 wait complete. 2023-08-11T15:53:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.966+0000 7f83ad10f640 1 Processor -- start 2023-08-11T15:53:10.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.966+0000 7f83ad10f640 1 -- start start 2023-08-11T15:53:10.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.967+0000 7f83ad10f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83a806b670 0x7f83a80cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.967+0000 7f83ad10f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 0x7f83a80c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.967+0000 7f83ad10f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a815c760 0x7f83a81510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.967+0000 7f83ad10f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80c6320 con 0x7f83a815c760 2023-08-11T15:53:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.967+0000 7f83ad10f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80c9ca0 con 0x7f83a806b670 2023-08-11T15:53:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.967+0000 7f83ad10f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80c9f70 con 0x7f83a80d2850 2023-08-11T15:53:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.967+0000 7f83a759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a815c760 0x7f83a81510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.967+0000 7f83a759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a815c760 0x7f83a81510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40198/0 (socket says 172.21.15.27:40198) 2023-08-11T15:53:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.967+0000 7f83a759e640 1 -- 172.21.15.27:0/476715591 learned_addr learned my addr 172.21.15.27:0/476715591 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83a6d9d640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83a806b670 0x7f83a80cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83a759e640 1 -- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 msgr2=0x7f83a80c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83a659c640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 0x7f83a80c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83a759e640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 0x7f83a80c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83a759e640 1 -- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83a806b670 msgr2=0x7f83a80cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83a759e640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83a806b670 0x7f83a80cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83a759e640 1 -- 172.21.15.27:0/476715591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8390009400 con 0x7f83a815c760 2023-08-11T15:53:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83a6d9d640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83a806b670 0x7f83a80cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83a659c640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 0x7f83a80c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83a759e640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a815c760 0x7f83a81510b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f83a000d800 tx=0x7f83a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f839f7fe640 1 -- 172.21.15.27:0/476715591 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a000b6c0 con 0x7f83a815c760 2023-08-11T15:53:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f839f7fe640 1 -- 172.21.15.27:0/476715591 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a000b840 con 0x7f83a815c760 2023-08-11T15:53:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f83ad10f640 1 -- 172.21.15.27:0/476715591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a80ca250 con 0x7f83a815c760 2023-08-11T15:53:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.968+0000 7f839f7fe640 1 -- 172.21.15.27:0/476715591 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a000bbf0 con 0x7f83a815c760 2023-08-11T15:53:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.969+0000 7f83ad10f640 1 -- 172.21.15.27:0/476715591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a81515f0 con 0x7f83a815c760 2023-08-11T15:53:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.970+0000 7f83ad10f640 1 -- 172.21.15.27:0/476715591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a80b5a50 con 0x7f83a815c760 2023-08-11T15:53:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.971+0000 7f839f7fe640 1 -- 172.21.15.27:0/476715591 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f83a000bd50 con 0x7f83a815c760 2023-08-11T15:53:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.972+0000 7f839f7fe640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f838806a050 0x7f838806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:10.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.972+0000 7f839f7fe640 1 -- 172.21.15.27:0/476715591 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f83a0088c50 con 0x7f83a815c760 2023-08-11T15:53:10.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.972+0000 7f83a6d9d640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f838806a050 0x7f838806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:10.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.973+0000 7f83a6d9d640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f838806a050 0x7f838806c510 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8398005c00 tx=0x7f8398009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:10.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:10.976+0000 7f839f7fe640 1 -- 172.21.15.27:0/476715591 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f83a00541f0 con 0x7f83a815c760 2023-08-11T15:53:11.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.213+0000 7f83ad10f640 1 -- 172.21.15.27:0/476715591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f83a80ca770 con 0x7f83a815c760 2023-08-11T15:53:11.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.794+0000 7f839f7fe640 1 -- 172.21.15.27:0/476715591 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f83a0057810 con 0x7f83a815c760 2023-08-11T15:53:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.798+0000 7f83ad10f640 1 -- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f838806a050 msgr2=0x7f838806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.798+0000 7f83ad10f640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f838806a050 0x7f838806c510 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8398005c00 tx=0x7f8398009040 comp rx=0 tx=0).stop 2023-08-11T15:53:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.798+0000 7f83ad10f640 1 -- 172.21.15.27:0/476715591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a815c760 msgr2=0x7f83a81510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.798+0000 7f83ad10f640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a815c760 0x7f83a81510b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f83a000d800 tx=0x7f83a000dcc0 comp rx=0 tx=0).stop 2023-08-11T15:53:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.798+0000 7f83ad10f640 1 -- 172.21.15.27:0/476715591 shutdown_connections 2023-08-11T15:53:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.798+0000 7f83ad10f640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f838806a050 0x7f838806c510 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.798+0000 7f83ad10f640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f83a80d2850 0x7f83a80c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.798+0000 7f83ad10f640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a815c760 0x7f83a81510b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.798+0000 7f83ad10f640 1 --2- 172.21.15.27:0/476715591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83a806b670 0x7f83a80cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.798+0000 7f83ad10f640 1 -- 172.21.15.27:0/476715591 >> 172.21.15.27:0/476715591 conn(0x7f83a8089ff0 msgr2=0x7f83a815aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.799+0000 7f83ad10f640 1 -- 172.21.15.27:0/476715591 shutdown_connections 2023-08-11T15:53:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:11.799+0000 7f83ad10f640 1 -- 172.21.15.27:0/476715591 wait complete. 2023-08-11T15:53:11.821 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-11T15:53:11.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-11T15:53:12.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.044+0000 7f8018fb8640 1 Processor -- start 2023-08-11T15:53:12.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.044+0000 7f8018fb8640 1 -- start start 2023-08-11T15:53:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.046+0000 7f8018fb8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f801413ff20 0x7f80140cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.046+0000 7f8018fb8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f801413d260 0x7f80140ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:12.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.046+0000 7f8018fb8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80140cd1a0 con 0x7f801413ce80 2023-08-11T15:53:12.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.046+0000 7f8018fb8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8014140670 con 0x7f801413ff20 2023-08-11T15:53:12.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.046+0000 7f8018fb8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80140cd470 con 0x7f801413d260 2023-08-11T15:53:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.046+0000 7f80137fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f801413ce80 0x7f801413f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49014/0 (socket says 172.21.15.27:49014) 2023-08-11T15:53:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.046+0000 7f80137fe640 1 -- 172.21.15.27:0/3642696551 learned_addr learned my addr 172.21.15.27:0/3642696551 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.046+0000 7f8012ffd640 1 --2- 172.21.15.27:0/3642696551 >> v2:172.21.15.27:3300/0 conn(0x7f801413ff20 0x7f80140cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.047+0000 7f8013fff640 1 --2- 172.21.15.27:0/3642696551 >> v2:172.21.15.119:3300/0 conn(0x7f801413d260 0x7f80140ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.047+0000 7f8012ffd640 1 -- 172.21.15.27:0/3642696551 >> v2:172.21.15.119:3300/0 conn(0x7f801413d260 msgr2=0x7f80140ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.047+0000 7f8012ffd640 1 --2- 172.21.15.27:0/3642696551 >> v2:172.21.15.119:3300/0 conn(0x7f801413d260 0x7f80140ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.047+0000 7f80127fc640 1 -- 172.21.15.27:0/3642696551 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019687471 0 0) 0x7f80140cd1a0 con 0x7f801413ce80 2023-08-11T15:53:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.047+0000 7f8012ffd640 1 -- 172.21.15.27:0/3642696551 >> v1:172.21.15.27:6789/0 conn(0x7f801413ce80 legacy=0x7f801413f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.047+0000 7f8012ffd640 1 -- 172.21.15.27:0/3642696551 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140cd750 con 0x7f801413ff20 2023-08-11T15:53:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.048+0000 7f8012ffd640 1 --2- 172.21.15.27:0/3642696551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ff20 0x7f80140cc720 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f80140cb710 tx=0x7f7ffc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c527d0e05d700cf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.048+0000 7f80127fc640 1 -- 172.21.15.27:0/3642696551 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc02cbf0 con 0x7f801413ff20 2023-08-11T15:53:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.049+0000 7f80127fc640 1 -- 172.21.15.27:0/3642696551 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc02cd50 con 0x7f801413ff20 2023-08-11T15:53:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.049+0000 7f80127fc640 1 -- 172.21.15.27:0/3642696551 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc034660 con 0x7f801413ff20 2023-08-11T15:53:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.049+0000 7f8018fb8640 1 -- 172.21.15.27:0/3642696551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ff20 msgr2=0x7f80140cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.049+0000 7f8018fb8640 1 --2- 172.21.15.27:0/3642696551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ff20 0x7f80140cc720 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f80140cb710 tx=0x7f7ffc02be40 comp rx=0 tx=0).stop 2023-08-11T15:53:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.049+0000 7f8018fb8640 1 -- 172.21.15.27:0/3642696551 shutdown_connections 2023-08-11T15:53:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.049+0000 7f8018fb8640 1 --2- 172.21.15.27:0/3642696551 >> v2:172.21.15.119:3300/0 conn(0x7f801413d260 0x7f80140ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.049+0000 7f8018fb8640 1 --2- 172.21.15.27:0/3642696551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ff20 0x7f80140cc720 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.049+0000 7f8018fb8640 1 -- 172.21.15.27:0/3642696551 >> 172.21.15.27:0/3642696551 conn(0x7f8014069800 msgr2=0x7f8014069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.049+0000 7f8018fb8640 1 -- 172.21.15.27:0/3642696551 shutdown_connections 2023-08-11T15:53:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.049+0000 7f8018fb8640 1 -- 172.21.15.27:0/3642696551 wait complete. 2023-08-11T15:53:12.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.051+0000 7f8018fb8640 1 Processor -- start 2023-08-11T15:53:12.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.051+0000 7f8018fb8640 1 -- start start 2023-08-11T15:53:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.052+0000 7f8018fb8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ce80 0x7f8014152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.052+0000 7f8018fb8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f801413d260 0x7f8014150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.052+0000 7f8018fb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801413ff20 0x7f80141518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.052+0000 7f8018fb8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8014151de0 con 0x7f801413ce80 2023-08-11T15:53:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.052+0000 7f8018fb8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801413e6e0 con 0x7f801413ff20 2023-08-11T15:53:12.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.052+0000 7f8018fb8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8014152010 con 0x7f801413d260 2023-08-11T15:53:12.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.053+0000 7f80137fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ce80 0x7f8014152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:12.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.053+0000 7f80137fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ce80 0x7f8014152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40216/0 (socket says 172.21.15.27:40216) 2023-08-11T15:53:12.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.053+0000 7f80137fe640 1 -- 172.21.15.27:0/3791237955 learned_addr learned my addr 172.21.15.27:0/3791237955 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:12.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.053+0000 7f80137fe640 1 -- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f801413d260 msgr2=0x7f8014150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:12.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.053+0000 7f80137fe640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f801413d260 0x7f8014150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:12.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.053+0000 7f80137fe640 1 -- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801413ff20 msgr2=0x7f80141518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:12.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.053+0000 7f8013fff640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801413ff20 0x7f80141518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.053+0000 7f80137fe640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801413ff20 0x7f80141518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.053+0000 7f80137fe640 1 -- 172.21.15.27:0/3791237955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc009a70 con 0x7f801413ce80 2023-08-11T15:53:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.054+0000 7f80137fe640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ce80 0x7f8014152940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f800400c4d0 tx=0x7f800400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.054+0000 7f800bfff640 1 -- 172.21.15.27:0/3791237955 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040074e0 con 0x7f801413ce80 2023-08-11T15:53:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.054+0000 7f800bfff640 1 -- 172.21.15.27:0/3791237955 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8004007660 con 0x7f801413ce80 2023-08-11T15:53:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.054+0000 7f800bfff640 1 -- 172.21.15.27:0/3791237955 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004004a20 con 0x7f801413ce80 2023-08-11T15:53:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.054+0000 7f8018fb8640 1 -- 172.21.15.27:0/3791237955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801413e090 con 0x7f801413ce80 2023-08-11T15:53:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.054+0000 7f8018fb8640 1 -- 172.21.15.27:0/3791237955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801413dd40 con 0x7f801413ce80 2023-08-11T15:53:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.055+0000 7f8018fb8640 1 -- 172.21.15.27:0/3791237955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8014066e80 con 0x7f801413ce80 2023-08-11T15:53:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.056+0000 7f800bfff640 1 -- 172.21.15.27:0/3791237955 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8004004b80 con 0x7f801413ce80 2023-08-11T15:53:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.058+0000 7f800bfff640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ff406a050 0x7f7ff406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.058+0000 7f800bfff640 1 -- 172.21.15.27:0/3791237955 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f80040878f0 con 0x7f801413ce80 2023-08-11T15:53:12.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.058+0000 7f8012ffd640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ff406a050 0x7f7ff406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:12.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.060+0000 7f8012ffd640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ff406a050 0x7f7ff406c510 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc036000 tx=0x7f7ffc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:12.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.062+0000 7f800bfff640 1 -- 172.21.15.27:0/3791237955 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f8004052e90 con 0x7f801413ce80 2023-08-11T15:53:12.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.294+0000 7f8018fb8640 1 -- 172.21.15.27:0/3791237955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f80140ad860 con 0x7f801413ce80 2023-08-11T15:53:12.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.798+0000 7f800bfff640 1 -- 172.21.15.27:0/3791237955 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f80040564b0 con 0x7f801413ce80 2023-08-11T15:53:12.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.802+0000 7f8018fb8640 1 -- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ff406a050 msgr2=0x7f7ff406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.802+0000 7f8018fb8640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ff406a050 0x7f7ff406c510 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc036000 tx=0x7f7ffc02e000 comp rx=0 tx=0).stop 2023-08-11T15:53:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.802+0000 7f8018fb8640 1 -- 172.21.15.27:0/3791237955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ce80 msgr2=0x7f8014152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.802+0000 7f8018fb8640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ce80 0x7f8014152940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f800400c4d0 tx=0x7f800400c990 comp rx=0 tx=0).stop 2023-08-11T15:53:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.803+0000 7f8018fb8640 1 -- 172.21.15.27:0/3791237955 shutdown_connections 2023-08-11T15:53:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.803+0000 7f8018fb8640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7ff406a050 0x7f7ff406c510 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.803+0000 7f8018fb8640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f801413d260 0x7f8014150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.803+0000 7f8018fb8640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801413ff20 0x7f80141518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.803+0000 7f8018fb8640 1 --2- 172.21.15.27:0/3791237955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801413ce80 0x7f8014152940 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.803+0000 7f8018fb8640 1 -- 172.21.15.27:0/3791237955 >> 172.21.15.27:0/3791237955 conn(0x7f8014069800 msgr2=0x7f80140aba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.803+0000 7f8018fb8640 1 -- 172.21.15.27:0/3791237955 shutdown_connections 2023-08-11T15:53:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:12.803+0000 7f8018fb8640 1 -- 172.21.15.27:0/3791237955 wait complete. 2023-08-11T15:53:12.826 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-11T15:53:12.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:53:13.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.056+0000 7f32c8f24640 1 Processor -- start 2023-08-11T15:53:13.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.056+0000 7f32c8f24640 1 -- start start 2023-08-11T15:53:13.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.058+0000 7f32c8f24640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f32c40b55c0 0x7f32c40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.058+0000 7f32c8f24640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c40cfd80 con 0x7f32c40cf930 2023-08-11T15:53:13.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.058+0000 7f32c8f24640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c40cda30 con 0x7f32c415c620 2023-08-11T15:53:13.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.058+0000 7f32c8f24640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c40ccc10 con 0x7f32c40b55c0 2023-08-11T15:53:13.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.058+0000 7f32c2d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f32c40cf930 0x7f32c40d0b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49022/0 (socket says 172.21.15.27:49022) 2023-08-11T15:53:13.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.058+0000 7f32c2d9d640 1 -- 172.21.15.27:0/673450269 learned_addr learned my addr 172.21.15.27:0/673450269 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:13.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.058+0000 7f32c1d9b640 1 --2- 172.21.15.27:0/673450269 >> v2:172.21.15.119:3300/0 conn(0x7f32c40b55c0 0x7f32c40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130277776 0 0) 0x7f32bc001110 con 0x7f32c40cf930 2023-08-11T15:53:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b0003520 con 0x7f32c40cf930 2023-08-11T15:53:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1133544275 0 0) 0x7f32b0003520 con 0x7f32c40cf930 2023-08-11T15:53:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 >> v2:172.21.15.119:3300/0 conn(0x7f32c40b55c0 msgr2=0x7f32c40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 --2- 172.21.15.27:0/673450269 >> v2:172.21.15.119:3300/0 conn(0x7f32c40b55c0 0x7f32c40cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 >> v1:172.21.15.119:6789/0 conn(0x7f32c415c620 legacy=0x7f32c40b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c4161110 con 0x7f32c40cf930 2023-08-11T15:53:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f32bc001800 con 0x7f32c40cf930 2023-08-11T15:53:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 >> v1:172.21.15.27:6789/0 conn(0x7f32c40cf930 legacy=0x7f32c40d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32b0003970 0x7f32b0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32b00047c0 0x7f32b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b0005180 0x7f32b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40ccc10 con 0x7f32b00047c0 2023-08-11T15:53:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b000c000 con 0x7f32b0005180 2023-08-11T15:53:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.059+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b000c1c0 con 0x7f32b0003970 2023-08-11T15:53:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.060+0000 7f32c1d9b640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32b00047c0 0x7f32b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.060+0000 7f32c2d9d640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b0005180 0x7f32b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.060+0000 7f32c259c640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32b0003970 0x7f32b0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.060+0000 7f32c1d9b640 1 -- 172.21.15.27:0/673450269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32b0003970 msgr2=0x7f32b0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.060+0000 7f32c1d9b640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32b0003970 0x7f32b0003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.060+0000 7f32c1d9b640 1 -- 172.21.15.27:0/673450269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b0005180 msgr2=0x7f32b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.060+0000 7f32c1d9b640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b0005180 0x7f32b000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.060+0000 7f32c1d9b640 1 -- 172.21.15.27:0/673450269 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b0003520 con 0x7f32b00047c0 2023-08-11T15:53:13.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.060+0000 7f32c2d9d640 1 -- 172.21.15.27:0/673450269 reap_dead start 2023-08-11T15:53:13.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.061+0000 7f32c1d9b640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32b00047c0 0x7f32b0004c40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009ac0 tx=0x7f32b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7169c5d3e9078a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:13.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.062+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac02c6b0 con 0x7f32b00047c0 2023-08-11T15:53:13.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.062+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac02c810 con 0x7f32b00047c0 2023-08-11T15:53:13.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.062+0000 7f32c159a640 1 -- 172.21.15.27:0/673450269 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac02cba0 con 0x7f32b00047c0 2023-08-11T15:53:13.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.062+0000 7f32c8f24640 1 -- 172.21.15.27:0/673450269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32b00047c0 msgr2=0x7f32b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.062+0000 7f32c8f24640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32b00047c0 0x7f32b0004c40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009ac0 tx=0x7f32b00057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.062+0000 7f32c8f24640 1 -- 172.21.15.27:0/673450269 shutdown_connections 2023-08-11T15:53:13.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.062+0000 7f32c8f24640 1 --2- 172.21.15.27:0/673450269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32b00047c0 0x7f32b0004c40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.062+0000 7f32c8f24640 1 -- 172.21.15.27:0/673450269 >> 172.21.15.27:0/673450269 conn(0x7f32c4071110 msgr2=0x7f32c40714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:13.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.062+0000 7f32c8f24640 1 -- 172.21.15.27:0/673450269 shutdown_connections 2023-08-11T15:53:13.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.062+0000 7f32c8f24640 1 -- 172.21.15.27:0/673450269 wait complete. 2023-08-11T15:53:13.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.064+0000 7f32c8f24640 1 Processor -- start 2023-08-11T15:53:13.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.064+0000 7f32c8f24640 1 -- start start 2023-08-11T15:53:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.065+0000 7f32c8f24640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32b00047c0 0x7f32c414c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.065+0000 7f32c8f24640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c415f420 0x7f32c415e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.065+0000 7f32c8f24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c415f8a0 0x7f32c415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.065+0000 7f32c8f24640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c4147c30 con 0x7f32c415f420 2023-08-11T15:53:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.065+0000 7f32c8f24640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c4146da0 con 0x7f32c415f8a0 2023-08-11T15:53:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.065+0000 7f32c8f24640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c414c7b0 con 0x7f32b00047c0 2023-08-11T15:53:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.065+0000 7f32c1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c415f420 0x7f32c415e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.066+0000 7f32c1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c415f420 0x7f32c415e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40234/0 (socket says 172.21.15.27:40234) 2023-08-11T15:53:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.066+0000 7f32c1d9b640 1 -- 172.21.15.27:0/2578783281 learned_addr learned my addr 172.21.15.27:0/2578783281 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.066+0000 7f32c1d9b640 1 -- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32b00047c0 msgr2=0x7f32c414c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.066+0000 7f32c1d9b640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32b00047c0 0x7f32c414c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.066+0000 7f32c1d9b640 1 -- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c415f8a0 msgr2=0x7f32c415ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.066+0000 7f32c1d9b640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c415f8a0 0x7f32c415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.066+0000 7f32c1d9b640 1 -- 172.21.15.27:0/2578783281 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c4147020 con 0x7f32c415f420 2023-08-11T15:53:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.066+0000 7f32c1d9b640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c415f420 0x7f32c415e520 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f32ac037040 tx=0x7f32ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.066+0000 7f32baffd640 1 -- 172.21.15.27:0/2578783281 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac003810 con 0x7f32c415f420 2023-08-11T15:53:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.067+0000 7f32baffd640 1 -- 172.21.15.27:0/2578783281 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac003970 con 0x7f32c415f420 2023-08-11T15:53:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.067+0000 7f32c8f24640 1 -- 172.21.15.27:0/2578783281 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c4147c30 con 0x7f32c415f420 2023-08-11T15:53:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.067+0000 7f32baffd640 1 -- 172.21.15.27:0/2578783281 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac003d00 con 0x7f32c415f420 2023-08-11T15:53:13.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.067+0000 7f32c8f24640 1 -- 172.21.15.27:0/2578783281 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c41475e0 con 0x7f32c415f420 2023-08-11T15:53:13.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.068+0000 7f32c8f24640 1 -- 172.21.15.27:0/2578783281 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c4073990 con 0x7f32c415f420 2023-08-11T15:53:13.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.069+0000 7f32baffd640 1 -- 172.21.15.27:0/2578783281 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f32ac034440 con 0x7f32c415f420 2023-08-11T15:53:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.070+0000 7f32baffd640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32a406a050 0x7f32a406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.070+0000 7f32baffd640 1 -- 172.21.15.27:0/2578783281 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f32ac0aaaa0 con 0x7f32c415f420 2023-08-11T15:53:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.070+0000 7f32c259c640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32a406a050 0x7f32a406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.071+0000 7f32c259c640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32a406a050 0x7f32a406c510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f32c404f4d0 tx=0x7f32b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.074+0000 7f32baffd640 1 -- 172.21.15.27:0/2578783281 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f32ac076060 con 0x7f32c415f420 2023-08-11T15:53:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.310+0000 7f32c8f24640 1 -- 172.21.15.27:0/2578783281 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32c4147dc0 con 0x7f32c415f420 2023-08-11T15:53:13.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.311+0000 7f32baffd640 1 -- 172.21.15.27:0/2578783281 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f32ac079680 con 0x7f32c415f420 2023-08-11T15:53:13.314 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:13.314 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2294518038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2294518038},{"type":"v1","addr":"172.21.15.119:6836","nonce":2294518038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5188,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2104666724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2104666724},{"type":"v1","addr":"172.21.15.119:6837","nonce":2104666724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2379252997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2379252997},{"type":"v1","addr":"172.21.15.27:6833","nonce":2379252997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:53:09.220611+0000","modified":"2023-08-11T15:53:12.792531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5643},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5643":{"gid":5643,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/899405615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899405615},{"type":"v1","addr":"172.21.15.27:6835","nonce":899405615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-11T15:53:13.315 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 18 2023-08-11T15:53:13.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 -- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32a406a050 msgr2=0x7f32a406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32a406a050 0x7f32a406c510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f32c404f4d0 tx=0x7f32b4009040 comp rx=0 tx=0).stop 2023-08-11T15:53:13.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 -- 172.21.15.27:0/2578783281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c415f420 msgr2=0x7f32c415e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c415f420 0x7f32c415e520 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f32ac037040 tx=0x7f32ac004730 comp rx=0 tx=0).stop 2023-08-11T15:53:13.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 -- 172.21.15.27:0/2578783281 shutdown_connections 2023-08-11T15:53:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32a406a050 0x7f32a406c510 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c415f8a0 0x7f32c415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c415f420 0x7f32c415e520 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 --2- 172.21.15.27:0/2578783281 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32b00047c0 0x7f32c414c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 -- 172.21.15.27:0/2578783281 >> 172.21.15.27:0/2578783281 conn(0x7f32c4071110 msgr2=0x7f32c40b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 -- 172.21.15.27:0/2578783281 shutdown_connections 2023-08-11T15:53:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.314+0000 7f32c8f24640 1 -- 172.21.15.27:0/2578783281 wait complete. 2023-08-11T15:53:13.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:53:13.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.564+0000 7f1e4dcd7640 1 Processor -- start 2023-08-11T15:53:13.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.564+0000 7f1e4dcd7640 1 -- start start 2023-08-11T15:53:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.565+0000 7f1e4dcd7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e4815c1f0 0x7f1e480b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.566+0000 7f1e4dcd7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e4815ca30 0x7f1e48160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.566+0000 7f1e4dcd7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e480b5170 con 0x7f1e4815c5d0 2023-08-11T15:53:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.566+0000 7f1e4dcd7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4815ce70 con 0x7f1e4815ca30 2023-08-11T15:53:13.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.566+0000 7f1e4dcd7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4815d080 con 0x7f1e4815c1f0 2023-08-11T15:53:13.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.566+0000 7f1e47fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1e4815c5d0 0x7f1e480d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49026/0 (socket says 172.21.15.27:49026) 2023-08-11T15:53:13.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.566+0000 7f1e47fff640 1 -- 172.21.15.27:0/551834197 learned_addr learned my addr 172.21.15.27:0/551834197 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:13.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.566+0000 7f1e4d4d6640 1 --2- 172.21.15.27:0/551834197 >> v2:172.21.15.27:3300/0 conn(0x7f1e4815ca30 0x7f1e48160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.566+0000 7f1e4ccd5640 1 --2- 172.21.15.27:0/551834197 >> v2:172.21.15.119:3300/0 conn(0x7f1e4815c1f0 0x7f1e480b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.567+0000 7f1e4d4d6640 1 -- 172.21.15.27:0/551834197 >> v2:172.21.15.119:3300/0 conn(0x7f1e4815c1f0 msgr2=0x7f1e480b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:13.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.567+0000 7f1e4d4d6640 1 --2- 172.21.15.27:0/551834197 >> v2:172.21.15.119:3300/0 conn(0x7f1e4815c1f0 0x7f1e480b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.567+0000 7f1e4d4d6640 1 -- 172.21.15.27:0/551834197 >> v1:172.21.15.27:6789/0 conn(0x7f1e4815c5d0 legacy=0x7f1e480d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.567+0000 7f1e4d4d6640 1 -- 172.21.15.27:0/551834197 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4815d360 con 0x7f1e4815ca30 2023-08-11T15:53:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.568+0000 7f1e4d4d6640 1 --2- 172.21.15.27:0/551834197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815ca30 0x7f1e48160510 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1e4808b310 tx=0x7f1e4002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae1f20d2430b3e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.569+0000 7f1e477fe640 1 -- 172.21.15.27:0/551834197 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40030610 con 0x7f1e4815ca30 2023-08-11T15:53:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.569+0000 7f1e477fe640 1 -- 172.21.15.27:0/551834197 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e40030770 con 0x7f1e4815ca30 2023-08-11T15:53:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.569+0000 7f1e477fe640 1 -- 172.21.15.27:0/551834197 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40030b00 con 0x7f1e4815ca30 2023-08-11T15:53:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.569+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/551834197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815ca30 msgr2=0x7f1e48160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.569+0000 7f1e4dcd7640 1 --2- 172.21.15.27:0/551834197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815ca30 0x7f1e48160510 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1e4808b310 tx=0x7f1e4002b960 comp rx=0 tx=0).stop 2023-08-11T15:53:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.569+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/551834197 shutdown_connections 2023-08-11T15:53:13.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.569+0000 7f1e4dcd7640 1 --2- 172.21.15.27:0/551834197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815ca30 0x7f1e48160510 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.569+0000 7f1e4dcd7640 1 --2- 172.21.15.27:0/551834197 >> v2:172.21.15.119:3300/0 conn(0x7f1e4815c1f0 0x7f1e480b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.569+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/551834197 >> 172.21.15.27:0/551834197 conn(0x7f1e4806a250 msgr2=0x7f1e4806a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:13.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.570+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/551834197 shutdown_connections 2023-08-11T15:53:13.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.570+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/551834197 wait complete. 2023-08-11T15:53:13.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.571+0000 7f1e4dcd7640 1 Processor -- start 2023-08-11T15:53:13.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.571+0000 7f1e4dcd7640 1 -- start start 2023-08-11T15:53:13.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.572+0000 7f1e4dcd7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815c1f0 0x7f1e48151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.572+0000 7f1e4dcd7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e4815c5d0 0x7f1e4815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.572+0000 7f1e4dcd7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4815ca30 0x7f1e4815b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.572+0000 7f1e4dcd7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e480cba40 con 0x7f1e4815c1f0 2023-08-11T15:53:13.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.572+0000 7f1e4dcd7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4815d990 con 0x7f1e4815c5d0 2023-08-11T15:53:13.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.572+0000 7f1e4dcd7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4815dc70 con 0x7f1e4815ca30 2023-08-11T15:53:13.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e4ccd5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815c1f0 0x7f1e48151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e4ccd5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815c1f0 0x7f1e48151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40258/0 (socket says 172.21.15.27:40258) 2023-08-11T15:53:13.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e4ccd5640 1 -- 172.21.15.27:0/1002482133 learned_addr learned my addr 172.21.15.27:0/1002482133 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e4d4d6640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4815ca30 0x7f1e4815b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e47fff640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e4815c5d0 0x7f1e4815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e4ccd5640 1 -- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4815ca30 msgr2=0x7f1e4815b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e4ccd5640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4815ca30 0x7f1e4815b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e4ccd5640 1 -- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e4815c5d0 msgr2=0x7f1e4815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e4ccd5640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e4815c5d0 0x7f1e4815ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e4ccd5640 1 -- 172.21.15.27:0/1002482133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40009400 con 0x7f1e4815c1f0 2023-08-11T15:53:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e4d4d6640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4815ca30 0x7f1e4815b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.573+0000 7f1e47fff640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e4815c5d0 0x7f1e4815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:13.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.574+0000 7f1e4ccd5640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815c1f0 0x7f1e48151fd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800efc0 tx=0x7f1e3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:13.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.574+0000 7f1e457fa640 1 -- 172.21.15.27:0/1002482133 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38014480 con 0x7f1e4815c1f0 2023-08-11T15:53:13.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.574+0000 7f1e457fa640 1 -- 172.21.15.27:0/1002482133 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3800f040 con 0x7f1e4815c1f0 2023-08-11T15:53:13.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.574+0000 7f1e457fa640 1 -- 172.21.15.27:0/1002482133 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38014800 con 0x7f1e4815c1f0 2023-08-11T15:53:13.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.574+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/1002482133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4815df50 con 0x7f1e4815c1f0 2023-08-11T15:53:13.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.574+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/1002482133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4815e510 con 0x7f1e4815c1f0 2023-08-11T15:53:13.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.575+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/1002482133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4804e420 con 0x7f1e4815c1f0 2023-08-11T15:53:13.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.576+0000 7f1e457fa640 1 -- 172.21.15.27:0/1002482133 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e38014960 con 0x7f1e4815c1f0 2023-08-11T15:53:13.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.578+0000 7f1e457fa640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e3406a050 0x7f1e3406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:13.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.578+0000 7f1e457fa640 1 -- 172.21.15.27:0/1002482133 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1e380892d0 con 0x7f1e4815c1f0 2023-08-11T15:53:13.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.579+0000 7f1e47fff640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e3406a050 0x7f1e3406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:13.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.580+0000 7f1e47fff640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e3406a050 0x7f1e3406c510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1e48152e00 tx=0x7f1e3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:13.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.584+0000 7f1e457fa640 1 -- 172.21.15.27:0/1002482133 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f1e38058300 con 0x7f1e4815c1f0 2023-08-11T15:53:13.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.782+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/1002482133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e480cfd80 con 0x7f1e4815c1f0 2023-08-11T15:53:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.784+0000 7f1e457fa640 1 -- 172.21.15.27:0/1002482133 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18500 (secure 0 0 0) 0x7f1e38057e90 con 0x7f1e4815c1f0 2023-08-11T15:53:13.787 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:13.787 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":43,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:53:10.738370+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:53:06.898951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-11T15:53:08.258996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:53:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e3406a050 msgr2=0x7f1e3406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e3406a050 0x7f1e3406c510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1e48152e00 tx=0x7f1e3c009040 comp rx=0 tx=0).stop 2023-08-11T15:53:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/1002482133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815c1f0 msgr2=0x7f1e48151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815c1f0 0x7f1e48151fd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800efc0 tx=0x7f1e3800c490 comp rx=0 tx=0).stop 2023-08-11T15:53:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/1002482133 shutdown_connections 2023-08-11T15:53:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1e3406a050 0x7f1e3406c510 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4815ca30 0x7f1e4815b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e4815c5d0 0x7f1e4815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 --2- 172.21.15.27:0/1002482133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4815c1f0 0x7f1e48151fd0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:13.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/1002482133 >> 172.21.15.27:0/1002482133 conn(0x7f1e4806a250 msgr2=0x7f1e48154580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:13.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/1002482133 shutdown_connections 2023-08-11T15:53:13.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:13.787+0000 7f1e4dcd7640 1 -- 172.21.15.27:0/1002482133 wait complete. 2023-08-11T15:53:13.808 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-11T15:53:13.809 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:53:14.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.034+0000 7f43d58c5640 1 Processor -- start 2023-08-11T15:53:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.034+0000 7f43d58c5640 1 -- start start 2023-08-11T15:53:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.036+0000 7f43d58c5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43d006ae40 0x7f43d00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.036+0000 7f43d58c5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f43d006f0b0 0x7f43d015cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.036+0000 7f43d58c5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d006b220 con 0x7f43d015d220 2023-08-11T15:53:14.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.036+0000 7f43d58c5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d006f590 con 0x7f43d006ae40 2023-08-11T15:53:14.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.036+0000 7f43d58c5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d006f870 con 0x7f43d006f0b0 2023-08-11T15:53:14.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.036+0000 7f43ceffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43d006ae40 0x7f43d00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.036+0000 7f43ceffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43d006ae40 0x7f43d00cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40268/0 (socket says 172.21.15.27:40268) 2023-08-11T15:53:14.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.036+0000 7f43ceffd640 1 -- 172.21.15.27:0/848136631 learned_addr learned my addr 172.21.15.27:0/848136631 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:14.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.036+0000 7f43ce7fc640 1 --2- 172.21.15.27:0/848136631 >> v2:172.21.15.119:3300/0 conn(0x7f43d006f0b0 0x7f43d015cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.037+0000 7f43ceffd640 1 -- 172.21.15.27:0/848136631 >> v2:172.21.15.119:3300/0 conn(0x7f43d006f0b0 msgr2=0x7f43d015cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.037+0000 7f43ceffd640 1 --2- 172.21.15.27:0/848136631 >> v2:172.21.15.119:3300/0 conn(0x7f43d006f0b0 0x7f43d015cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.037+0000 7f43ceffd640 1 -- 172.21.15.27:0/848136631 >> v1:172.21.15.119:6789/0 conn(0x7f43d015d220 legacy=0x7f43d006dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.037+0000 7f43ceffd640 1 -- 172.21.15.27:0/848136631 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d006fb50 con 0x7f43d006ae40 2023-08-11T15:53:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.038+0000 7f43ceffd640 1 --2- 172.21.15.27:0/848136631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006ae40 0x7f43d00cf900 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f43d0072000 tx=0x7f43c002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cb337060123395 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.038+0000 7f43cdffb640 1 -- 172.21.15.27:0/848136631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c002c9f0 con 0x7f43d006ae40 2023-08-11T15:53:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.038+0000 7f43cdffb640 1 -- 172.21.15.27:0/848136631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c002cb50 con 0x7f43d006ae40 2023-08-11T15:53:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.038+0000 7f43cdffb640 1 -- 172.21.15.27:0/848136631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c00343f0 con 0x7f43d006ae40 2023-08-11T15:53:14.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.039+0000 7f43d58c5640 1 -- 172.21.15.27:0/848136631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006ae40 msgr2=0x7f43d00cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.039+0000 7f43d58c5640 1 --2- 172.21.15.27:0/848136631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006ae40 0x7f43d00cf900 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f43d0072000 tx=0x7f43c002be50 comp rx=0 tx=0).stop 2023-08-11T15:53:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.039+0000 7f43d58c5640 1 -- 172.21.15.27:0/848136631 shutdown_connections 2023-08-11T15:53:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.039+0000 7f43d58c5640 1 --2- 172.21.15.27:0/848136631 >> v2:172.21.15.119:3300/0 conn(0x7f43d006f0b0 0x7f43d015cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.039+0000 7f43d58c5640 1 --2- 172.21.15.27:0/848136631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006ae40 0x7f43d00cf900 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.039+0000 7f43d58c5640 1 -- 172.21.15.27:0/848136631 >> 172.21.15.27:0/848136631 conn(0x7f43d00d0220 msgr2=0x7f43d00d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.039+0000 7f43d58c5640 1 -- 172.21.15.27:0/848136631 shutdown_connections 2023-08-11T15:53:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.039+0000 7f43d58c5640 1 -- 172.21.15.27:0/848136631 wait complete. 2023-08-11T15:53:14.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.041+0000 7f43d58c5640 1 Processor -- start 2023-08-11T15:53:14.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.041+0000 7f43d58c5640 1 -- start start 2023-08-11T15:53:14.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.042+0000 7f43d58c5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f43d006ae40 0x7f43d01f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.042+0000 7f43d58c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006f0b0 0x7f43d01f9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.042+0000 7f43d58c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43d015d220 0x7f43d01f9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.042+0000 7f43d58c5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01f62f0 con 0x7f43d006f0b0 2023-08-11T15:53:14.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.042+0000 7f43d58c5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01fa820 con 0x7f43d015d220 2023-08-11T15:53:14.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.042+0000 7f43d58c5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d00b0870 con 0x7f43d006ae40 2023-08-11T15:53:14.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.042+0000 7f43ce7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006f0b0 0x7f43d01f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.042+0000 7f43ce7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006f0b0 0x7f43d01f9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40276/0 (socket says 172.21.15.27:40276) 2023-08-11T15:53:14.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.042+0000 7f43ce7fc640 1 -- 172.21.15.27:0/1141853805 learned_addr learned my addr 172.21.15.27:0/1141853805 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:14.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43ce7fc640 1 -- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f43d006ae40 msgr2=0x7f43d01f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43ce7fc640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f43d006ae40 0x7f43d01f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43ce7fc640 1 -- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43d015d220 msgr2=0x7f43d01f9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43ce7fc640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43d015d220 0x7f43d01f9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43ce7fc640 1 -- 172.21.15.27:0/1141853805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0009b30 con 0x7f43d006f0b0 2023-08-11T15:53:14.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43ce7fc640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006f0b0 0x7f43d01f9980 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c7f0 tx=0x7f43b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43c7fff640 1 -- 172.21.15.27:0/1141853805 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8004220 con 0x7f43d006f0b0 2023-08-11T15:53:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43c7fff640 1 -- 172.21.15.27:0/1141853805 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b80043a0 con 0x7f43d006f0b0 2023-08-11T15:53:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43c7fff640 1 -- 172.21.15.27:0/1141853805 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8007860 con 0x7f43d006f0b0 2023-08-11T15:53:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43d58c5640 1 -- 172.21.15.27:0/1141853805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d00b0b50 con 0x7f43d006f0b0 2023-08-11T15:53:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.043+0000 7f43d58c5640 1 -- 172.21.15.27:0/1141853805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d00b1110 con 0x7f43d006f0b0 2023-08-11T15:53:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.045+0000 7f43d58c5640 1 -- 172.21.15.27:0/1141853805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d00b1300 con 0x7f43d006f0b0 2023-08-11T15:53:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.045+0000 7f43c7fff640 1 -- 172.21.15.27:0/1141853805 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f43b8007bd0 con 0x7f43d006f0b0 2023-08-11T15:53:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.046+0000 7f43c7fff640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f43b006a050 0x7f43b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.047+0000 7f43c7fff640 1 -- 172.21.15.27:0/1141853805 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f43b80887c0 con 0x7f43d006f0b0 2023-08-11T15:53:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.047+0000 7f43ceffd640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f43b006a050 0x7f43b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.048+0000 7f43ceffd640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f43b006a050 0x7f43b006c510 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f43c0035000 tx=0x7f43c002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.051+0000 7f43c7fff640 1 -- 172.21.15.27:0/1141853805 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f43b8053cd0 con 0x7f43d006f0b0 2023-08-11T15:53:14.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.239+0000 7f43d58c5640 1 -- 172.21.15.27:0/1141853805 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43d00b09e0 con 0x7f43b006a050 2023-08-11T15:53:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.269+0000 7f43c7fff640 1 -- 172.21.15.27:0/1141853805 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309081 (secure 0 0 0) 0x7f43d00b09e0 con 0x7f43b006a050 2023-08-11T15:53:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:14.277 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:53:14.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 -- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f43b006a050 msgr2=0x7f43b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f43b006a050 0x7f43b006c510 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f43c0035000 tx=0x7f43c002e000 comp rx=0 tx=0).stop 2023-08-11T15:53:14.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 -- 172.21.15.27:0/1141853805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006f0b0 msgr2=0x7f43d01f9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006f0b0 0x7f43d01f9980 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c7f0 tx=0x7f43b800ccb0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 -- 172.21.15.27:0/1141853805 shutdown_connections 2023-08-11T15:53:14.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f43b006a050 0x7f43b006c510 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43d015d220 0x7f43d01f9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d006f0b0 0x7f43d01f9980 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 --2- 172.21.15.27:0/1141853805 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f43d006ae40 0x7f43d01f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 -- 172.21.15.27:0/1141853805 >> 172.21.15.27:0/1141853805 conn(0x7f43d00d0220 msgr2=0x7f43d00cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:14.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.276+0000 7f43d58c5640 1 -- 172.21.15.27:0/1141853805 shutdown_connections 2023-08-11T15:53:14.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.277+0000 7f43d58c5640 1 -- 172.21.15.27:0/1141853805 wait complete. 2023-08-11T15:53:14.301 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2023-08-11T15:53:12.761421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233552,"kb_used_data":5084,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754741168,"statfs":{"total":773094113280,"available":772854956032,"internally_reserved":0,"allocated":5206016,"data_stored":2955151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.055947"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180399+0000","last_change":"2023-08-11T15:53:09.773210+0000","last_active":"2023-08-11T15:53:11.180399+0000","last_peered":"2023-08-11T15:53:11.180399+0000","last_clean":"2023-08-11T15:53:11.180399+0000","last_became_active":"2023-08-11T15:53:09.772902+0000","last_became_peered":"2023-08-11T15:53:09.772902+0000","last_unstale":"2023-08-11T15:53:11.180399+0000","last_undegraded":"2023-08-11T15:53:11.180399+0000","last_fullsized":"2023-08-11T15:53:11.180399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:12.451179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753016+0000","last_change":"2023-08-11T15:53:09.791848+0000","last_active":"2023-08-11T15:53:10.753016+0000","last_peered":"2023-08-11T15:53:10.753016+0000","last_clean":"2023-08-11T15:53:10.753016+0000","last_became_active":"2023-08-11T15:53:09.791542+0000","last_became_peered":"2023-08-11T15:53:09.791542+0000","last_unstale":"2023-08-11T15:53:10.753016+0000","last_undegraded":"2023-08-11T15:53:10.753016+0000","last_fullsized":"2023-08-11T15:53:10.753016+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:10.218889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754347+0000","last_change":"2023-08-11T15:53:09.781474+0000","last_active":"2023-08-11T15:53:10.754347+0000","last_peered":"2023-08-11T15:53:10.754347+0000","last_clean":"2023-08-11T15:53:10.754347+0000","last_became_active":"2023-08-11T15:53:09.781167+0000","last_became_peered":"2023-08-11T15:53:09.781167+0000","last_unstale":"2023-08-11T15:53:10.754347+0000","last_undegraded":"2023-08-11T15:53:10.754347+0000","last_fullsized":"2023-08-11T15:53:10.754347+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:24:33.839216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181867+0000","last_change":"2023-08-11T15:53:09.785095+0000","last_active":"2023-08-11T15:53:11.181867+0000","last_peered":"2023-08-11T15:53:11.181867+0000","last_clean":"2023-08-11T15:53:11.181867+0000","last_became_active":"2023-08-11T15:53:09.783871+0000","last_became_peered":"2023-08-11T15:53:09.783871+0000","last_unstale":"2023-08-11T15:53:11.181867+0000","last_undegraded":"2023-08-11T15:53:11.181867+0000","last_fullsized":"2023-08-11T15:53:11.181867+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:10:51.539728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751022+0000","last_change":"2023-08-11T15:53:09.793442+0000","last_active":"2023-08-11T15:53:10.751022+0000","last_peered":"2023-08-11T15:53:10.751022+0000","last_clean":"2023-08-11T15:53:10.751022+0000","last_became_active":"2023-08-11T15:53:09.793158+0000","last_became_peered":"2023-08-11T15:53:09.793158+0000","last_unstale":"2023-08-11T15:53:10.751022+0000","last_undegraded":"2023-08-11T15:53:10.751022+0000","last_fullsized":"2023-08-11T15:53:10.751022+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:49.810015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106013+0000","last_change":"2023-08-11T15:53:09.782227+0000","last_active":"2023-08-11T15:53:11.106013+0000","last_peered":"2023-08-11T15:53:11.106013+0000","last_clean":"2023-08-11T15:53:11.106013+0000","last_became_active":"2023-08-11T15:53:09.781914+0000","last_became_peered":"2023-08-11T15:53:09.781914+0000","last_unstale":"2023-08-11T15:53:11.106013+0000","last_undegraded":"2023-08-11T15:53:11.106013+0000","last_fullsized":"2023-08-11T15:53:11.106013+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:28:49.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180066+0000","last_change":"2023-08-11T15:53:09.769652+0000","last_active":"2023-08-11T15:53:11.180066+0000","last_peered":"2023-08-11T15:53:11.180066+0000","last_clean":"2023-08-11T15:53:11.180066+0000","last_became_active":"2023-08-11T15:53:09.769356+0000","last_became_peered":"2023-08-11T15:53:09.769356+0000","last_unstale":"2023-08-11T15:53:11.180066+0000","last_undegraded":"2023-08-11T15:53:11.180066+0000","last_fullsized":"2023-08-11T15:53:11.180066+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:04:07.355560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.183055+0000","last_change":"2023-08-11T15:53:09.782362+0000","last_active":"2023-08-11T15:53:11.183055+0000","last_peered":"2023-08-11T15:53:11.183055+0000","last_clean":"2023-08-11T15:53:11.183055+0000","last_became_active":"2023-08-11T15:53:09.782053+0000","last_became_peered":"2023-08-11T15:53:09.782053+0000","last_unstale":"2023-08-11T15:53:11.183055+0000","last_undegraded":"2023-08-11T15:53:11.183055+0000","last_fullsized":"2023-08-11T15:53:11.183055+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:45:27.150501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181078+0000","last_change":"2023-08-11T15:53:09.782949+0000","last_active":"2023-08-11T15:53:11.181078+0000","last_peered":"2023-08-11T15:53:11.181078+0000","last_clean":"2023-08-11T15:53:11.181078+0000","last_became_active":"2023-08-11T15:53:09.782603+0000","last_became_peered":"2023-08-11T15:53:09.782603+0000","last_unstale":"2023-08-11T15:53:11.181078+0000","last_undegraded":"2023-08-11T15:53:11.181078+0000","last_fullsized":"2023-08-11T15:53:11.181078+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:31:07.338552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182401+0000","last_change":"2023-08-11T15:53:09.789197+0000","last_active":"2023-08-11T15:53:11.182401+0000","last_peered":"2023-08-11T15:53:11.182401+0000","last_clean":"2023-08-11T15:53:11.182401+0000","last_became_active":"2023-08-11T15:53:09.788353+0000","last_became_peered":"2023-08-11T15:53:09.788353+0000","last_unstale":"2023-08-11T15:53:11.182401+0000","last_undegraded":"2023-08-11T15:53:11.182401+0000","last_fullsized":"2023-08-11T15:53:11.182401+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:41:25.658590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.183187+0000","last_change":"2023-08-11T15:53:09.791697+0000","last_active":"2023-08-11T15:53:11.183187+0000","last_peered":"2023-08-11T15:53:11.183187+0000","last_clean":"2023-08-11T15:53:11.183187+0000","last_became_active":"2023-08-11T15:53:09.791410+0000","last_became_peered":"2023-08-11T15:53:09.791410+0000","last_unstale":"2023-08-11T15:53:11.183187+0000","last_undegraded":"2023-08-11T15:53:11.183187+0000","last_fullsized":"2023-08-11T15:53:11.183187+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:24:26.104141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750984+0000","last_change":"2023-08-11T15:53:09.776182+0000","last_active":"2023-08-11T15:53:10.750984+0000","last_peered":"2023-08-11T15:53:10.750984+0000","last_clean":"2023-08-11T15:53:10.750984+0000","last_became_active":"2023-08-11T15:53:09.775906+0000","last_became_peered":"2023-08-11T15:53:09.775906+0000","last_unstale":"2023-08-11T15:53:10.750984+0000","last_undegraded":"2023-08-11T15:53:10.750984+0000","last_fullsized":"2023-08-11T15:53:10.750984+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:50:11.087522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181594+0000","last_change":"2023-08-11T15:53:09.780882+0000","last_active":"2023-08-11T15:53:11.181594+0000","last_peered":"2023-08-11T15:53:11.181594+0000","last_clean":"2023-08-11T15:53:11.181594+0000","last_became_active":"2023-08-11T15:53:09.780459+0000","last_became_peered":"2023-08-11T15:53:09.780459+0000","last_unstale":"2023-08-11T15:53:11.181594+0000","last_undegraded":"2023-08-11T15:53:11.181594+0000","last_fullsized":"2023-08-11T15:53:11.181594+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:36:41.825732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181130+0000","last_change":"2023-08-11T15:53:09.782350+0000","last_active":"2023-08-11T15:53:11.181130+0000","last_peered":"2023-08-11T15:53:11.181130+0000","last_clean":"2023-08-11T15:53:11.181130+0000","last_became_active":"2023-08-11T15:53:09.781999+0000","last_became_peered":"2023-08-11T15:53:09.781999+0000","last_unstale":"2023-08-11T15:53:11.181130+0000","last_undegraded":"2023-08-11T15:53:11.181130+0000","last_fullsized":"2023-08-11T15:53:11.181130+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:54:54.260711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.105963+0000","last_change":"2023-08-11T15:53:09.777919+0000","last_active":"2023-08-11T15:53:11.105963+0000","last_peered":"2023-08-11T15:53:11.105963+0000","last_clean":"2023-08-11T15:53:11.105963+0000","last_became_active":"2023-08-11T15:53:09.777623+0000","last_became_peered":"2023-08-11T15:53:09.777623+0000","last_unstale":"2023-08-11T15:53:11.105963+0000","last_undegraded":"2023-08-11T15:53:11.105963+0000","last_fullsized":"2023-08-11T15:53:11.105963+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:04:07.355560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751415+0000","last_change":"2023-08-11T15:53:09.786416+0000","last_active":"2023-08-11T15:53:10.751415+0000","last_peered":"2023-08-11T15:53:10.751415+0000","last_clean":"2023-08-11T15:53:10.751415+0000","last_became_active":"2023-08-11T15:53:09.786039+0000","last_became_peered":"2023-08-11T15:53:09.786039+0000","last_unstale":"2023-08-11T15:53:10.751415+0000","last_undegraded":"2023-08-11T15:53:10.751415+0000","last_fullsized":"2023-08-11T15:53:10.751415+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:40:17.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754308+0000","last_change":"2023-08-11T15:53:09.771798+0000","last_active":"2023-08-11T15:53:10.754308+0000","last_peered":"2023-08-11T15:53:10.754308+0000","last_clean":"2023-08-11T15:53:10.754308+0000","last_became_active":"2023-08-11T15:53:09.771479+0000","last_became_peered":"2023-08-11T15:53:09.771479+0000","last_unstale":"2023-08-11T15:53:10.754308+0000","last_undegraded":"2023-08-11T15:53:10.754308+0000","last_fullsized":"2023-08-11T15:53:10.754308+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:44:46.953266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754113+0000","last_change":"2023-08-11T15:53:09.790561+0000","last_active":"2023-08-11T15:53:10.754113+0000","last_peered":"2023-08-11T15:53:10.754113+0000","last_clean":"2023-08-11T15:53:10.754113+0000","last_became_active":"2023-08-11T15:53:09.790261+0000","last_became_peered":"2023-08-11T15:53:09.790261+0000","last_unstale":"2023-08-11T15:53:10.754113+0000","last_undegraded":"2023-08-11T15:53:10.754113+0000","last_fullsized":"2023-08-11T15:53:10.754113+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:16:25.172596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753423+0000","last_change":"2023-08-11T15:53:09.788819+0000","last_active":"2023-08-11T15:53:10.753423+0000","last_peered":"2023-08-11T15:53:10.753423+0000","last_clean":"2023-08-11T15:53:10.753423+0000","last_became_active":"2023-08-11T15:53:09.788567+0000","last_became_peered":"2023-08-11T15:53:09.788567+0000","last_unstale":"2023-08-11T15:53:10.753423+0000","last_undegraded":"2023-08-11T15:53:10.753423+0000","last_fullsized":"2023-08-11T15:53:10.753423+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:30:12.859168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182993+0000","last_change":"2023-08-11T15:53:09.786551+0000","last_active":"2023-08-11T15:53:11.182993+0000","last_peered":"2023-08-11T15:53:11.182993+0000","last_clean":"2023-08-11T15:53:11.182993+0000","last_became_active":"2023-08-11T15:53:09.786173+0000","last_became_peered":"2023-08-11T15:53:09.786173+0000","last_unstale":"2023-08-11T15:53:11.182993+0000","last_undegraded":"2023-08-11T15:53:11.182993+0000","last_fullsized":"2023-08-11T15:53:11.182993+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:43:20.589597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106710+0000","last_change":"2023-08-11T15:53:09.780023+0000","last_active":"2023-08-11T15:53:11.106710+0000","last_peered":"2023-08-11T15:53:11.106710+0000","last_clean":"2023-08-11T15:53:11.106710+0000","last_became_active":"2023-08-11T15:53:09.779747+0000","last_became_peered":"2023-08-11T15:53:09.779747+0000","last_unstale":"2023-08-11T15:53:11.106710+0000","last_undegraded":"2023-08-11T15:53:11.106710+0000","last_fullsized":"2023-08-11T15:53:11.106710+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:24.779884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181260+0000","last_change":"2023-08-11T15:53:09.785222+0000","last_active":"2023-08-11T15:53:11.181260+0000","last_peered":"2023-08-11T15:53:11.181260+0000","last_clean":"2023-08-11T15:53:11.181260+0000","last_became_active":"2023-08-11T15:53:09.781496+0000","last_became_peered":"2023-08-11T15:53:09.781496+0000","last_unstale":"2023-08-11T15:53:11.181260+0000","last_undegraded":"2023-08-11T15:53:11.181260+0000","last_fullsized":"2023-08-11T15:53:11.181260+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:18.503920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751658+0000","last_change":"2023-08-11T15:53:09.762513+0000","last_active":"2023-08-11T15:53:10.751658+0000","last_peered":"2023-08-11T15:53:10.751658+0000","last_clean":"2023-08-11T15:53:10.751658+0000","last_became_active":"2023-08-11T15:53:09.762138+0000","last_became_peered":"2023-08-11T15:53:09.762138+0000","last_unstale":"2023-08-11T15:53:10.751658+0000","last_undegraded":"2023-08-11T15:53:10.751658+0000","last_fullsized":"2023-08-11T15:53:10.751658+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:37:28.533073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751940+0000","last_change":"2023-08-11T15:53:09.762892+0000","last_active":"2023-08-11T15:53:10.751940+0000","last_peered":"2023-08-11T15:53:10.751940+0000","last_clean":"2023-08-11T15:53:10.751940+0000","last_became_active":"2023-08-11T15:53:09.762422+0000","last_became_peered":"2023-08-11T15:53:09.762422+0000","last_unstale":"2023-08-11T15:53:10.751940+0000","last_undegraded":"2023-08-11T15:53:10.751940+0000","last_fullsized":"2023-08-11T15:53:10.751940+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:09:16.571916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753257+0000","last_change":"2023-08-11T15:53:09.768901+0000","last_active":"2023-08-11T15:53:10.753257+0000","last_peered":"2023-08-11T15:53:10.753257+0000","last_clean":"2023-08-11T15:53:10.753257+0000","last_became_active":"2023-08-11T15:53:09.768592+0000","last_became_peered":"2023-08-11T15:53:09.768592+0000","last_unstale":"2023-08-11T15:53:10.753257+0000","last_undegraded":"2023-08-11T15:53:10.753257+0000","last_fullsized":"2023-08-11T15:53:10.753257+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:22:57.491029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182215+0000","last_change":"2023-08-11T15:53:09.776020+0000","last_active":"2023-08-11T15:53:11.182215+0000","last_peered":"2023-08-11T15:53:11.182215+0000","last_clean":"2023-08-11T15:53:11.182215+0000","last_became_active":"2023-08-11T15:53:09.775742+0000","last_became_peered":"2023-08-11T15:53:09.775742+0000","last_unstale":"2023-08-11T15:53:11.182215+0000","last_undegraded":"2023-08-11T15:53:11.182215+0000","last_fullsized":"2023-08-11T15:53:11.182215+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:34:06.466622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753967+0000","last_change":"2023-08-11T15:53:09.771280+0000","last_active":"2023-08-11T15:53:10.753967+0000","last_peered":"2023-08-11T15:53:10.753967+0000","last_clean":"2023-08-11T15:53:10.753967+0000","last_became_active":"2023-08-11T15:53:09.770977+0000","last_became_peered":"2023-08-11T15:53:09.770977+0000","last_unstale":"2023-08-11T15:53:10.753967+0000","last_undegraded":"2023-08-11T15:53:10.753967+0000","last_fullsized":"2023-08-11T15:53:10.753967+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:06.295544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753323+0000","last_change":"2023-08-11T15:53:09.767829+0000","last_active":"2023-08-11T15:53:10.753323+0000","last_peered":"2023-08-11T15:53:10.753323+0000","last_clean":"2023-08-11T15:53:10.753323+0000","last_became_active":"2023-08-11T15:53:09.767503+0000","last_became_peered":"2023-08-11T15:53:09.767503+0000","last_unstale":"2023-08-11T15:53:10.753323+0000","last_undegraded":"2023-08-11T15:53:10.753323+0000","last_fullsized":"2023-08-11T15:53:10.753323+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:37:22.863178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752475+0000","last_change":"2023-08-11T15:53:09.768971+0000","last_active":"2023-08-11T15:53:10.752475+0000","last_peered":"2023-08-11T15:53:10.752475+0000","last_clean":"2023-08-11T15:53:10.752475+0000","last_became_active":"2023-08-11T15:53:09.768669+0000","last_became_peered":"2023-08-11T15:53:09.768669+0000","last_unstale":"2023-08-11T15:53:10.752475+0000","last_undegraded":"2023-08-11T15:53:10.752475+0000","last_fullsized":"2023-08-11T15:53:10.752475+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:01.051346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750777+0000","last_change":"2023-08-11T15:53:09.773565+0000","last_active":"2023-08-11T15:53:10.750777+0000","last_peered":"2023-08-11T15:53:10.750777+0000","last_clean":"2023-08-11T15:53:10.750777+0000","last_became_active":"2023-08-11T15:53:09.772543+0000","last_became_peered":"2023-08-11T15:53:09.772543+0000","last_unstale":"2023-08-11T15:53:10.750777+0000","last_undegraded":"2023-08-11T15:53:10.750777+0000","last_fullsized":"2023-08-11T15:53:10.750777+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:28:45.132262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.105874+0000","last_change":"2023-08-11T15:53:09.771905+0000","last_active":"2023-08-11T15:53:11.105874+0000","last_peered":"2023-08-11T15:53:11.105874+0000","last_clean":"2023-08-11T15:53:11.105874+0000","last_became_active":"2023-08-11T15:53:09.771513+0000","last_became_peered":"2023-08-11T15:53:09.771513+0000","last_unstale":"2023-08-11T15:53:11.105874+0000","last_undegraded":"2023-08-11T15:53:11.105874+0000","last_fullsized":"2023-08-11T15:53:11.105874+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:14:28.311384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180100+0000","last_change":"2023-08-11T15:53:09.769904+0000","last_active":"2023-08-11T15:53:11.180100+0000","last_peered":"2023-08-11T15:53:11.180100+0000","last_clean":"2023-08-11T15:53:11.180100+0000","last_became_active":"2023-08-11T15:53:09.769477+0000","last_became_peered":"2023-08-11T15:53:09.769477+0000","last_unstale":"2023-08-11T15:53:11.180100+0000","last_undegraded":"2023-08-11T15:53:11.180100+0000","last_fullsized":"2023-08-11T15:53:11.180100+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:30:25.907270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.178486+0000","last_change":"2023-08-11T15:53:09.776091+0000","last_active":"2023-08-11T15:53:11.178486+0000","last_peered":"2023-08-11T15:53:11.178486+0000","last_clean":"2023-08-11T15:53:11.178486+0000","last_became_active":"2023-08-11T15:53:09.775817+0000","last_became_peered":"2023-08-11T15:53:09.775817+0000","last_unstale":"2023-08-11T15:53:11.178486+0000","last_undegraded":"2023-08-11T15:53:11.178486+0000","last_fullsized":"2023-08-11T15:53:11.178486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:03:49.496617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107302+0000","last_change":"2023-08-11T15:53:08.748593+0000","last_active":"2023-08-11T15:53:11.107302+0000","last_peered":"2023-08-11T15:53:11.107302+0000","last_clean":"2023-08-11T15:53:11.107302+0000","last_became_active":"2023-08-11T15:53:08.748357+0000","last_became_peered":"2023-08-11T15:53:08.748357+0000","last_unstale":"2023-08-11T15:53:11.107302+0000","last_undegraded":"2023-08-11T15:53:11.107302+0000","last_fullsized":"2023-08-11T15:53:11.107302+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:37:21.858059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753273+0000","last_change":"2023-08-11T15:53:09.772220+0000","last_active":"2023-08-11T15:53:10.753273+0000","last_peered":"2023-08-11T15:53:10.753273+0000","last_clean":"2023-08-11T15:53:10.753273+0000","last_became_active":"2023-08-11T15:53:09.771903+0000","last_became_peered":"2023-08-11T15:53:09.771903+0000","last_unstale":"2023-08-11T15:53:10.753273+0000","last_undegraded":"2023-08-11T15:53:10.753273+0000","last_fullsized":"2023-08-11T15:53:10.753273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:16:12.516630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754459+0000","last_change":"2023-08-11T15:53:08.748687+0000","last_active":"2023-08-11T15:53:10.754459+0000","last_peered":"2023-08-11T15:53:10.754459+0000","last_clean":"2023-08-11T15:53:10.754459+0000","last_became_active":"2023-08-11T15:53:08.748363+0000","last_became_peered":"2023-08-11T15:53:08.748363+0000","last_unstale":"2023-08-11T15:53:10.754459+0000","last_undegraded":"2023-08-11T15:53:10.754459+0000","last_fullsized":"2023-08-11T15:53:10.754459+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:37.020312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753432+0000","last_change":"2023-08-11T15:53:09.774231+0000","last_active":"2023-08-11T15:53:10.753432+0000","last_peered":"2023-08-11T15:53:10.753432+0000","last_clean":"2023-08-11T15:53:10.753432+0000","last_became_active":"2023-08-11T15:53:09.773949+0000","last_became_peered":"2023-08-11T15:53:09.773949+0000","last_unstale":"2023-08-11T15:53:10.753432+0000","last_undegraded":"2023-08-11T15:53:10.753432+0000","last_fullsized":"2023-08-11T15:53:10.753432+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:09:03.177518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.179654+0000","last_change":"2023-08-11T15:53:08.740718+0000","last_active":"2023-08-11T15:53:11.179654+0000","last_peered":"2023-08-11T15:53:11.179654+0000","last_clean":"2023-08-11T15:53:11.179654+0000","last_became_active":"2023-08-11T15:53:08.740379+0000","last_became_peered":"2023-08-11T15:53:08.740379+0000","last_unstale":"2023-08-11T15:53:11.179654+0000","last_undegraded":"2023-08-11T15:53:11.179654+0000","last_fullsized":"2023-08-11T15:53:11.179654+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:06:51.600650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.179484+0000","last_change":"2023-08-11T15:53:09.766544+0000","last_active":"2023-08-11T15:53:11.179484+0000","last_peered":"2023-08-11T15:53:11.179484+0000","last_clean":"2023-08-11T15:53:11.179484+0000","last_became_active":"2023-08-11T15:53:09.766107+0000","last_became_peered":"2023-08-11T15:53:09.766107+0000","last_unstale":"2023-08-11T15:53:11.179484+0000","last_undegraded":"2023-08-11T15:53:11.179484+0000","last_fullsized":"2023-08-11T15:53:11.179484+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:02:07.798605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180483+0000","last_change":"2023-08-11T15:53:08.742961+0000","last_active":"2023-08-11T15:53:11.180483+0000","last_peered":"2023-08-11T15:53:11.180483+0000","last_clean":"2023-08-11T15:53:11.180483+0000","last_became_active":"2023-08-11T15:53:08.742699+0000","last_became_peered":"2023-08-11T15:53:08.742699+0000","last_unstale":"2023-08-11T15:53:11.180483+0000","last_undegraded":"2023-08-11T15:53:11.180483+0000","last_fullsized":"2023-08-11T15:53:11.180483+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:50:03.246394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751701+0000","last_change":"2023-08-11T15:53:09.777301+0000","last_active":"2023-08-11T15:53:10.751701+0000","last_peered":"2023-08-11T15:53:10.751701+0000","last_clean":"2023-08-11T15:53:10.751701+0000","last_became_active":"2023-08-11T15:53:09.776921+0000","last_became_peered":"2023-08-11T15:53:09.776921+0000","last_unstale":"2023-08-11T15:53:10.751701+0000","last_undegraded":"2023-08-11T15:53:10.751701+0000","last_fullsized":"2023-08-11T15:53:10.751701+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:30:04.778864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107258+0000","last_change":"2023-08-11T15:53:08.745370+0000","last_active":"2023-08-11T15:53:11.107258+0000","last_peered":"2023-08-11T15:53:11.107258+0000","last_clean":"2023-08-11T15:53:11.107258+0000","last_became_active":"2023-08-11T15:53:08.745073+0000","last_became_peered":"2023-08-11T15:53:08.745073+0000","last_unstale":"2023-08-11T15:53:11.107258+0000","last_undegraded":"2023-08-11T15:53:11.107258+0000","last_fullsized":"2023-08-11T15:53:11.107258+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.334880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181516+0000","last_change":"2023-08-11T15:53:09.787372+0000","last_active":"2023-08-11T15:53:11.181516+0000","last_peered":"2023-08-11T15:53:11.181516+0000","last_clean":"2023-08-11T15:53:11.181516+0000","last_became_active":"2023-08-11T15:53:09.787086+0000","last_became_peered":"2023-08-11T15:53:09.787086+0000","last_unstale":"2023-08-11T15:53:11.181516+0000","last_undegraded":"2023-08-11T15:53:11.181516+0000","last_fullsized":"2023-08-11T15:53:11.181516+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:54:17.954707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752037+0000","last_change":"2023-08-11T15:53:08.744766+0000","last_active":"2023-08-11T15:53:10.752037+0000","last_peered":"2023-08-11T15:53:10.752037+0000","last_clean":"2023-08-11T15:53:10.752037+0000","last_became_active":"2023-08-11T15:53:08.743573+0000","last_became_peered":"2023-08-11T15:53:08.743573+0000","last_unstale":"2023-08-11T15:53:10.752037+0000","last_undegraded":"2023-08-11T15:53:10.752037+0000","last_fullsized":"2023-08-11T15:53:10.752037+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:32:10.006528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182304+0000","last_change":"2023-08-11T15:53:09.780061+0000","last_active":"2023-08-11T15:53:11.182304+0000","last_peered":"2023-08-11T15:53:11.182304+0000","last_clean":"2023-08-11T15:53:11.182304+0000","last_became_active":"2023-08-11T15:53:09.779628+0000","last_became_peered":"2023-08-11T15:53:09.779628+0000","last_unstale":"2023-08-11T15:53:11.182304+0000","last_undegraded":"2023-08-11T15:53:11.182304+0000","last_fullsized":"2023-08-11T15:53:11.182304+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:39:34.729202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750150+0000","last_change":"2023-08-11T15:53:08.740221+0000","last_active":"2023-08-11T15:53:10.750150+0000","last_peered":"2023-08-11T15:53:10.750150+0000","last_clean":"2023-08-11T15:53:10.750150+0000","last_became_active":"2023-08-11T15:53:08.739950+0000","last_became_peered":"2023-08-11T15:53:08.739950+0000","last_unstale":"2023-08-11T15:53:10.750150+0000","last_undegraded":"2023-08-11T15:53:10.750150+0000","last_fullsized":"2023-08-11T15:53:10.750150+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:50:03.246394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751661+0000","last_change":"2023-08-11T15:53:09.762734+0000","last_active":"2023-08-11T15:53:10.751661+0000","last_peered":"2023-08-11T15:53:10.751661+0000","last_clean":"2023-08-11T15:53:10.751661+0000","last_became_active":"2023-08-11T15:53:09.762126+0000","last_became_peered":"2023-08-11T15:53:09.762126+0000","last_unstale":"2023-08-11T15:53:10.751661+0000","last_undegraded":"2023-08-11T15:53:10.751661+0000","last_fullsized":"2023-08-11T15:53:10.751661+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:25:54.207324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182422+0000","last_change":"2023-08-11T15:53:08.737858+0000","last_active":"2023-08-11T15:53:11.182422+0000","last_peered":"2023-08-11T15:53:11.182422+0000","last_clean":"2023-08-11T15:53:11.182422+0000","last_became_active":"2023-08-11T15:53:08.737526+0000","last_became_peered":"2023-08-11T15:53:08.737526+0000","last_unstale":"2023-08-11T15:53:11.182422+0000","last_undegraded":"2023-08-11T15:53:11.182422+0000","last_fullsized":"2023-08-11T15:53:11.182422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:10:04.293967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754434+0000","last_change":"2023-08-11T15:53:09.790581+0000","last_active":"2023-08-11T15:53:10.754434+0000","last_peered":"2023-08-11T15:53:10.754434+0000","last_clean":"2023-08-11T15:53:10.754434+0000","last_became_active":"2023-08-11T15:53:09.790274+0000","last_became_peered":"2023-08-11T15:53:09.790274+0000","last_unstale":"2023-08-11T15:53:10.754434+0000","last_undegraded":"2023-08-11T15:53:10.754434+0000","last_fullsized":"2023-08-11T15:53:10.754434+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:56.289946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754796+0000","last_change":"2023-08-11T15:53:08.733462+0000","last_active":"2023-08-11T15:53:10.754796+0000","last_peered":"2023-08-11T15:53:10.754796+0000","last_clean":"2023-08-11T15:53:10.754796+0000","last_became_active":"2023-08-11T15:53:08.733010+0000","last_became_peered":"2023-08-11T15:53:08.733010+0000","last_unstale":"2023-08-11T15:53:10.754796+0000","last_undegraded":"2023-08-11T15:53:10.754796+0000","last_fullsized":"2023-08-11T15:53:10.754796+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:28:00.974391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180864+0000","last_change":"2023-08-11T15:53:09.767772+0000","last_active":"2023-08-11T15:53:11.180864+0000","last_peered":"2023-08-11T15:53:11.180864+0000","last_clean":"2023-08-11T15:53:11.180864+0000","last_became_active":"2023-08-11T15:53:09.767438+0000","last_became_peered":"2023-08-11T15:53:09.767438+0000","last_unstale":"2023-08-11T15:53:11.180864+0000","last_undegraded":"2023-08-11T15:53:11.180864+0000","last_fullsized":"2023-08-11T15:53:11.180864+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:20.339999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182940+0000","last_change":"2023-08-11T15:53:08.743600+0000","last_active":"2023-08-11T15:53:11.182940+0000","last_peered":"2023-08-11T15:53:11.182940+0000","last_clean":"2023-08-11T15:53:11.182940+0000","last_became_active":"2023-08-11T15:53:08.743263+0000","last_became_peered":"2023-08-11T15:53:08.743263+0000","last_unstale":"2023-08-11T15:53:11.182940+0000","last_undegraded":"2023-08-11T15:53:11.182940+0000","last_fullsized":"2023-08-11T15:53:11.182940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:39.683699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107177+0000","last_change":"2023-08-11T15:53:09.784595+0000","last_active":"2023-08-11T15:53:11.107177+0000","last_peered":"2023-08-11T15:53:11.107177+0000","last_clean":"2023-08-11T15:53:11.107177+0000","last_became_active":"2023-08-11T15:53:09.780448+0000","last_became_peered":"2023-08-11T15:53:09.780448+0000","last_unstale":"2023-08-11T15:53:11.107177+0000","last_undegraded":"2023-08-11T15:53:11.107177+0000","last_fullsized":"2023-08-11T15:53:11.107177+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:56:34.581615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180919+0000","last_change":"2023-08-11T15:53:08.750910+0000","last_active":"2023-08-11T15:53:11.180919+0000","last_peered":"2023-08-11T15:53:11.180919+0000","last_clean":"2023-08-11T15:53:11.180919+0000","last_became_active":"2023-08-11T15:53:08.750600+0000","last_became_peered":"2023-08-11T15:53:08.750600+0000","last_unstale":"2023-08-11T15:53:11.180919+0000","last_undegraded":"2023-08-11T15:53:11.180919+0000","last_fullsized":"2023-08-11T15:53:11.180919+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:09.213770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180279+0000","last_change":"2023-08-11T15:53:09.768466+0000","last_active":"2023-08-11T15:53:11.180279+0000","last_peered":"2023-08-11T15:53:11.180279+0000","last_clean":"2023-08-11T15:53:11.180279+0000","last_became_active":"2023-08-11T15:53:09.768172+0000","last_became_peered":"2023-08-11T15:53:09.768172+0000","last_unstale":"2023-08-11T15:53:11.180279+0000","last_undegraded":"2023-08-11T15:53:11.180279+0000","last_fullsized":"2023-08-11T15:53:11.180279+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:28:49.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181521+0000","last_change":"2023-08-11T15:53:08.734705+0000","last_active":"2023-08-11T15:53:11.181521+0000","last_peered":"2023-08-11T15:53:11.181521+0000","last_clean":"2023-08-11T15:53:11.181521+0000","last_became_active":"2023-08-11T15:53:08.734399+0000","last_became_peered":"2023-08-11T15:53:08.734399+0000","last_unstale":"2023-08-11T15:53:11.181521+0000","last_undegraded":"2023-08-11T15:53:11.181521+0000","last_fullsized":"2023-08-11T15:53:11.181521+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:48:12.780203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752030+0000","last_change":"2023-08-11T15:53:09.786330+0000","last_active":"2023-08-11T15:53:10.752030+0000","last_peered":"2023-08-11T15:53:10.752030+0000","last_clean":"2023-08-11T15:53:10.752030+0000","last_became_active":"2023-08-11T15:53:09.785968+0000","last_became_peered":"2023-08-11T15:53:09.785968+0000","last_unstale":"2023-08-11T15:53:10.752030+0000","last_undegraded":"2023-08-11T15:53:10.752030+0000","last_fullsized":"2023-08-11T15:53:10.752030+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:31:25.473751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107264+0000","last_change":"2023-08-11T15:53:08.741591+0000","last_active":"2023-08-11T15:53:11.107264+0000","last_peered":"2023-08-11T15:53:11.107264+0000","last_clean":"2023-08-11T15:53:11.107264+0000","last_became_active":"2023-08-11T15:53:08.741284+0000","last_became_peered":"2023-08-11T15:53:08.741284+0000","last_unstale":"2023-08-11T15:53:11.107264+0000","last_undegraded":"2023-08-11T15:53:11.107264+0000","last_fullsized":"2023-08-11T15:53:11.107264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:09.213770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181808+0000","last_change":"2023-08-11T15:53:09.780157+0000","last_active":"2023-08-11T15:53:11.181808+0000","last_peered":"2023-08-11T15:53:11.181808+0000","last_clean":"2023-08-11T15:53:11.181808+0000","last_became_active":"2023-08-11T15:53:09.779865+0000","last_became_peered":"2023-08-11T15:53:09.779865+0000","last_unstale":"2023-08-11T15:53:11.181808+0000","last_undegraded":"2023-08-11T15:53:11.181808+0000","last_fullsized":"2023-08-11T15:53:11.181808+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:33:56.983793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180527+0000","last_change":"2023-08-11T15:53:08.739293+0000","last_active":"2023-08-11T15:53:11.180527+0000","last_peered":"2023-08-11T15:53:11.180527+0000","last_clean":"2023-08-11T15:53:11.180527+0000","last_became_active":"2023-08-11T15:53:08.739017+0000","last_became_peered":"2023-08-11T15:53:08.739017+0000","last_unstale":"2023-08-11T15:53:11.180527+0000","last_undegraded":"2023-08-11T15:53:11.180527+0000","last_fullsized":"2023-08-11T15:53:11.180527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:23.774765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.105942+0000","last_change":"2023-08-11T15:53:09.783386+0000","last_active":"2023-08-11T15:53:11.105942+0000","last_peered":"2023-08-11T15:53:11.105942+0000","last_clean":"2023-08-11T15:53:11.105942+0000","last_became_active":"2023-08-11T15:53:09.783094+0000","last_became_peered":"2023-08-11T15:53:09.783094+0000","last_unstale":"2023-08-11T15:53:11.105942+0000","last_undegraded":"2023-08-11T15:53:11.105942+0000","last_fullsized":"2023-08-11T15:53:11.105942+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:05:14.964206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106001+0000","last_change":"2023-08-11T15:53:08.741032+0000","last_active":"2023-08-11T15:53:11.106001+0000","last_peered":"2023-08-11T15:53:11.106001+0000","last_clean":"2023-08-11T15:53:11.106001+0000","last_became_active":"2023-08-11T15:53:08.740718+0000","last_became_peered":"2023-08-11T15:53:08.740718+0000","last_unstale":"2023-08-11T15:53:11.106001+0000","last_undegraded":"2023-08-11T15:53:11.106001+0000","last_fullsized":"2023-08-11T15:53:11.106001+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:34.637074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180295+0000","last_change":"2023-08-11T15:53:09.762947+0000","last_active":"2023-08-11T15:53:11.180295+0000","last_peered":"2023-08-11T15:53:11.180295+0000","last_clean":"2023-08-11T15:53:11.180295+0000","last_became_active":"2023-08-11T15:53:09.762608+0000","last_became_peered":"2023-08-11T15:53:09.762608+0000","last_unstale":"2023-08-11T15:53:11.180295+0000","last_undegraded":"2023-08-11T15:53:11.180295+0000","last_fullsized":"2023-08-11T15:53:11.180295+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:49.301086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182550+0000","last_change":"2023-08-11T15:53:08.744714+0000","last_active":"2023-08-11T15:53:11.182550+0000","last_peered":"2023-08-11T15:53:11.182550+0000","last_clean":"2023-08-11T15:53:11.182550+0000","last_became_active":"2023-08-11T15:53:08.744259+0000","last_became_peered":"2023-08-11T15:53:08.744259+0000","last_unstale":"2023-08-11T15:53:11.182550+0000","last_undegraded":"2023-08-11T15:53:11.182550+0000","last_fullsized":"2023-08-11T15:53:11.182550+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:06:08.293733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182557+0000","last_change":"2023-08-11T15:53:09.764295+0000","last_active":"2023-08-11T15:53:11.182557+0000","last_peered":"2023-08-11T15:53:11.182557+0000","last_clean":"2023-08-11T15:53:11.182557+0000","last_became_active":"2023-08-11T15:53:09.763798+0000","last_became_peered":"2023-08-11T15:53:09.763798+0000","last_unstale":"2023-08-11T15:53:11.182557+0000","last_undegraded":"2023-08-11T15:53:11.182557+0000","last_fullsized":"2023-08-11T15:53:11.182557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:56.000165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182381+0000","last_change":"2023-08-11T15:53:08.734098+0000","last_active":"2023-08-11T15:53:11.182381+0000","last_peered":"2023-08-11T15:53:11.182381+0000","last_clean":"2023-08-11T15:53:11.182381+0000","last_became_active":"2023-08-11T15:53:08.733793+0000","last_became_peered":"2023-08-11T15:53:08.733793+0000","last_unstale":"2023-08-11T15:53:11.182381+0000","last_undegraded":"2023-08-11T15:53:11.182381+0000","last_fullsized":"2023-08-11T15:53:11.182381+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:06:41.748325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.183161+0000","last_change":"2023-08-11T15:53:09.790589+0000","last_active":"2023-08-11T15:53:11.183161+0000","last_peered":"2023-08-11T15:53:11.183161+0000","last_clean":"2023-08-11T15:53:11.183161+0000","last_became_active":"2023-08-11T15:53:09.790277+0000","last_became_peered":"2023-08-11T15:53:09.790277+0000","last_unstale":"2023-08-11T15:53:11.183161+0000","last_undegraded":"2023-08-11T15:53:11.183161+0000","last_fullsized":"2023-08-11T15:53:11.183161+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:52:13.686728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754864+0000","last_change":"2023-08-11T15:53:08.739707+0000","last_active":"2023-08-11T15:53:10.754864+0000","last_peered":"2023-08-11T15:53:10.754864+0000","last_clean":"2023-08-11T15:53:10.754864+0000","last_became_active":"2023-08-11T15:53:08.735001+0000","last_became_peered":"2023-08-11T15:53:08.735001+0000","last_unstale":"2023-08-11T15:53:10.754864+0000","last_undegraded":"2023-08-11T15:53:10.754864+0000","last_fullsized":"2023-08-11T15:53:10.754864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:37:42.368869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752790+0000","last_change":"2023-08-11T15:53:09.767296+0000","last_active":"2023-08-11T15:53:10.752790+0000","last_peered":"2023-08-11T15:53:10.752790+0000","last_clean":"2023-08-11T15:53:10.752790+0000","last_became_active":"2023-08-11T15:53:09.766939+0000","last_became_peered":"2023-08-11T15:53:09.766939+0000","last_unstale":"2023-08-11T15:53:10.752790+0000","last_undegraded":"2023-08-11T15:53:10.752790+0000","last_fullsized":"2023-08-11T15:53:10.752790+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:35:59.980509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752395+0000","last_change":"2023-08-11T15:53:08.744460+0000","last_active":"2023-08-11T15:53:10.752395+0000","last_peered":"2023-08-11T15:53:10.752395+0000","last_clean":"2023-08-11T15:53:10.752395+0000","last_became_active":"2023-08-11T15:53:08.744011+0000","last_became_peered":"2023-08-11T15:53:08.744011+0000","last_unstale":"2023-08-11T15:53:10.752395+0000","last_undegraded":"2023-08-11T15:53:10.752395+0000","last_fullsized":"2023-08-11T15:53:10.752395+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.334880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752914+0000","last_change":"2023-08-11T15:53:09.778529+0000","last_active":"2023-08-11T15:53:10.752914+0000","last_peered":"2023-08-11T15:53:10.752914+0000","last_clean":"2023-08-11T15:53:10.752914+0000","last_became_active":"2023-08-11T15:53:09.778176+0000","last_became_peered":"2023-08-11T15:53:09.778176+0000","last_unstale":"2023-08-11T15:53:10.752914+0000","last_undegraded":"2023-08-11T15:53:10.752914+0000","last_fullsized":"2023-08-11T15:53:10.752914+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:43:49.491851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107099+0000","last_change":"2023-08-11T15:53:08.748386+0000","last_active":"2023-08-11T15:53:11.107099+0000","last_peered":"2023-08-11T15:53:11.107099+0000","last_clean":"2023-08-11T15:53:11.107099+0000","last_became_active":"2023-08-11T15:53:08.748054+0000","last_became_peered":"2023-08-11T15:53:08.748054+0000","last_unstale":"2023-08-11T15:53:11.107099+0000","last_undegraded":"2023-08-11T15:53:11.107099+0000","last_fullsized":"2023-08-11T15:53:11.107099+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:48.804896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106108+0000","last_change":"2023-08-11T15:53:09.779061+0000","last_active":"2023-08-11T15:53:11.106108+0000","last_peered":"2023-08-11T15:53:11.106108+0000","last_clean":"2023-08-11T15:53:11.106108+0000","last_became_active":"2023-08-11T15:53:09.778786+0000","last_became_peered":"2023-08-11T15:53:09.778786+0000","last_unstale":"2023-08-11T15:53:11.106108+0000","last_undegraded":"2023-08-11T15:53:11.106108+0000","last_fullsized":"2023-08-11T15:53:11.106108+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:49.301086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107674+0000","last_change":"2023-08-11T15:53:08.744974+0000","last_active":"2023-08-11T15:53:11.107674+0000","last_peered":"2023-08-11T15:53:11.107674+0000","last_clean":"2023-08-11T15:53:11.107674+0000","last_became_active":"2023-08-11T15:53:08.744738+0000","last_became_peered":"2023-08-11T15:53:08.744738+0000","last_unstale":"2023-08-11T15:53:11.107674+0000","last_undegraded":"2023-08-11T15:53:11.107674+0000","last_fullsized":"2023-08-11T15:53:11.107674+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:09:02.172399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.179121+0000","last_change":"2023-08-11T15:53:09.769152+0000","last_active":"2023-08-11T15:53:11.179121+0000","last_peered":"2023-08-11T15:53:11.179121+0000","last_clean":"2023-08-11T15:53:11.179121+0000","last_became_active":"2023-08-11T15:53:09.768472+0000","last_became_peered":"2023-08-11T15:53:09.768472+0000","last_unstale":"2023-08-11T15:53:11.179121+0000","last_undegraded":"2023-08-11T15:53:11.179121+0000","last_fullsized":"2023-08-11T15:53:11.179121+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:31:44.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750123+0000","last_change":"2023-08-11T15:53:08.750963+0000","last_active":"2023-08-11T15:53:10.750123+0000","last_peered":"2023-08-11T15:53:10.750123+0000","last_clean":"2023-08-11T15:53:10.750123+0000","last_became_active":"2023-08-11T15:53:08.750546+0000","last_became_peered":"2023-08-11T15:53:08.750546+0000","last_unstale":"2023-08-11T15:53:10.750123+0000","last_undegraded":"2023-08-11T15:53:10.750123+0000","last_fullsized":"2023-08-11T15:53:10.750123+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:11.446060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180238+0000","last_change":"2023-08-11T15:53:09.766778+0000","last_active":"2023-08-11T15:53:11.180238+0000","last_peered":"2023-08-11T15:53:11.180238+0000","last_clean":"2023-08-11T15:53:11.180238+0000","last_became_active":"2023-08-11T15:53:09.766457+0000","last_became_peered":"2023-08-11T15:53:09.766457+0000","last_unstale":"2023-08-11T15:53:11.180238+0000","last_undegraded":"2023-08-11T15:53:11.180238+0000","last_fullsized":"2023-08-11T15:53:11.180238+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:59:07.514405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180854+0000","last_change":"2023-08-11T15:53:08.745022+0000","last_active":"2023-08-11T15:53:11.180854+0000","last_peered":"2023-08-11T15:53:11.180854+0000","last_clean":"2023-08-11T15:53:11.180854+0000","last_became_active":"2023-08-11T15:53:08.741244+0000","last_became_peered":"2023-08-11T15:53:08.741244+0000","last_unstale":"2023-08-11T15:53:11.180854+0000","last_undegraded":"2023-08-11T15:53:11.180854+0000","last_fullsized":"2023-08-11T15:53:11.180854+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:53:43.175908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180730+0000","last_change":"2023-08-11T15:53:09.783571+0000","last_active":"2023-08-11T15:53:11.180730+0000","last_peered":"2023-08-11T15:53:11.180730+0000","last_clean":"2023-08-11T15:53:11.180730+0000","last_became_active":"2023-08-11T15:53:09.783194+0000","last_became_peered":"2023-08-11T15:53:09.783194+0000","last_unstale":"2023-08-11T15:53:11.180730+0000","last_undegraded":"2023-08-11T15:53:11.180730+0000","last_fullsized":"2023-08-11T15:53:11.180730+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:14:23.224719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180069+0000","last_change":"2023-08-11T15:53:08.738754+0000","last_active":"2023-08-11T15:53:11.180069+0000","last_peered":"2023-08-11T15:53:11.180069+0000","last_clean":"2023-08-11T15:53:11.180069+0000","last_became_active":"2023-08-11T15:53:08.738462+0000","last_became_peered":"2023-08-11T15:53:08.738462+0000","last_unstale":"2023-08-11T15:53:11.180069+0000","last_undegraded":"2023-08-11T15:53:11.180069+0000","last_fullsized":"2023-08-11T15:53:11.180069+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:34.637074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106414+0000","last_change":"2023-08-11T15:53:09.790391+0000","last_active":"2023-08-11T15:53:11.106414+0000","last_peered":"2023-08-11T15:53:11.106414+0000","last_clean":"2023-08-11T15:53:11.106414+0000","last_became_active":"2023-08-11T15:53:09.790046+0000","last_became_peered":"2023-08-11T15:53:09.790046+0000","last_unstale":"2023-08-11T15:53:11.106414+0000","last_undegraded":"2023-08-11T15:53:11.106414+0000","last_fullsized":"2023-08-11T15:53:11.106414+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:12.451179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180720+0000","last_change":"2023-08-11T15:53:08.751384+0000","last_active":"2023-08-11T15:53:11.180720+0000","last_peered":"2023-08-11T15:53:11.180720+0000","last_clean":"2023-08-11T15:53:11.180720+0000","last_became_active":"2023-08-11T15:53:08.751062+0000","last_became_peered":"2023-08-11T15:53:08.751062+0000","last_unstale":"2023-08-11T15:53:11.180720+0000","last_undegraded":"2023-08-11T15:53:11.180720+0000","last_fullsized":"2023-08-11T15:53:11.180720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:56:33.576496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753539+0000","last_change":"2023-08-11T15:50:54.667601+0000","last_active":"2023-08-11T15:53:10.753539+0000","last_peered":"2023-08-11T15:53:10.753539+0000","last_clean":"2023-08-11T15:53:10.753539+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:53:10.753539+0000","last_undegraded":"2023-08-11T15:53:10.753539+0000","last_fullsized":"2023-08-11T15:53:10.753539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:28:24.398965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106698+0000","last_change":"2023-08-11T15:53:09.781908+0000","last_active":"2023-08-11T15:53:11.106698+0000","last_peered":"2023-08-11T15:53:11.106698+0000","last_clean":"2023-08-11T15:53:11.106698+0000","last_became_active":"2023-08-11T15:53:09.781535+0000","last_became_peered":"2023-08-11T15:53:09.781535+0000","last_unstale":"2023-08-11T15:53:11.106698+0000","last_undegraded":"2023-08-11T15:53:11.106698+0000","last_fullsized":"2023-08-11T15:53:11.106698+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:50:04.251513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752492+0000","last_change":"2023-08-11T15:53:08.740595+0000","last_active":"2023-08-11T15:53:10.752492+0000","last_peered":"2023-08-11T15:53:10.752492+0000","last_clean":"2023-08-11T15:53:10.752492+0000","last_became_active":"2023-08-11T15:53:08.734795+0000","last_became_peered":"2023-08-11T15:53:08.734795+0000","last_unstale":"2023-08-11T15:53:10.752492+0000","last_undegraded":"2023-08-11T15:53:10.752492+0000","last_fullsized":"2023-08-11T15:53:10.752492+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:28:51.472838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.178669+0000","last_change":"2023-08-11T15:50:54.662968+0000","last_active":"2023-08-11T15:53:11.178669+0000","last_peered":"2023-08-11T15:53:11.178669+0000","last_clean":"2023-08-11T15:53:11.178669+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:53:11.178669+0000","last_undegraded":"2023-08-11T15:53:11.178669+0000","last_fullsized":"2023-08-11T15:53:11.178669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:21:13.932510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.749850+0000","last_change":"2023-08-11T15:53:09.773008+0000","last_active":"2023-08-11T15:53:10.749850+0000","last_peered":"2023-08-11T15:53:10.749850+0000","last_clean":"2023-08-11T15:53:10.749850+0000","last_became_active":"2023-08-11T15:53:09.772271+0000","last_became_peered":"2023-08-11T15:53:09.772271+0000","last_unstale":"2023-08-11T15:53:10.749850+0000","last_undegraded":"2023-08-11T15:53:10.749850+0000","last_fullsized":"2023-08-11T15:53:10.749850+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:49.301086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.178894+0000","last_change":"2023-08-11T15:53:08.747721+0000","last_active":"2023-08-11T15:53:11.178894+0000","last_peered":"2023-08-11T15:53:11.178894+0000","last_clean":"2023-08-11T15:53:11.178894+0000","last_became_active":"2023-08-11T15:53:08.747339+0000","last_became_peered":"2023-08-11T15:53:08.747339+0000","last_unstale":"2023-08-11T15:53:11.178894+0000","last_undegraded":"2023-08-11T15:53:11.178894+0000","last_fullsized":"2023-08-11T15:53:11.178894+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:09:56.665706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181034+0000","last_change":"2023-08-11T15:50:55.272783+0000","last_active":"2023-08-11T15:53:11.181034+0000","last_peered":"2023-08-11T15:53:11.181034+0000","last_clean":"2023-08-11T15:53:11.181034+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:53:11.181034+0000","last_undegraded":"2023-08-11T15:53:11.181034+0000","last_fullsized":"2023-08-11T15:53:11.181034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:06:48.106587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750996+0000","last_change":"2023-08-11T15:53:09.778080+0000","last_active":"2023-08-11T15:53:10.750996+0000","last_peered":"2023-08-11T15:53:10.750996+0000","last_clean":"2023-08-11T15:53:10.750996+0000","last_became_active":"2023-08-11T15:53:09.777754+0000","last_became_peered":"2023-08-11T15:53:09.777754+0000","last_unstale":"2023-08-11T15:53:10.750996+0000","last_undegraded":"2023-08-11T15:53:10.750996+0000","last_fullsized":"2023-08-11T15:53:10.750996+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:30:40.840167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182680+0000","last_change":"2023-08-11T15:53:08.737070+0000","last_active":"2023-08-11T15:53:11.182680+0000","last_peered":"2023-08-11T15:53:11.182680+0000","last_clean":"2023-08-11T15:53:11.182680+0000","last_became_active":"2023-08-11T15:53:08.736773+0000","last_became_peered":"2023-08-11T15:53:08.736773+0000","last_unstale":"2023-08-11T15:53:11.182680+0000","last_undegraded":"2023-08-11T15:53:11.182680+0000","last_fullsized":"2023-08-11T15:53:11.182680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:43:19.728568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.178275+0000","last_change":"2023-08-11T15:50:55.270974+0000","last_active":"2023-08-11T15:53:11.178275+0000","last_peered":"2023-08-11T15:53:11.178275+0000","last_clean":"2023-08-11T15:53:11.178275+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:53:11.178275+0000","last_undegraded":"2023-08-11T15:53:11.178275+0000","last_fullsized":"2023-08-11T15:53:11.178275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:27:00.317635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753641+0000","last_change":"2023-08-11T15:53:09.786536+0000","last_active":"2023-08-11T15:53:10.753641+0000","last_peered":"2023-08-11T15:53:10.753641+0000","last_clean":"2023-08-11T15:53:10.753641+0000","last_became_active":"2023-08-11T15:53:09.786189+0000","last_became_peered":"2023-08-11T15:53:09.786189+0000","last_unstale":"2023-08-11T15:53:10.753641+0000","last_undegraded":"2023-08-11T15:53:10.753641+0000","last_fullsized":"2023-08-11T15:53:10.753641+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:46:31.671075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106982+0000","last_change":"2023-08-11T15:53:08.743492+0000","last_active":"2023-08-11T15:53:11.106982+0000","last_peered":"2023-08-11T15:53:11.106982+0000","last_clean":"2023-08-11T15:53:11.106982+0000","last_became_active":"2023-08-11T15:53:08.743162+0000","last_became_peered":"2023-08-11T15:53:08.743162+0000","last_unstale":"2023-08-11T15:53:11.106982+0000","last_undegraded":"2023-08-11T15:53:11.106982+0000","last_fullsized":"2023-08-11T15:53:11.106982+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:50:10.082403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750591+0000","last_change":"2023-08-11T15:50:54.660862+0000","last_active":"2023-08-11T15:53:10.750591+0000","last_peered":"2023-08-11T15:53:10.750591+0000","last_clean":"2023-08-11T15:53:10.750591+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:53:10.750591+0000","last_undegraded":"2023-08-11T15:53:10.750591+0000","last_fullsized":"2023-08-11T15:53:10.750591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:22:09.708953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181772+0000","last_change":"2023-08-11T15:53:09.786440+0000","last_active":"2023-08-11T15:53:11.181772+0000","last_peered":"2023-08-11T15:53:11.181772+0000","last_clean":"2023-08-11T15:53:11.181772+0000","last_became_active":"2023-08-11T15:53:09.786031+0000","last_became_peered":"2023-08-11T15:53:09.786031+0000","last_unstale":"2023-08-11T15:53:11.181772+0000","last_undegraded":"2023-08-11T15:53:11.181772+0000","last_fullsized":"2023-08-11T15:53:11.181772+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:58:29.111103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181041+0000","last_change":"2023-08-11T15:53:08.750894+0000","last_active":"2023-08-11T15:53:11.181041+0000","last_peered":"2023-08-11T15:53:11.181041+0000","last_clean":"2023-08-11T15:53:11.181041+0000","last_became_active":"2023-08-11T15:53:08.750562+0000","last_became_peered":"2023-08-11T15:53:08.750562+0000","last_unstale":"2023-08-11T15:53:11.181041+0000","last_undegraded":"2023-08-11T15:53:11.181041+0000","last_fullsized":"2023-08-11T15:53:11.181041+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:37:21.858059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180151+0000","last_change":"2023-08-11T15:50:54.660575+0000","last_active":"2023-08-11T15:53:11.180151+0000","last_peered":"2023-08-11T15:53:11.180151+0000","last_clean":"2023-08-11T15:53:11.180151+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:53:11.180151+0000","last_undegraded":"2023-08-11T15:53:11.180151+0000","last_fullsized":"2023-08-11T15:53:11.180151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:13:57.717733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180705+0000","last_change":"2023-08-11T15:53:09.768911+0000","last_active":"2023-08-11T15:53:11.180705+0000","last_peered":"2023-08-11T15:53:11.180705+0000","last_clean":"2023-08-11T15:53:11.180705+0000","last_became_active":"2023-08-11T15:53:09.768610+0000","last_became_peered":"2023-08-11T15:53:09.768610+0000","last_unstale":"2023-08-11T15:53:11.180705+0000","last_undegraded":"2023-08-11T15:53:11.180705+0000","last_fullsized":"2023-08-11T15:53:11.180705+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:49.810015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.178189+0000","last_change":"2023-08-11T15:53:08.743107+0000","last_active":"2023-08-11T15:53:11.178189+0000","last_peered":"2023-08-11T15:53:11.178189+0000","last_clean":"2023-08-11T15:53:11.178189+0000","last_became_active":"2023-08-11T15:53:08.742738+0000","last_became_peered":"2023-08-11T15:53:08.742738+0000","last_unstale":"2023-08-11T15:53:11.178189+0000","last_undegraded":"2023-08-11T15:53:11.178189+0000","last_fullsized":"2023-08-11T15:53:11.178189+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:53:03.427605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.179866+0000","last_change":"2023-08-11T15:50:54.669675+0000","last_active":"2023-08-11T15:53:11.179866+0000","last_peered":"2023-08-11T15:53:11.179866+0000","last_clean":"2023-08-11T15:53:11.179866+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:53:11.179866+0000","last_undegraded":"2023-08-11T15:53:11.179866+0000","last_fullsized":"2023-08-11T15:53:11.179866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:02:59.893275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182688+0000","last_change":"2023-08-11T15:53:09.788950+0000","last_active":"2023-08-11T15:53:11.182688+0000","last_peered":"2023-08-11T15:53:11.182688+0000","last_clean":"2023-08-11T15:53:11.182688+0000","last_became_active":"2023-08-11T15:53:09.785438+0000","last_became_peered":"2023-08-11T15:53:09.785438+0000","last_unstale":"2023-08-11T15:53:11.182688+0000","last_undegraded":"2023-08-11T15:53:11.182688+0000","last_fullsized":"2023-08-11T15:53:11.182688+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:09:24.668922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754046+0000","last_change":"2023-08-11T15:53:08.743904+0000","last_active":"2023-08-11T15:53:10.754046+0000","last_peered":"2023-08-11T15:53:10.754046+0000","last_clean":"2023-08-11T15:53:10.754046+0000","last_became_active":"2023-08-11T15:53:08.743585+0000","last_became_peered":"2023-08-11T15:53:08.743585+0000","last_unstale":"2023-08-11T15:53:10.754046+0000","last_undegraded":"2023-08-11T15:53:10.754046+0000","last_fullsized":"2023-08-11T15:53:10.754046+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:43:35.240762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751222+0000","last_change":"2023-08-11T15:50:54.663730+0000","last_active":"2023-08-11T15:53:10.751222+0000","last_peered":"2023-08-11T15:53:10.751222+0000","last_clean":"2023-08-11T15:53:10.751222+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:53:10.751222+0000","last_undegraded":"2023-08-11T15:53:10.751222+0000","last_fullsized":"2023-08-11T15:53:10.751222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:54:19.510684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180453+0000","last_change":"2023-08-11T15:50:48.426393+0000","last_active":"2023-08-11T15:53:11.180453+0000","last_peered":"2023-08-11T15:53:11.180453+0000","last_clean":"2023-08-11T15:53:11.180453+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:53:11.180453+0000","last_undegraded":"2023-08-11T15:53:11.180453+0000","last_fullsized":"2023-08-11T15:53:11.180453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_clean_scrub_stamp":"2023-08-11T15:50:47.133560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:47:49.518872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1081344,"data_stored":805556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.093,"15min":1.1160000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.042},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.216,"15min":1.222},"min":{"1min":0.80900000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5509999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.98699999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3129999999999999,"15min":1.327},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.7370000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.333,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.091}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.177,"15min":1.175},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.232,"5min":1.254,"15min":1.258},"min":{"1min":0.87,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5169999999999999,"5min":2.02,"15min":2.02},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2909999999999999,"15min":1.3069999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.671,"5min":1.869,"15min":1.869},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4890000000000001,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.86199999999999999}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4219999999999999,"15min":1.4279999999999999},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.885,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.181},{"interface":"front","average":{"1min":1.282,"5min":1.395,"15min":1.4139999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6910000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.2410000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.373,"15min":1.381},"min":{"1min":0.93500000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5449999999999999,"15min":1.5509999999999999},"min":{"1min":1.2170000000000001,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":1.9330000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.286}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.5029999999999999,"15min":1.5209999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.768,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.544,"15min":1.5580000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.861,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.3999999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542177,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":222743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0609999999999999,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1339999999999999,"15min":1.1359999999999999},"min":{"1min":0.751,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.115,"15min":1.1140000000000001},"min":{"1min":0.73599999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.591,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.878},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.66100000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.0820000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2390000000000001,"15min":1.24},"min":{"1min":0.85599999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.3180000000000001,"15min":1.3400000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5600000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.327}]},{"osd":3,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.369,"15min":1.377},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.77,"5min":2.774,"15min":2.774},"last":1.145},{"interface":"front","average":{"1min":1.282,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7330000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.575}]},{"osd":4,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.319,"15min":1.327},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8879999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.468,"15min":1.4910000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.665,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.518}]},{"osd":5,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.496,"15min":1.5069999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8149999999999999,"5min":3.371,"15min":3.371},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.486,"15min":1.504},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0009999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.2789999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.615,"15min":1.6479999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.762,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.6739999999999999,"15min":1.704},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.921,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.752}]}]},{"osd":5,"up_from":14,"seq":60129542178,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":222141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0389999999999999,"15min":1.0269999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.077},{"interface":"front","average":{"1min":1.073,"5min":1.1499999999999999,"15min":1.163},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.133}]},{"osd":1,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1419999999999999,"15min":1.1399999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.93999999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.621,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.516}]},{"osd":2,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.262,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.615,"5min":1.827,"15min":1.827},"last":1.482},{"interface":"front","average":{"1min":1.194,"5min":1.351,"15min":1.377},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8300000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.3919999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.353,"15min":1.353},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.702,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.292},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.444},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.782,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.339}]},{"osd":4,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4379999999999999,"15min":1.466},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8779999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5880000000000001,"15min":1.6080000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.847,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.96499999999999997}]},{"osd":6,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.4690000000000001,"15min":1.492},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.796,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.6120000000000001,"15min":1.6379999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.2160000000000002,"5min":3.9870000000000001,"15min":3.9870000000000001},"last":1.6759999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.5820000000000001,"15min":1.605},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9990000000000001,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.637,"15min":1.677},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.855,"5min":3.367,"15min":3.367},"last":1.6020000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":223683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0669999999999999,"15min":1.0609999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5700000000000001,"5min":1.776,"15min":1.776},"last":1.444},{"interface":"front","average":{"1min":1.05,"5min":1.115,"15min":1.1259999999999999},"min":{"1min":0.64900000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.4930000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.91500000000000004}]},{"osd":1,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1200000000000001,"15min":1.129},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5089999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1879999999999999,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6659999999999999,"5min":1.756,"15min":1.756},"last":1.1419999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.26,"15min":1.2769999999999999},"min":{"1min":0.89700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3720000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.395},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2310000000000001,"15min":1.2230000000000001},"min":{"1min":0.88900000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.5609999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.82199999999999995,"5min":0.75,"15min":0.75},"max":{"1min":1.524,"5min":1.736,"15min":1.736},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.357,"15min":1.3640000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.738,"5min":2.121,"15min":2.121},"last":1.6559999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2929999999999999,"15min":1.2949999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8759999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.746,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6419999999999999,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.3}]},{"osd":6,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3819999999999999,"15min":1.3839999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.825,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.9450000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.349}]},{"osd":7,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.405,"15min":1.415},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.617,"5min":2.286,"15min":2.286},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.577,"15min":1.5940000000000001},"min":{"1min":1.008,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.9139999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.9359999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542177,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0589999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.075},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97999999999999998},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.96799999999999997}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0249999999999999,"15min":1.0449999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.458,"5min":1.78,"15min":1.78},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.97599999999999998,"15min":0.99199999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.24,"5min":1.623,"15min":1.623},"last":0.90700000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.55,"5min":3.7799999999999998,"15min":3.7799999999999998},"last":1.339},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.202,"15min":1.2290000000000001},"min":{"1min":0.628,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5920000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.2589999999999999}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3620000000000001,"15min":1.3799999999999999},"min":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.369,"15min":1.383},"min":{"1min":0.88500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.7190000000000001,"5min":2.573,"15min":2.573},"last":1.671}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.5629999999999999,"15min":1.6000000000000001},"min":{"1min":0.99399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":1.851},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.514,"15min":1.5700000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6299999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":1.3140000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.534,"15min":1.5580000000000001},"min":{"1min":1.0569999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.873,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.6060000000000001,"15min":1.6459999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.758,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":1.389}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.6499999999999999,"15min":1.6899999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.825,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.6379999999999999,"15min":1.6879999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.738,"5min":3.4340000000000002,"15min":3.4340000000000002},"last":1.4430000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542176,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":223705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89900000000000002,"15min":0.90300000000000002},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3009999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.121},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.024,"15min":1.036},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.405,"5min":1.865,"15min":1.865},"last":0.92000000000000004}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.383,"5min":1.468,"15min":1.468},"last":1.333},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.079,"15min":1.091},"min":{"1min":0.64900000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.1850000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4570000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.2},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.879,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.4119999999999999}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4139999999999999,"15min":1.4179999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.8759999999999999,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.93100000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.847,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.4770000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.508,"15min":1.5229999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9730000000000001,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.4399999999999999,"15min":1.462},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6990000000000001,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.4339999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7829999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.5069999999999999,"15min":1.5129999999999999},"min":{"1min":0.97799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.0289999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.6040000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.6060000000000001,"15min":1.6279999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9279999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.593,"15min":1.6140000000000001},"min":{"1min":1.054,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.907,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":1.55}]}]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":499712,"data_stored":222141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.575,"15min":1.575},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0089999999999999,"15min":1.0149999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.69499999999999995}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.026,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.782,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.034,"5min":1.0569999999999999,"15min":1.0609999999999999},"min":{"1min":0.64000000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.377,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.1879999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.282,"15min":1.292},"min":{"1min":0.70999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6200000000000001,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.2709999999999999,"15min":1.266},"min":{"1min":0.62,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.962,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":0.745}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.411,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.7410000000000001,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4199999999999999,"15min":1.4359999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.994,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":1.1160000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1059999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.018},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5009999999999999,"15min":1.51},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.8700000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.304}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.524,"15min":1.518},"min":{"1min":1.0529999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.3500000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.458,"15min":1.4670000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.919,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.2230000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.5209999999999999,"15min":1.524},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.0449999999999999,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.617,"15min":1.617},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":2.4529999999999998,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.3999999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542178,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1093632,"data_stored":812473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.83799999999999997},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.109,"5min":1.288,"15min":1.288},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":0.92100000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.379}]},{"osd":2,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.052},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0249999999999999,"15min":1.0129999999999999},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.113}]},{"osd":3,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.51200000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3080000000000001,"5min":1.522,"15min":1.522},"last":1.857},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.137,"15min":1.149},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6879999999999999,"5min":1.728,"15min":1.728},"last":1.329}]},{"osd":4,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.323,"15min":1.3360000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.724,"5min":2.23,"15min":2.23},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2869999999999999,"15min":1.298},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.738}]},{"osd":5,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.333,"15min":1.337},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.889,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.524},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.4330000000000001,"15min":1.4490000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.7549999999999999,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":1.587}]},{"osd":6,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.3460000000000001,"15min":1.3700000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.677,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.22},{"interface":"front","average":{"1min":1.347,"5min":1.413,"15min":1.4239999999999999},"min":{"1min":0.877,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.6579999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4990000000000001,"15min":1.5069999999999999},"min":{"1min":0.86199999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.99,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.7},{"interface":"front","average":{"1min":1.337,"5min":1.468,"15min":1.49},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.7729999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4239999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:53:14.309 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:53:14.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.581+0000 7f90e6942640 1 Processor -- start 2023-08-11T15:53:14.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.581+0000 7f90e6942640 1 -- start start 2023-08-11T15:53:14.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.583+0000 7f90e6942640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90e015a140 0x7f90e006b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.583+0000 7f90e6942640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f90e00cb430 0x7f90e00b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.583+0000 7f90e6942640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e00c8a30 con 0x7f90e006f5d0 2023-08-11T15:53:14.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.583+0000 7f90e6942640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e006c0c0 con 0x7f90e00cb430 2023-08-11T15:53:14.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.583+0000 7f90e6942640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e006c3a0 con 0x7f90e015a140 2023-08-11T15:53:14.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.583+0000 7f90e4edf640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f90e006f5d0 0x7f90e00c8ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49038/0 (socket says 172.21.15.27:49038) 2023-08-11T15:53:14.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.583+0000 7f90e4edf640 1 -- 172.21.15.27:0/3117883186 learned_addr learned my addr 172.21.15.27:0/3117883186 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:14.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.583+0000 7f90df7fe640 1 --2- 172.21.15.27:0/3117883186 >> v2:172.21.15.27:3300/0 conn(0x7f90e00cb430 0x7f90e00b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.584+0000 7f90dffff640 1 --2- 172.21.15.27:0/3117883186 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90e015a140 0x7f90e006b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.584+0000 7f90deffd640 1 -- 172.21.15.27:0/3117883186 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692569135 0 0) 0x7f90e00c8a30 con 0x7f90e006f5d0 2023-08-11T15:53:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.584+0000 7f90deffd640 1 -- 172.21.15.27:0/3117883186 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c8003520 con 0x7f90e006f5d0 2023-08-11T15:53:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.584+0000 7f90df7fe640 1 -- 172.21.15.27:0/3117883186 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90e015a140 msgr2=0x7f90e006b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.584+0000 7f90df7fe640 1 --2- 172.21.15.27:0/3117883186 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90e015a140 0x7f90e006b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.584+0000 7f90df7fe640 1 -- 172.21.15.27:0/3117883186 >> v1:172.21.15.27:6789/0 conn(0x7f90e006f5d0 legacy=0x7f90e00c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.584+0000 7f90df7fe640 1 -- 172.21.15.27:0/3117883186 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e006c680 con 0x7f90e00cb430 2023-08-11T15:53:14.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.585+0000 7f90df7fe640 1 --2- 172.21.15.27:0/3117883186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 0x7f90e00b0e70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f90d40097a0 tx=0x7f90d402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b29aceb2d8ae947 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.586+0000 7f90deffd640 1 -- 172.21.15.27:0/3117883186 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d402d040 con 0x7f90e00cb430 2023-08-11T15:53:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.586+0000 7f90deffd640 1 -- 172.21.15.27:0/3117883186 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d4031810 con 0x7f90e00cb430 2023-08-11T15:53:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.586+0000 7f90deffd640 1 -- 172.21.15.27:0/3117883186 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4031b80 con 0x7f90e00cb430 2023-08-11T15:53:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.586+0000 7f90e6942640 1 -- 172.21.15.27:0/3117883186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 msgr2=0x7f90e00b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.586+0000 7f90e6942640 1 --2- 172.21.15.27:0/3117883186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 0x7f90e00b0e70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f90d40097a0 tx=0x7f90d402b8b0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.586+0000 7f90e6942640 1 -- 172.21.15.27:0/3117883186 shutdown_connections 2023-08-11T15:53:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.586+0000 7f90e6942640 1 --2- 172.21.15.27:0/3117883186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 0x7f90e00b0e70 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.586+0000 7f90e6942640 1 --2- 172.21.15.27:0/3117883186 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90e015a140 0x7f90e006b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.587+0000 7f90e6942640 1 -- 172.21.15.27:0/3117883186 >> 172.21.15.27:0/3117883186 conn(0x7f90e00afb70 msgr2=0x7f90e00aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.587+0000 7f90e6942640 1 -- 172.21.15.27:0/3117883186 shutdown_connections 2023-08-11T15:53:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.587+0000 7f90e6942640 1 -- 172.21.15.27:0/3117883186 wait complete. 2023-08-11T15:53:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.588+0000 7f90e6942640 1 Processor -- start 2023-08-11T15:53:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.589+0000 7f90e6942640 1 -- start start 2023-08-11T15:53:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.589+0000 7f90e6942640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90e006f5d0 0x7f90e00c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.589+0000 7f90e6942640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 0x7f90e014fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90e6942640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e015a140 0x7f90e00c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90e6942640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e01f29a0 con 0x7f90e00cb430 2023-08-11T15:53:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90e6942640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e01f26c0 con 0x7f90e015a140 2023-08-11T15:53:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90e6942640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e01e50e0 con 0x7f90e006f5d0 2023-08-11T15:53:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 0x7f90e014fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 0x7f90e014fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40308/0 (socket says 172.21.15.27:40308) 2023-08-11T15:53:14.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90df7fe640 1 -- 172.21.15.27:0/2125851791 learned_addr learned my addr 172.21.15.27:0/2125851791 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:14.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90dffff640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90e006f5d0 0x7f90e00c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90e4edf640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e015a140 0x7f90e00c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90df7fe640 1 -- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90e006f5d0 msgr2=0x7f90e00c2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90df7fe640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90e006f5d0 0x7f90e00c2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90df7fe640 1 -- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e015a140 msgr2=0x7f90e00c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90df7fe640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e015a140 0x7f90e00c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90df7fe640 1 -- 172.21.15.27:0/2125851791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d4009400 con 0x7f90e00cb430 2023-08-11T15:53:14.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.590+0000 7f90e4edf640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e015a140 0x7f90e00c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:14.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.591+0000 7f90df7fe640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 0x7f90e014fde0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f90d402b8f0 tx=0x7f90d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.591+0000 7f90dcff9640 1 -- 172.21.15.27:0/2125851791 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4006e10 con 0x7f90e00cb430 2023-08-11T15:53:14.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.591+0000 7f90dcff9640 1 -- 172.21.15.27:0/2125851791 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d403f070 con 0x7f90e00cb430 2023-08-11T15:53:14.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.591+0000 7f90dcff9640 1 -- 172.21.15.27:0/2125851791 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4003a60 con 0x7f90e00cb430 2023-08-11T15:53:14.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.591+0000 7f90e6942640 1 -- 172.21.15.27:0/2125851791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e01e53c0 con 0x7f90e00cb430 2023-08-11T15:53:14.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.591+0000 7f90e6942640 1 -- 172.21.15.27:0/2125851791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e0150320 con 0x7f90e00cb430 2023-08-11T15:53:14.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.592+0000 7f90e6942640 1 -- 172.21.15.27:0/2125851791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e01e58e0 con 0x7f90e00cb430 2023-08-11T15:53:14.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.593+0000 7f90dcff9640 1 -- 172.21.15.27:0/2125851791 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f90d4003bc0 con 0x7f90e00cb430 2023-08-11T15:53:14.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.594+0000 7f90dcff9640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f90cc06a050 0x7f90cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.594+0000 7f90dcff9640 1 -- 172.21.15.27:0/2125851791 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f90d4041070 con 0x7f90e00cb430 2023-08-11T15:53:14.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.595+0000 7f90dffff640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f90cc06a050 0x7f90cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.596+0000 7f90dffff640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f90cc06a050 0x7f90cc06c510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009960 tx=0x7f90d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.598+0000 7f90dcff9640 1 -- 172.21.15.27:0/2125851791 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f90e01e58e0 con 0x7f90e00cb430 2023-08-11T15:53:14.664 INFO:tasks.ceph.mds.a.smithi027.stderr:2023-08-11T15:53:14.662+0000 7f0d7beb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-11T15:53:14.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.790+0000 7f90e6942640 1 -- 172.21.15.27:0/2125851791 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90e01e4f60 con 0x7f90cc06a050 2023-08-11T15:53:14.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.815+0000 7f90dcff9640 1 -- 172.21.15.27:0/2125851791 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309081 (secure 0 0 0) 0x7f90d0010870 con 0x7f90cc06a050 2023-08-11T15:53:14.819 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:14.823 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:53:14.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.822+0000 7f90e6942640 1 -- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f90cc06a050 msgr2=0x7f90cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.822+0000 7f90e6942640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f90cc06a050 0x7f90cc06c510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009960 tx=0x7f90d0008040 comp rx=0 tx=0).stop 2023-08-11T15:53:14.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.822+0000 7f90e6942640 1 -- 172.21.15.27:0/2125851791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 msgr2=0x7f90e014fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.822+0000 7f90e6942640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 0x7f90e014fde0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f90d402b8f0 tx=0x7f90d40046c0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.823+0000 7f90e6942640 1 -- 172.21.15.27:0/2125851791 shutdown_connections 2023-08-11T15:53:14.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.823+0000 7f90e6942640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f90cc06a050 0x7f90cc06c510 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.823+0000 7f90e6942640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e015a140 0x7f90e00c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.823+0000 7f90e6942640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90e00cb430 0x7f90e014fde0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.823+0000 7f90e6942640 1 --2- 172.21.15.27:0/2125851791 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90e006f5d0 0x7f90e00c2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.823+0000 7f90e6942640 1 -- 172.21.15.27:0/2125851791 >> 172.21.15.27:0/2125851791 conn(0x7f90e00afb70 msgr2=0x7f90e00c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:14.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.823+0000 7f90e6942640 1 -- 172.21.15.27:0/2125851791 shutdown_connections 2023-08-11T15:53:14.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:14.823+0000 7f90e6942640 1 -- 172.21.15.27:0/2125851791 wait complete. 2023-08-11T15:53:14.847 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-08-11T15:53:14.762332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233552,"kb_used_data":5084,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754741168,"statfs":{"total":773094113280,"available":772854956032,"internally_reserved":0,"allocated":5206016,"data_stored":2955151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.049920"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180399+0000","last_change":"2023-08-11T15:53:09.773210+0000","last_active":"2023-08-11T15:53:11.180399+0000","last_peered":"2023-08-11T15:53:11.180399+0000","last_clean":"2023-08-11T15:53:11.180399+0000","last_became_active":"2023-08-11T15:53:09.772902+0000","last_became_peered":"2023-08-11T15:53:09.772902+0000","last_unstale":"2023-08-11T15:53:11.180399+0000","last_undegraded":"2023-08-11T15:53:11.180399+0000","last_fullsized":"2023-08-11T15:53:11.180399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:12.451179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753016+0000","last_change":"2023-08-11T15:53:09.791848+0000","last_active":"2023-08-11T15:53:10.753016+0000","last_peered":"2023-08-11T15:53:10.753016+0000","last_clean":"2023-08-11T15:53:10.753016+0000","last_became_active":"2023-08-11T15:53:09.791542+0000","last_became_peered":"2023-08-11T15:53:09.791542+0000","last_unstale":"2023-08-11T15:53:10.753016+0000","last_undegraded":"2023-08-11T15:53:10.753016+0000","last_fullsized":"2023-08-11T15:53:10.753016+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:10.218889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754347+0000","last_change":"2023-08-11T15:53:09.781474+0000","last_active":"2023-08-11T15:53:10.754347+0000","last_peered":"2023-08-11T15:53:10.754347+0000","last_clean":"2023-08-11T15:53:10.754347+0000","last_became_active":"2023-08-11T15:53:09.781167+0000","last_became_peered":"2023-08-11T15:53:09.781167+0000","last_unstale":"2023-08-11T15:53:10.754347+0000","last_undegraded":"2023-08-11T15:53:10.754347+0000","last_fullsized":"2023-08-11T15:53:10.754347+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:24:33.839216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181867+0000","last_change":"2023-08-11T15:53:09.785095+0000","last_active":"2023-08-11T15:53:11.181867+0000","last_peered":"2023-08-11T15:53:11.181867+0000","last_clean":"2023-08-11T15:53:11.181867+0000","last_became_active":"2023-08-11T15:53:09.783871+0000","last_became_peered":"2023-08-11T15:53:09.783871+0000","last_unstale":"2023-08-11T15:53:11.181867+0000","last_undegraded":"2023-08-11T15:53:11.181867+0000","last_fullsized":"2023-08-11T15:53:11.181867+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:10:51.539728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751022+0000","last_change":"2023-08-11T15:53:09.793442+0000","last_active":"2023-08-11T15:53:10.751022+0000","last_peered":"2023-08-11T15:53:10.751022+0000","last_clean":"2023-08-11T15:53:10.751022+0000","last_became_active":"2023-08-11T15:53:09.793158+0000","last_became_peered":"2023-08-11T15:53:09.793158+0000","last_unstale":"2023-08-11T15:53:10.751022+0000","last_undegraded":"2023-08-11T15:53:10.751022+0000","last_fullsized":"2023-08-11T15:53:10.751022+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:49.810015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106013+0000","last_change":"2023-08-11T15:53:09.782227+0000","last_active":"2023-08-11T15:53:11.106013+0000","last_peered":"2023-08-11T15:53:11.106013+0000","last_clean":"2023-08-11T15:53:11.106013+0000","last_became_active":"2023-08-11T15:53:09.781914+0000","last_became_peered":"2023-08-11T15:53:09.781914+0000","last_unstale":"2023-08-11T15:53:11.106013+0000","last_undegraded":"2023-08-11T15:53:11.106013+0000","last_fullsized":"2023-08-11T15:53:11.106013+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:28:49.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180066+0000","last_change":"2023-08-11T15:53:09.769652+0000","last_active":"2023-08-11T15:53:11.180066+0000","last_peered":"2023-08-11T15:53:11.180066+0000","last_clean":"2023-08-11T15:53:11.180066+0000","last_became_active":"2023-08-11T15:53:09.769356+0000","last_became_peered":"2023-08-11T15:53:09.769356+0000","last_unstale":"2023-08-11T15:53:11.180066+0000","last_undegraded":"2023-08-11T15:53:11.180066+0000","last_fullsized":"2023-08-11T15:53:11.180066+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:04:07.355560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.183055+0000","last_change":"2023-08-11T15:53:09.782362+0000","last_active":"2023-08-11T15:53:11.183055+0000","last_peered":"2023-08-11T15:53:11.183055+0000","last_clean":"2023-08-11T15:53:11.183055+0000","last_became_active":"2023-08-11T15:53:09.782053+0000","last_became_peered":"2023-08-11T15:53:09.782053+0000","last_unstale":"2023-08-11T15:53:11.183055+0000","last_undegraded":"2023-08-11T15:53:11.183055+0000","last_fullsized":"2023-08-11T15:53:11.183055+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:45:27.150501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181078+0000","last_change":"2023-08-11T15:53:09.782949+0000","last_active":"2023-08-11T15:53:11.181078+0000","last_peered":"2023-08-11T15:53:11.181078+0000","last_clean":"2023-08-11T15:53:11.181078+0000","last_became_active":"2023-08-11T15:53:09.782603+0000","last_became_peered":"2023-08-11T15:53:09.782603+0000","last_unstale":"2023-08-11T15:53:11.181078+0000","last_undegraded":"2023-08-11T15:53:11.181078+0000","last_fullsized":"2023-08-11T15:53:11.181078+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:31:07.338552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182401+0000","last_change":"2023-08-11T15:53:09.789197+0000","last_active":"2023-08-11T15:53:11.182401+0000","last_peered":"2023-08-11T15:53:11.182401+0000","last_clean":"2023-08-11T15:53:11.182401+0000","last_became_active":"2023-08-11T15:53:09.788353+0000","last_became_peered":"2023-08-11T15:53:09.788353+0000","last_unstale":"2023-08-11T15:53:11.182401+0000","last_undegraded":"2023-08-11T15:53:11.182401+0000","last_fullsized":"2023-08-11T15:53:11.182401+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:41:25.658590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.183187+0000","last_change":"2023-08-11T15:53:09.791697+0000","last_active":"2023-08-11T15:53:11.183187+0000","last_peered":"2023-08-11T15:53:11.183187+0000","last_clean":"2023-08-11T15:53:11.183187+0000","last_became_active":"2023-08-11T15:53:09.791410+0000","last_became_peered":"2023-08-11T15:53:09.791410+0000","last_unstale":"2023-08-11T15:53:11.183187+0000","last_undegraded":"2023-08-11T15:53:11.183187+0000","last_fullsized":"2023-08-11T15:53:11.183187+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:24:26.104141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750984+0000","last_change":"2023-08-11T15:53:09.776182+0000","last_active":"2023-08-11T15:53:10.750984+0000","last_peered":"2023-08-11T15:53:10.750984+0000","last_clean":"2023-08-11T15:53:10.750984+0000","last_became_active":"2023-08-11T15:53:09.775906+0000","last_became_peered":"2023-08-11T15:53:09.775906+0000","last_unstale":"2023-08-11T15:53:10.750984+0000","last_undegraded":"2023-08-11T15:53:10.750984+0000","last_fullsized":"2023-08-11T15:53:10.750984+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:50:11.087522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181594+0000","last_change":"2023-08-11T15:53:09.780882+0000","last_active":"2023-08-11T15:53:11.181594+0000","last_peered":"2023-08-11T15:53:11.181594+0000","last_clean":"2023-08-11T15:53:11.181594+0000","last_became_active":"2023-08-11T15:53:09.780459+0000","last_became_peered":"2023-08-11T15:53:09.780459+0000","last_unstale":"2023-08-11T15:53:11.181594+0000","last_undegraded":"2023-08-11T15:53:11.181594+0000","last_fullsized":"2023-08-11T15:53:11.181594+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:36:41.825732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181130+0000","last_change":"2023-08-11T15:53:09.782350+0000","last_active":"2023-08-11T15:53:11.181130+0000","last_peered":"2023-08-11T15:53:11.181130+0000","last_clean":"2023-08-11T15:53:11.181130+0000","last_became_active":"2023-08-11T15:53:09.781999+0000","last_became_peered":"2023-08-11T15:53:09.781999+0000","last_unstale":"2023-08-11T15:53:11.181130+0000","last_undegraded":"2023-08-11T15:53:11.181130+0000","last_fullsized":"2023-08-11T15:53:11.181130+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:54:54.260711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.105963+0000","last_change":"2023-08-11T15:53:09.777919+0000","last_active":"2023-08-11T15:53:11.105963+0000","last_peered":"2023-08-11T15:53:11.105963+0000","last_clean":"2023-08-11T15:53:11.105963+0000","last_became_active":"2023-08-11T15:53:09.777623+0000","last_became_peered":"2023-08-11T15:53:09.777623+0000","last_unstale":"2023-08-11T15:53:11.105963+0000","last_undegraded":"2023-08-11T15:53:11.105963+0000","last_fullsized":"2023-08-11T15:53:11.105963+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:04:07.355560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751415+0000","last_change":"2023-08-11T15:53:09.786416+0000","last_active":"2023-08-11T15:53:10.751415+0000","last_peered":"2023-08-11T15:53:10.751415+0000","last_clean":"2023-08-11T15:53:10.751415+0000","last_became_active":"2023-08-11T15:53:09.786039+0000","last_became_peered":"2023-08-11T15:53:09.786039+0000","last_unstale":"2023-08-11T15:53:10.751415+0000","last_undegraded":"2023-08-11T15:53:10.751415+0000","last_fullsized":"2023-08-11T15:53:10.751415+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:40:17.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754308+0000","last_change":"2023-08-11T15:53:09.771798+0000","last_active":"2023-08-11T15:53:10.754308+0000","last_peered":"2023-08-11T15:53:10.754308+0000","last_clean":"2023-08-11T15:53:10.754308+0000","last_became_active":"2023-08-11T15:53:09.771479+0000","last_became_peered":"2023-08-11T15:53:09.771479+0000","last_unstale":"2023-08-11T15:53:10.754308+0000","last_undegraded":"2023-08-11T15:53:10.754308+0000","last_fullsized":"2023-08-11T15:53:10.754308+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:44:46.953266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754113+0000","last_change":"2023-08-11T15:53:09.790561+0000","last_active":"2023-08-11T15:53:10.754113+0000","last_peered":"2023-08-11T15:53:10.754113+0000","last_clean":"2023-08-11T15:53:10.754113+0000","last_became_active":"2023-08-11T15:53:09.790261+0000","last_became_peered":"2023-08-11T15:53:09.790261+0000","last_unstale":"2023-08-11T15:53:10.754113+0000","last_undegraded":"2023-08-11T15:53:10.754113+0000","last_fullsized":"2023-08-11T15:53:10.754113+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:16:25.172596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753423+0000","last_change":"2023-08-11T15:53:09.788819+0000","last_active":"2023-08-11T15:53:10.753423+0000","last_peered":"2023-08-11T15:53:10.753423+0000","last_clean":"2023-08-11T15:53:10.753423+0000","last_became_active":"2023-08-11T15:53:09.788567+0000","last_became_peered":"2023-08-11T15:53:09.788567+0000","last_unstale":"2023-08-11T15:53:10.753423+0000","last_undegraded":"2023-08-11T15:53:10.753423+0000","last_fullsized":"2023-08-11T15:53:10.753423+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:30:12.859168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182993+0000","last_change":"2023-08-11T15:53:09.786551+0000","last_active":"2023-08-11T15:53:11.182993+0000","last_peered":"2023-08-11T15:53:11.182993+0000","last_clean":"2023-08-11T15:53:11.182993+0000","last_became_active":"2023-08-11T15:53:09.786173+0000","last_became_peered":"2023-08-11T15:53:09.786173+0000","last_unstale":"2023-08-11T15:53:11.182993+0000","last_undegraded":"2023-08-11T15:53:11.182993+0000","last_fullsized":"2023-08-11T15:53:11.182993+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:43:20.589597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106710+0000","last_change":"2023-08-11T15:53:09.780023+0000","last_active":"2023-08-11T15:53:11.106710+0000","last_peered":"2023-08-11T15:53:11.106710+0000","last_clean":"2023-08-11T15:53:11.106710+0000","last_became_active":"2023-08-11T15:53:09.779747+0000","last_became_peered":"2023-08-11T15:53:09.779747+0000","last_unstale":"2023-08-11T15:53:11.106710+0000","last_undegraded":"2023-08-11T15:53:11.106710+0000","last_fullsized":"2023-08-11T15:53:11.106710+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:24.779884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181260+0000","last_change":"2023-08-11T15:53:09.785222+0000","last_active":"2023-08-11T15:53:11.181260+0000","last_peered":"2023-08-11T15:53:11.181260+0000","last_clean":"2023-08-11T15:53:11.181260+0000","last_became_active":"2023-08-11T15:53:09.781496+0000","last_became_peered":"2023-08-11T15:53:09.781496+0000","last_unstale":"2023-08-11T15:53:11.181260+0000","last_undegraded":"2023-08-11T15:53:11.181260+0000","last_fullsized":"2023-08-11T15:53:11.181260+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:18.503920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751658+0000","last_change":"2023-08-11T15:53:09.762513+0000","last_active":"2023-08-11T15:53:10.751658+0000","last_peered":"2023-08-11T15:53:10.751658+0000","last_clean":"2023-08-11T15:53:10.751658+0000","last_became_active":"2023-08-11T15:53:09.762138+0000","last_became_peered":"2023-08-11T15:53:09.762138+0000","last_unstale":"2023-08-11T15:53:10.751658+0000","last_undegraded":"2023-08-11T15:53:10.751658+0000","last_fullsized":"2023-08-11T15:53:10.751658+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:37:28.533073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751940+0000","last_change":"2023-08-11T15:53:09.762892+0000","last_active":"2023-08-11T15:53:10.751940+0000","last_peered":"2023-08-11T15:53:10.751940+0000","last_clean":"2023-08-11T15:53:10.751940+0000","last_became_active":"2023-08-11T15:53:09.762422+0000","last_became_peered":"2023-08-11T15:53:09.762422+0000","last_unstale":"2023-08-11T15:53:10.751940+0000","last_undegraded":"2023-08-11T15:53:10.751940+0000","last_fullsized":"2023-08-11T15:53:10.751940+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:09:16.571916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753257+0000","last_change":"2023-08-11T15:53:09.768901+0000","last_active":"2023-08-11T15:53:10.753257+0000","last_peered":"2023-08-11T15:53:10.753257+0000","last_clean":"2023-08-11T15:53:10.753257+0000","last_became_active":"2023-08-11T15:53:09.768592+0000","last_became_peered":"2023-08-11T15:53:09.768592+0000","last_unstale":"2023-08-11T15:53:10.753257+0000","last_undegraded":"2023-08-11T15:53:10.753257+0000","last_fullsized":"2023-08-11T15:53:10.753257+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:22:57.491029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182215+0000","last_change":"2023-08-11T15:53:09.776020+0000","last_active":"2023-08-11T15:53:11.182215+0000","last_peered":"2023-08-11T15:53:11.182215+0000","last_clean":"2023-08-11T15:53:11.182215+0000","last_became_active":"2023-08-11T15:53:09.775742+0000","last_became_peered":"2023-08-11T15:53:09.775742+0000","last_unstale":"2023-08-11T15:53:11.182215+0000","last_undegraded":"2023-08-11T15:53:11.182215+0000","last_fullsized":"2023-08-11T15:53:11.182215+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:34:06.466622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753967+0000","last_change":"2023-08-11T15:53:09.771280+0000","last_active":"2023-08-11T15:53:10.753967+0000","last_peered":"2023-08-11T15:53:10.753967+0000","last_clean":"2023-08-11T15:53:10.753967+0000","last_became_active":"2023-08-11T15:53:09.770977+0000","last_became_peered":"2023-08-11T15:53:09.770977+0000","last_unstale":"2023-08-11T15:53:10.753967+0000","last_undegraded":"2023-08-11T15:53:10.753967+0000","last_fullsized":"2023-08-11T15:53:10.753967+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:34:06.295544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753323+0000","last_change":"2023-08-11T15:53:09.767829+0000","last_active":"2023-08-11T15:53:10.753323+0000","last_peered":"2023-08-11T15:53:10.753323+0000","last_clean":"2023-08-11T15:53:10.753323+0000","last_became_active":"2023-08-11T15:53:09.767503+0000","last_became_peered":"2023-08-11T15:53:09.767503+0000","last_unstale":"2023-08-11T15:53:10.753323+0000","last_undegraded":"2023-08-11T15:53:10.753323+0000","last_fullsized":"2023-08-11T15:53:10.753323+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:37:22.863178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752475+0000","last_change":"2023-08-11T15:53:09.768971+0000","last_active":"2023-08-11T15:53:10.752475+0000","last_peered":"2023-08-11T15:53:10.752475+0000","last_clean":"2023-08-11T15:53:10.752475+0000","last_became_active":"2023-08-11T15:53:09.768669+0000","last_became_peered":"2023-08-11T15:53:09.768669+0000","last_unstale":"2023-08-11T15:53:10.752475+0000","last_undegraded":"2023-08-11T15:53:10.752475+0000","last_fullsized":"2023-08-11T15:53:10.752475+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:01.051346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750777+0000","last_change":"2023-08-11T15:53:09.773565+0000","last_active":"2023-08-11T15:53:10.750777+0000","last_peered":"2023-08-11T15:53:10.750777+0000","last_clean":"2023-08-11T15:53:10.750777+0000","last_became_active":"2023-08-11T15:53:09.772543+0000","last_became_peered":"2023-08-11T15:53:09.772543+0000","last_unstale":"2023-08-11T15:53:10.750777+0000","last_undegraded":"2023-08-11T15:53:10.750777+0000","last_fullsized":"2023-08-11T15:53:10.750777+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:28:45.132262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.105874+0000","last_change":"2023-08-11T15:53:09.771905+0000","last_active":"2023-08-11T15:53:11.105874+0000","last_peered":"2023-08-11T15:53:11.105874+0000","last_clean":"2023-08-11T15:53:11.105874+0000","last_became_active":"2023-08-11T15:53:09.771513+0000","last_became_peered":"2023-08-11T15:53:09.771513+0000","last_unstale":"2023-08-11T15:53:11.105874+0000","last_undegraded":"2023-08-11T15:53:11.105874+0000","last_fullsized":"2023-08-11T15:53:11.105874+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:14:28.311384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180100+0000","last_change":"2023-08-11T15:53:09.769904+0000","last_active":"2023-08-11T15:53:11.180100+0000","last_peered":"2023-08-11T15:53:11.180100+0000","last_clean":"2023-08-11T15:53:11.180100+0000","last_became_active":"2023-08-11T15:53:09.769477+0000","last_became_peered":"2023-08-11T15:53:09.769477+0000","last_unstale":"2023-08-11T15:53:11.180100+0000","last_undegraded":"2023-08-11T15:53:11.180100+0000","last_fullsized":"2023-08-11T15:53:11.180100+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:30:25.907270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.178486+0000","last_change":"2023-08-11T15:53:09.776091+0000","last_active":"2023-08-11T15:53:11.178486+0000","last_peered":"2023-08-11T15:53:11.178486+0000","last_clean":"2023-08-11T15:53:11.178486+0000","last_became_active":"2023-08-11T15:53:09.775817+0000","last_became_peered":"2023-08-11T15:53:09.775817+0000","last_unstale":"2023-08-11T15:53:11.178486+0000","last_undegraded":"2023-08-11T15:53:11.178486+0000","last_fullsized":"2023-08-11T15:53:11.178486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:03:49.496617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107302+0000","last_change":"2023-08-11T15:53:08.748593+0000","last_active":"2023-08-11T15:53:11.107302+0000","last_peered":"2023-08-11T15:53:11.107302+0000","last_clean":"2023-08-11T15:53:11.107302+0000","last_became_active":"2023-08-11T15:53:08.748357+0000","last_became_peered":"2023-08-11T15:53:08.748357+0000","last_unstale":"2023-08-11T15:53:11.107302+0000","last_undegraded":"2023-08-11T15:53:11.107302+0000","last_fullsized":"2023-08-11T15:53:11.107302+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:37:21.858059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753273+0000","last_change":"2023-08-11T15:53:09.772220+0000","last_active":"2023-08-11T15:53:10.753273+0000","last_peered":"2023-08-11T15:53:10.753273+0000","last_clean":"2023-08-11T15:53:10.753273+0000","last_became_active":"2023-08-11T15:53:09.771903+0000","last_became_peered":"2023-08-11T15:53:09.771903+0000","last_unstale":"2023-08-11T15:53:10.753273+0000","last_undegraded":"2023-08-11T15:53:10.753273+0000","last_fullsized":"2023-08-11T15:53:10.753273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:16:12.516630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754459+0000","last_change":"2023-08-11T15:53:08.748687+0000","last_active":"2023-08-11T15:53:10.754459+0000","last_peered":"2023-08-11T15:53:10.754459+0000","last_clean":"2023-08-11T15:53:10.754459+0000","last_became_active":"2023-08-11T15:53:08.748363+0000","last_became_peered":"2023-08-11T15:53:08.748363+0000","last_unstale":"2023-08-11T15:53:10.754459+0000","last_undegraded":"2023-08-11T15:53:10.754459+0000","last_fullsized":"2023-08-11T15:53:10.754459+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:37.020312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753432+0000","last_change":"2023-08-11T15:53:09.774231+0000","last_active":"2023-08-11T15:53:10.753432+0000","last_peered":"2023-08-11T15:53:10.753432+0000","last_clean":"2023-08-11T15:53:10.753432+0000","last_became_active":"2023-08-11T15:53:09.773949+0000","last_became_peered":"2023-08-11T15:53:09.773949+0000","last_unstale":"2023-08-11T15:53:10.753432+0000","last_undegraded":"2023-08-11T15:53:10.753432+0000","last_fullsized":"2023-08-11T15:53:10.753432+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:09:03.177518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.179654+0000","last_change":"2023-08-11T15:53:08.740718+0000","last_active":"2023-08-11T15:53:11.179654+0000","last_peered":"2023-08-11T15:53:11.179654+0000","last_clean":"2023-08-11T15:53:11.179654+0000","last_became_active":"2023-08-11T15:53:08.740379+0000","last_became_peered":"2023-08-11T15:53:08.740379+0000","last_unstale":"2023-08-11T15:53:11.179654+0000","last_undegraded":"2023-08-11T15:53:11.179654+0000","last_fullsized":"2023-08-11T15:53:11.179654+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:06:51.600650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.179484+0000","last_change":"2023-08-11T15:53:09.766544+0000","last_active":"2023-08-11T15:53:11.179484+0000","last_peered":"2023-08-11T15:53:11.179484+0000","last_clean":"2023-08-11T15:53:11.179484+0000","last_became_active":"2023-08-11T15:53:09.766107+0000","last_became_peered":"2023-08-11T15:53:09.766107+0000","last_unstale":"2023-08-11T15:53:11.179484+0000","last_undegraded":"2023-08-11T15:53:11.179484+0000","last_fullsized":"2023-08-11T15:53:11.179484+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:02:07.798605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180483+0000","last_change":"2023-08-11T15:53:08.742961+0000","last_active":"2023-08-11T15:53:11.180483+0000","last_peered":"2023-08-11T15:53:11.180483+0000","last_clean":"2023-08-11T15:53:11.180483+0000","last_became_active":"2023-08-11T15:53:08.742699+0000","last_became_peered":"2023-08-11T15:53:08.742699+0000","last_unstale":"2023-08-11T15:53:11.180483+0000","last_undegraded":"2023-08-11T15:53:11.180483+0000","last_fullsized":"2023-08-11T15:53:11.180483+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:50:03.246394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751701+0000","last_change":"2023-08-11T15:53:09.777301+0000","last_active":"2023-08-11T15:53:10.751701+0000","last_peered":"2023-08-11T15:53:10.751701+0000","last_clean":"2023-08-11T15:53:10.751701+0000","last_became_active":"2023-08-11T15:53:09.776921+0000","last_became_peered":"2023-08-11T15:53:09.776921+0000","last_unstale":"2023-08-11T15:53:10.751701+0000","last_undegraded":"2023-08-11T15:53:10.751701+0000","last_fullsized":"2023-08-11T15:53:10.751701+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:30:04.778864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107258+0000","last_change":"2023-08-11T15:53:08.745370+0000","last_active":"2023-08-11T15:53:11.107258+0000","last_peered":"2023-08-11T15:53:11.107258+0000","last_clean":"2023-08-11T15:53:11.107258+0000","last_became_active":"2023-08-11T15:53:08.745073+0000","last_became_peered":"2023-08-11T15:53:08.745073+0000","last_unstale":"2023-08-11T15:53:11.107258+0000","last_undegraded":"2023-08-11T15:53:11.107258+0000","last_fullsized":"2023-08-11T15:53:11.107258+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.334880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181516+0000","last_change":"2023-08-11T15:53:09.787372+0000","last_active":"2023-08-11T15:53:11.181516+0000","last_peered":"2023-08-11T15:53:11.181516+0000","last_clean":"2023-08-11T15:53:11.181516+0000","last_became_active":"2023-08-11T15:53:09.787086+0000","last_became_peered":"2023-08-11T15:53:09.787086+0000","last_unstale":"2023-08-11T15:53:11.181516+0000","last_undegraded":"2023-08-11T15:53:11.181516+0000","last_fullsized":"2023-08-11T15:53:11.181516+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:54:17.954707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752037+0000","last_change":"2023-08-11T15:53:08.744766+0000","last_active":"2023-08-11T15:53:10.752037+0000","last_peered":"2023-08-11T15:53:10.752037+0000","last_clean":"2023-08-11T15:53:10.752037+0000","last_became_active":"2023-08-11T15:53:08.743573+0000","last_became_peered":"2023-08-11T15:53:08.743573+0000","last_unstale":"2023-08-11T15:53:10.752037+0000","last_undegraded":"2023-08-11T15:53:10.752037+0000","last_fullsized":"2023-08-11T15:53:10.752037+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:32:10.006528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182304+0000","last_change":"2023-08-11T15:53:09.780061+0000","last_active":"2023-08-11T15:53:11.182304+0000","last_peered":"2023-08-11T15:53:11.182304+0000","last_clean":"2023-08-11T15:53:11.182304+0000","last_became_active":"2023-08-11T15:53:09.779628+0000","last_became_peered":"2023-08-11T15:53:09.779628+0000","last_unstale":"2023-08-11T15:53:11.182304+0000","last_undegraded":"2023-08-11T15:53:11.182304+0000","last_fullsized":"2023-08-11T15:53:11.182304+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:39:34.729202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750150+0000","last_change":"2023-08-11T15:53:08.740221+0000","last_active":"2023-08-11T15:53:10.750150+0000","last_peered":"2023-08-11T15:53:10.750150+0000","last_clean":"2023-08-11T15:53:10.750150+0000","last_became_active":"2023-08-11T15:53:08.739950+0000","last_became_peered":"2023-08-11T15:53:08.739950+0000","last_unstale":"2023-08-11T15:53:10.750150+0000","last_undegraded":"2023-08-11T15:53:10.750150+0000","last_fullsized":"2023-08-11T15:53:10.750150+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:50:03.246394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751661+0000","last_change":"2023-08-11T15:53:09.762734+0000","last_active":"2023-08-11T15:53:10.751661+0000","last_peered":"2023-08-11T15:53:10.751661+0000","last_clean":"2023-08-11T15:53:10.751661+0000","last_became_active":"2023-08-11T15:53:09.762126+0000","last_became_peered":"2023-08-11T15:53:09.762126+0000","last_unstale":"2023-08-11T15:53:10.751661+0000","last_undegraded":"2023-08-11T15:53:10.751661+0000","last_fullsized":"2023-08-11T15:53:10.751661+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:25:54.207324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182422+0000","last_change":"2023-08-11T15:53:08.737858+0000","last_active":"2023-08-11T15:53:11.182422+0000","last_peered":"2023-08-11T15:53:11.182422+0000","last_clean":"2023-08-11T15:53:11.182422+0000","last_became_active":"2023-08-11T15:53:08.737526+0000","last_became_peered":"2023-08-11T15:53:08.737526+0000","last_unstale":"2023-08-11T15:53:11.182422+0000","last_undegraded":"2023-08-11T15:53:11.182422+0000","last_fullsized":"2023-08-11T15:53:11.182422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:10:04.293967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754434+0000","last_change":"2023-08-11T15:53:09.790581+0000","last_active":"2023-08-11T15:53:10.754434+0000","last_peered":"2023-08-11T15:53:10.754434+0000","last_clean":"2023-08-11T15:53:10.754434+0000","last_became_active":"2023-08-11T15:53:09.790274+0000","last_became_peered":"2023-08-11T15:53:09.790274+0000","last_unstale":"2023-08-11T15:53:10.754434+0000","last_undegraded":"2023-08-11T15:53:10.754434+0000","last_fullsized":"2023-08-11T15:53:10.754434+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:56.289946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754796+0000","last_change":"2023-08-11T15:53:08.733462+0000","last_active":"2023-08-11T15:53:10.754796+0000","last_peered":"2023-08-11T15:53:10.754796+0000","last_clean":"2023-08-11T15:53:10.754796+0000","last_became_active":"2023-08-11T15:53:08.733010+0000","last_became_peered":"2023-08-11T15:53:08.733010+0000","last_unstale":"2023-08-11T15:53:10.754796+0000","last_undegraded":"2023-08-11T15:53:10.754796+0000","last_fullsized":"2023-08-11T15:53:10.754796+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:28:00.974391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180864+0000","last_change":"2023-08-11T15:53:09.767772+0000","last_active":"2023-08-11T15:53:11.180864+0000","last_peered":"2023-08-11T15:53:11.180864+0000","last_clean":"2023-08-11T15:53:11.180864+0000","last_became_active":"2023-08-11T15:53:09.767438+0000","last_became_peered":"2023-08-11T15:53:09.767438+0000","last_unstale":"2023-08-11T15:53:11.180864+0000","last_undegraded":"2023-08-11T15:53:11.180864+0000","last_fullsized":"2023-08-11T15:53:11.180864+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:20.339999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182940+0000","last_change":"2023-08-11T15:53:08.743600+0000","last_active":"2023-08-11T15:53:11.182940+0000","last_peered":"2023-08-11T15:53:11.182940+0000","last_clean":"2023-08-11T15:53:11.182940+0000","last_became_active":"2023-08-11T15:53:08.743263+0000","last_became_peered":"2023-08-11T15:53:08.743263+0000","last_unstale":"2023-08-11T15:53:11.182940+0000","last_undegraded":"2023-08-11T15:53:11.182940+0000","last_fullsized":"2023-08-11T15:53:11.182940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:39.683699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107177+0000","last_change":"2023-08-11T15:53:09.784595+0000","last_active":"2023-08-11T15:53:11.107177+0000","last_peered":"2023-08-11T15:53:11.107177+0000","last_clean":"2023-08-11T15:53:11.107177+0000","last_became_active":"2023-08-11T15:53:09.780448+0000","last_became_peered":"2023-08-11T15:53:09.780448+0000","last_unstale":"2023-08-11T15:53:11.107177+0000","last_undegraded":"2023-08-11T15:53:11.107177+0000","last_fullsized":"2023-08-11T15:53:11.107177+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:56:34.581615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180919+0000","last_change":"2023-08-11T15:53:08.750910+0000","last_active":"2023-08-11T15:53:11.180919+0000","last_peered":"2023-08-11T15:53:11.180919+0000","last_clean":"2023-08-11T15:53:11.180919+0000","last_became_active":"2023-08-11T15:53:08.750600+0000","last_became_peered":"2023-08-11T15:53:08.750600+0000","last_unstale":"2023-08-11T15:53:11.180919+0000","last_undegraded":"2023-08-11T15:53:11.180919+0000","last_fullsized":"2023-08-11T15:53:11.180919+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:09.213770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180279+0000","last_change":"2023-08-11T15:53:09.768466+0000","last_active":"2023-08-11T15:53:11.180279+0000","last_peered":"2023-08-11T15:53:11.180279+0000","last_clean":"2023-08-11T15:53:11.180279+0000","last_became_active":"2023-08-11T15:53:09.768172+0000","last_became_peered":"2023-08-11T15:53:09.768172+0000","last_unstale":"2023-08-11T15:53:11.180279+0000","last_undegraded":"2023-08-11T15:53:11.180279+0000","last_fullsized":"2023-08-11T15:53:11.180279+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:28:49.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181521+0000","last_change":"2023-08-11T15:53:08.734705+0000","last_active":"2023-08-11T15:53:11.181521+0000","last_peered":"2023-08-11T15:53:11.181521+0000","last_clean":"2023-08-11T15:53:11.181521+0000","last_became_active":"2023-08-11T15:53:08.734399+0000","last_became_peered":"2023-08-11T15:53:08.734399+0000","last_unstale":"2023-08-11T15:53:11.181521+0000","last_undegraded":"2023-08-11T15:53:11.181521+0000","last_fullsized":"2023-08-11T15:53:11.181521+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:48:12.780203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752030+0000","last_change":"2023-08-11T15:53:09.786330+0000","last_active":"2023-08-11T15:53:10.752030+0000","last_peered":"2023-08-11T15:53:10.752030+0000","last_clean":"2023-08-11T15:53:10.752030+0000","last_became_active":"2023-08-11T15:53:09.785968+0000","last_became_peered":"2023-08-11T15:53:09.785968+0000","last_unstale":"2023-08-11T15:53:10.752030+0000","last_undegraded":"2023-08-11T15:53:10.752030+0000","last_fullsized":"2023-08-11T15:53:10.752030+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:31:25.473751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107264+0000","last_change":"2023-08-11T15:53:08.741591+0000","last_active":"2023-08-11T15:53:11.107264+0000","last_peered":"2023-08-11T15:53:11.107264+0000","last_clean":"2023-08-11T15:53:11.107264+0000","last_became_active":"2023-08-11T15:53:08.741284+0000","last_became_peered":"2023-08-11T15:53:08.741284+0000","last_unstale":"2023-08-11T15:53:11.107264+0000","last_undegraded":"2023-08-11T15:53:11.107264+0000","last_fullsized":"2023-08-11T15:53:11.107264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:09.213770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181808+0000","last_change":"2023-08-11T15:53:09.780157+0000","last_active":"2023-08-11T15:53:11.181808+0000","last_peered":"2023-08-11T15:53:11.181808+0000","last_clean":"2023-08-11T15:53:11.181808+0000","last_became_active":"2023-08-11T15:53:09.779865+0000","last_became_peered":"2023-08-11T15:53:09.779865+0000","last_unstale":"2023-08-11T15:53:11.181808+0000","last_undegraded":"2023-08-11T15:53:11.181808+0000","last_fullsized":"2023-08-11T15:53:11.181808+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:33:56.983793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180527+0000","last_change":"2023-08-11T15:53:08.739293+0000","last_active":"2023-08-11T15:53:11.180527+0000","last_peered":"2023-08-11T15:53:11.180527+0000","last_clean":"2023-08-11T15:53:11.180527+0000","last_became_active":"2023-08-11T15:53:08.739017+0000","last_became_peered":"2023-08-11T15:53:08.739017+0000","last_unstale":"2023-08-11T15:53:11.180527+0000","last_undegraded":"2023-08-11T15:53:11.180527+0000","last_fullsized":"2023-08-11T15:53:11.180527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:23.774765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.105942+0000","last_change":"2023-08-11T15:53:09.783386+0000","last_active":"2023-08-11T15:53:11.105942+0000","last_peered":"2023-08-11T15:53:11.105942+0000","last_clean":"2023-08-11T15:53:11.105942+0000","last_became_active":"2023-08-11T15:53:09.783094+0000","last_became_peered":"2023-08-11T15:53:09.783094+0000","last_unstale":"2023-08-11T15:53:11.105942+0000","last_undegraded":"2023-08-11T15:53:11.105942+0000","last_fullsized":"2023-08-11T15:53:11.105942+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:05:14.964206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106001+0000","last_change":"2023-08-11T15:53:08.741032+0000","last_active":"2023-08-11T15:53:11.106001+0000","last_peered":"2023-08-11T15:53:11.106001+0000","last_clean":"2023-08-11T15:53:11.106001+0000","last_became_active":"2023-08-11T15:53:08.740718+0000","last_became_peered":"2023-08-11T15:53:08.740718+0000","last_unstale":"2023-08-11T15:53:11.106001+0000","last_undegraded":"2023-08-11T15:53:11.106001+0000","last_fullsized":"2023-08-11T15:53:11.106001+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:34.637074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180295+0000","last_change":"2023-08-11T15:53:09.762947+0000","last_active":"2023-08-11T15:53:11.180295+0000","last_peered":"2023-08-11T15:53:11.180295+0000","last_clean":"2023-08-11T15:53:11.180295+0000","last_became_active":"2023-08-11T15:53:09.762608+0000","last_became_peered":"2023-08-11T15:53:09.762608+0000","last_unstale":"2023-08-11T15:53:11.180295+0000","last_undegraded":"2023-08-11T15:53:11.180295+0000","last_fullsized":"2023-08-11T15:53:11.180295+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:49.301086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182550+0000","last_change":"2023-08-11T15:53:08.744714+0000","last_active":"2023-08-11T15:53:11.182550+0000","last_peered":"2023-08-11T15:53:11.182550+0000","last_clean":"2023-08-11T15:53:11.182550+0000","last_became_active":"2023-08-11T15:53:08.744259+0000","last_became_peered":"2023-08-11T15:53:08.744259+0000","last_unstale":"2023-08-11T15:53:11.182550+0000","last_undegraded":"2023-08-11T15:53:11.182550+0000","last_fullsized":"2023-08-11T15:53:11.182550+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:06:08.293733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182557+0000","last_change":"2023-08-11T15:53:09.764295+0000","last_active":"2023-08-11T15:53:11.182557+0000","last_peered":"2023-08-11T15:53:11.182557+0000","last_clean":"2023-08-11T15:53:11.182557+0000","last_became_active":"2023-08-11T15:53:09.763798+0000","last_became_peered":"2023-08-11T15:53:09.763798+0000","last_unstale":"2023-08-11T15:53:11.182557+0000","last_undegraded":"2023-08-11T15:53:11.182557+0000","last_fullsized":"2023-08-11T15:53:11.182557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:56.000165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182381+0000","last_change":"2023-08-11T15:53:08.734098+0000","last_active":"2023-08-11T15:53:11.182381+0000","last_peered":"2023-08-11T15:53:11.182381+0000","last_clean":"2023-08-11T15:53:11.182381+0000","last_became_active":"2023-08-11T15:53:08.733793+0000","last_became_peered":"2023-08-11T15:53:08.733793+0000","last_unstale":"2023-08-11T15:53:11.182381+0000","last_undegraded":"2023-08-11T15:53:11.182381+0000","last_fullsized":"2023-08-11T15:53:11.182381+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:06:41.748325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.183161+0000","last_change":"2023-08-11T15:53:09.790589+0000","last_active":"2023-08-11T15:53:11.183161+0000","last_peered":"2023-08-11T15:53:11.183161+0000","last_clean":"2023-08-11T15:53:11.183161+0000","last_became_active":"2023-08-11T15:53:09.790277+0000","last_became_peered":"2023-08-11T15:53:09.790277+0000","last_unstale":"2023-08-11T15:53:11.183161+0000","last_undegraded":"2023-08-11T15:53:11.183161+0000","last_fullsized":"2023-08-11T15:53:11.183161+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:52:13.686728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754864+0000","last_change":"2023-08-11T15:53:08.739707+0000","last_active":"2023-08-11T15:53:10.754864+0000","last_peered":"2023-08-11T15:53:10.754864+0000","last_clean":"2023-08-11T15:53:10.754864+0000","last_became_active":"2023-08-11T15:53:08.735001+0000","last_became_peered":"2023-08-11T15:53:08.735001+0000","last_unstale":"2023-08-11T15:53:10.754864+0000","last_undegraded":"2023-08-11T15:53:10.754864+0000","last_fullsized":"2023-08-11T15:53:10.754864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:37:42.368869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752790+0000","last_change":"2023-08-11T15:53:09.767296+0000","last_active":"2023-08-11T15:53:10.752790+0000","last_peered":"2023-08-11T15:53:10.752790+0000","last_clean":"2023-08-11T15:53:10.752790+0000","last_became_active":"2023-08-11T15:53:09.766939+0000","last_became_peered":"2023-08-11T15:53:09.766939+0000","last_unstale":"2023-08-11T15:53:10.752790+0000","last_undegraded":"2023-08-11T15:53:10.752790+0000","last_fullsized":"2023-08-11T15:53:10.752790+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:35:59.980509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752395+0000","last_change":"2023-08-11T15:53:08.744460+0000","last_active":"2023-08-11T15:53:10.752395+0000","last_peered":"2023-08-11T15:53:10.752395+0000","last_clean":"2023-08-11T15:53:10.752395+0000","last_became_active":"2023-08-11T15:53:08.744011+0000","last_became_peered":"2023-08-11T15:53:08.744011+0000","last_unstale":"2023-08-11T15:53:10.752395+0000","last_undegraded":"2023-08-11T15:53:10.752395+0000","last_fullsized":"2023-08-11T15:53:10.752395+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.334880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752914+0000","last_change":"2023-08-11T15:53:09.778529+0000","last_active":"2023-08-11T15:53:10.752914+0000","last_peered":"2023-08-11T15:53:10.752914+0000","last_clean":"2023-08-11T15:53:10.752914+0000","last_became_active":"2023-08-11T15:53:09.778176+0000","last_became_peered":"2023-08-11T15:53:09.778176+0000","last_unstale":"2023-08-11T15:53:10.752914+0000","last_undegraded":"2023-08-11T15:53:10.752914+0000","last_fullsized":"2023-08-11T15:53:10.752914+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:43:49.491851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107099+0000","last_change":"2023-08-11T15:53:08.748386+0000","last_active":"2023-08-11T15:53:11.107099+0000","last_peered":"2023-08-11T15:53:11.107099+0000","last_clean":"2023-08-11T15:53:11.107099+0000","last_became_active":"2023-08-11T15:53:08.748054+0000","last_became_peered":"2023-08-11T15:53:08.748054+0000","last_unstale":"2023-08-11T15:53:11.107099+0000","last_undegraded":"2023-08-11T15:53:11.107099+0000","last_fullsized":"2023-08-11T15:53:11.107099+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:48.804896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106108+0000","last_change":"2023-08-11T15:53:09.779061+0000","last_active":"2023-08-11T15:53:11.106108+0000","last_peered":"2023-08-11T15:53:11.106108+0000","last_clean":"2023-08-11T15:53:11.106108+0000","last_became_active":"2023-08-11T15:53:09.778786+0000","last_became_peered":"2023-08-11T15:53:09.778786+0000","last_unstale":"2023-08-11T15:53:11.106108+0000","last_undegraded":"2023-08-11T15:53:11.106108+0000","last_fullsized":"2023-08-11T15:53:11.106108+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:49.301086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.107674+0000","last_change":"2023-08-11T15:53:08.744974+0000","last_active":"2023-08-11T15:53:11.107674+0000","last_peered":"2023-08-11T15:53:11.107674+0000","last_clean":"2023-08-11T15:53:11.107674+0000","last_became_active":"2023-08-11T15:53:08.744738+0000","last_became_peered":"2023-08-11T15:53:08.744738+0000","last_unstale":"2023-08-11T15:53:11.107674+0000","last_undegraded":"2023-08-11T15:53:11.107674+0000","last_fullsized":"2023-08-11T15:53:11.107674+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:09:02.172399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.179121+0000","last_change":"2023-08-11T15:53:09.769152+0000","last_active":"2023-08-11T15:53:11.179121+0000","last_peered":"2023-08-11T15:53:11.179121+0000","last_clean":"2023-08-11T15:53:11.179121+0000","last_became_active":"2023-08-11T15:53:09.768472+0000","last_became_peered":"2023-08-11T15:53:09.768472+0000","last_unstale":"2023-08-11T15:53:11.179121+0000","last_undegraded":"2023-08-11T15:53:11.179121+0000","last_fullsized":"2023-08-11T15:53:11.179121+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:31:44.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750123+0000","last_change":"2023-08-11T15:53:08.750963+0000","last_active":"2023-08-11T15:53:10.750123+0000","last_peered":"2023-08-11T15:53:10.750123+0000","last_clean":"2023-08-11T15:53:10.750123+0000","last_became_active":"2023-08-11T15:53:08.750546+0000","last_became_peered":"2023-08-11T15:53:08.750546+0000","last_unstale":"2023-08-11T15:53:10.750123+0000","last_undegraded":"2023-08-11T15:53:10.750123+0000","last_fullsized":"2023-08-11T15:53:10.750123+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:11.446060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180238+0000","last_change":"2023-08-11T15:53:09.766778+0000","last_active":"2023-08-11T15:53:11.180238+0000","last_peered":"2023-08-11T15:53:11.180238+0000","last_clean":"2023-08-11T15:53:11.180238+0000","last_became_active":"2023-08-11T15:53:09.766457+0000","last_became_peered":"2023-08-11T15:53:09.766457+0000","last_unstale":"2023-08-11T15:53:11.180238+0000","last_undegraded":"2023-08-11T15:53:11.180238+0000","last_fullsized":"2023-08-11T15:53:11.180238+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:59:07.514405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180854+0000","last_change":"2023-08-11T15:53:08.745022+0000","last_active":"2023-08-11T15:53:11.180854+0000","last_peered":"2023-08-11T15:53:11.180854+0000","last_clean":"2023-08-11T15:53:11.180854+0000","last_became_active":"2023-08-11T15:53:08.741244+0000","last_became_peered":"2023-08-11T15:53:08.741244+0000","last_unstale":"2023-08-11T15:53:11.180854+0000","last_undegraded":"2023-08-11T15:53:11.180854+0000","last_fullsized":"2023-08-11T15:53:11.180854+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:53:43.175908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180730+0000","last_change":"2023-08-11T15:53:09.783571+0000","last_active":"2023-08-11T15:53:11.180730+0000","last_peered":"2023-08-11T15:53:11.180730+0000","last_clean":"2023-08-11T15:53:11.180730+0000","last_became_active":"2023-08-11T15:53:09.783194+0000","last_became_peered":"2023-08-11T15:53:09.783194+0000","last_unstale":"2023-08-11T15:53:11.180730+0000","last_undegraded":"2023-08-11T15:53:11.180730+0000","last_fullsized":"2023-08-11T15:53:11.180730+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:14:23.224719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180069+0000","last_change":"2023-08-11T15:53:08.738754+0000","last_active":"2023-08-11T15:53:11.180069+0000","last_peered":"2023-08-11T15:53:11.180069+0000","last_clean":"2023-08-11T15:53:11.180069+0000","last_became_active":"2023-08-11T15:53:08.738462+0000","last_became_peered":"2023-08-11T15:53:08.738462+0000","last_unstale":"2023-08-11T15:53:11.180069+0000","last_undegraded":"2023-08-11T15:53:11.180069+0000","last_fullsized":"2023-08-11T15:53:11.180069+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:34.637074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106414+0000","last_change":"2023-08-11T15:53:09.790391+0000","last_active":"2023-08-11T15:53:11.106414+0000","last_peered":"2023-08-11T15:53:11.106414+0000","last_clean":"2023-08-11T15:53:11.106414+0000","last_became_active":"2023-08-11T15:53:09.790046+0000","last_became_peered":"2023-08-11T15:53:09.790046+0000","last_unstale":"2023-08-11T15:53:11.106414+0000","last_undegraded":"2023-08-11T15:53:11.106414+0000","last_fullsized":"2023-08-11T15:53:11.106414+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:12.451179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180720+0000","last_change":"2023-08-11T15:53:08.751384+0000","last_active":"2023-08-11T15:53:11.180720+0000","last_peered":"2023-08-11T15:53:11.180720+0000","last_clean":"2023-08-11T15:53:11.180720+0000","last_became_active":"2023-08-11T15:53:08.751062+0000","last_became_peered":"2023-08-11T15:53:08.751062+0000","last_unstale":"2023-08-11T15:53:11.180720+0000","last_undegraded":"2023-08-11T15:53:11.180720+0000","last_fullsized":"2023-08-11T15:53:11.180720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:56:33.576496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753539+0000","last_change":"2023-08-11T15:50:54.667601+0000","last_active":"2023-08-11T15:53:10.753539+0000","last_peered":"2023-08-11T15:53:10.753539+0000","last_clean":"2023-08-11T15:53:10.753539+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:53:10.753539+0000","last_undegraded":"2023-08-11T15:53:10.753539+0000","last_fullsized":"2023-08-11T15:53:10.753539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:28:24.398965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106698+0000","last_change":"2023-08-11T15:53:09.781908+0000","last_active":"2023-08-11T15:53:11.106698+0000","last_peered":"2023-08-11T15:53:11.106698+0000","last_clean":"2023-08-11T15:53:11.106698+0000","last_became_active":"2023-08-11T15:53:09.781535+0000","last_became_peered":"2023-08-11T15:53:09.781535+0000","last_unstale":"2023-08-11T15:53:11.106698+0000","last_undegraded":"2023-08-11T15:53:11.106698+0000","last_fullsized":"2023-08-11T15:53:11.106698+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:50:04.251513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.752492+0000","last_change":"2023-08-11T15:53:08.740595+0000","last_active":"2023-08-11T15:53:10.752492+0000","last_peered":"2023-08-11T15:53:10.752492+0000","last_clean":"2023-08-11T15:53:10.752492+0000","last_became_active":"2023-08-11T15:53:08.734795+0000","last_became_peered":"2023-08-11T15:53:08.734795+0000","last_unstale":"2023-08-11T15:53:10.752492+0000","last_undegraded":"2023-08-11T15:53:10.752492+0000","last_fullsized":"2023-08-11T15:53:10.752492+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:28:51.472838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.178669+0000","last_change":"2023-08-11T15:50:54.662968+0000","last_active":"2023-08-11T15:53:11.178669+0000","last_peered":"2023-08-11T15:53:11.178669+0000","last_clean":"2023-08-11T15:53:11.178669+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:53:11.178669+0000","last_undegraded":"2023-08-11T15:53:11.178669+0000","last_fullsized":"2023-08-11T15:53:11.178669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:21:13.932510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.749850+0000","last_change":"2023-08-11T15:53:09.773008+0000","last_active":"2023-08-11T15:53:10.749850+0000","last_peered":"2023-08-11T15:53:10.749850+0000","last_clean":"2023-08-11T15:53:10.749850+0000","last_became_active":"2023-08-11T15:53:09.772271+0000","last_became_peered":"2023-08-11T15:53:09.772271+0000","last_unstale":"2023-08-11T15:53:10.749850+0000","last_undegraded":"2023-08-11T15:53:10.749850+0000","last_fullsized":"2023-08-11T15:53:10.749850+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:49.301086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.178894+0000","last_change":"2023-08-11T15:53:08.747721+0000","last_active":"2023-08-11T15:53:11.178894+0000","last_peered":"2023-08-11T15:53:11.178894+0000","last_clean":"2023-08-11T15:53:11.178894+0000","last_became_active":"2023-08-11T15:53:08.747339+0000","last_became_peered":"2023-08-11T15:53:08.747339+0000","last_unstale":"2023-08-11T15:53:11.178894+0000","last_undegraded":"2023-08-11T15:53:11.178894+0000","last_fullsized":"2023-08-11T15:53:11.178894+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:09:56.665706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181034+0000","last_change":"2023-08-11T15:50:55.272783+0000","last_active":"2023-08-11T15:53:11.181034+0000","last_peered":"2023-08-11T15:53:11.181034+0000","last_clean":"2023-08-11T15:53:11.181034+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:53:11.181034+0000","last_undegraded":"2023-08-11T15:53:11.181034+0000","last_fullsized":"2023-08-11T15:53:11.181034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:06:48.106587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750996+0000","last_change":"2023-08-11T15:53:09.778080+0000","last_active":"2023-08-11T15:53:10.750996+0000","last_peered":"2023-08-11T15:53:10.750996+0000","last_clean":"2023-08-11T15:53:10.750996+0000","last_became_active":"2023-08-11T15:53:09.777754+0000","last_became_peered":"2023-08-11T15:53:09.777754+0000","last_unstale":"2023-08-11T15:53:10.750996+0000","last_undegraded":"2023-08-11T15:53:10.750996+0000","last_fullsized":"2023-08-11T15:53:10.750996+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:30:40.840167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182680+0000","last_change":"2023-08-11T15:53:08.737070+0000","last_active":"2023-08-11T15:53:11.182680+0000","last_peered":"2023-08-11T15:53:11.182680+0000","last_clean":"2023-08-11T15:53:11.182680+0000","last_became_active":"2023-08-11T15:53:08.736773+0000","last_became_peered":"2023-08-11T15:53:08.736773+0000","last_unstale":"2023-08-11T15:53:11.182680+0000","last_undegraded":"2023-08-11T15:53:11.182680+0000","last_fullsized":"2023-08-11T15:53:11.182680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:43:19.728568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.178275+0000","last_change":"2023-08-11T15:50:55.270974+0000","last_active":"2023-08-11T15:53:11.178275+0000","last_peered":"2023-08-11T15:53:11.178275+0000","last_clean":"2023-08-11T15:53:11.178275+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:53:11.178275+0000","last_undegraded":"2023-08-11T15:53:11.178275+0000","last_fullsized":"2023-08-11T15:53:11.178275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:27:00.317635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.753641+0000","last_change":"2023-08-11T15:53:09.786536+0000","last_active":"2023-08-11T15:53:10.753641+0000","last_peered":"2023-08-11T15:53:10.753641+0000","last_clean":"2023-08-11T15:53:10.753641+0000","last_became_active":"2023-08-11T15:53:09.786189+0000","last_became_peered":"2023-08-11T15:53:09.786189+0000","last_unstale":"2023-08-11T15:53:10.753641+0000","last_undegraded":"2023-08-11T15:53:10.753641+0000","last_fullsized":"2023-08-11T15:53:10.753641+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:46:31.671075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.106982+0000","last_change":"2023-08-11T15:53:08.743492+0000","last_active":"2023-08-11T15:53:11.106982+0000","last_peered":"2023-08-11T15:53:11.106982+0000","last_clean":"2023-08-11T15:53:11.106982+0000","last_became_active":"2023-08-11T15:53:08.743162+0000","last_became_peered":"2023-08-11T15:53:08.743162+0000","last_unstale":"2023-08-11T15:53:11.106982+0000","last_undegraded":"2023-08-11T15:53:11.106982+0000","last_fullsized":"2023-08-11T15:53:11.106982+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:50:10.082403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.750591+0000","last_change":"2023-08-11T15:50:54.660862+0000","last_active":"2023-08-11T15:53:10.750591+0000","last_peered":"2023-08-11T15:53:10.750591+0000","last_clean":"2023-08-11T15:53:10.750591+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:53:10.750591+0000","last_undegraded":"2023-08-11T15:53:10.750591+0000","last_fullsized":"2023-08-11T15:53:10.750591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:22:09.708953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181772+0000","last_change":"2023-08-11T15:53:09.786440+0000","last_active":"2023-08-11T15:53:11.181772+0000","last_peered":"2023-08-11T15:53:11.181772+0000","last_clean":"2023-08-11T15:53:11.181772+0000","last_became_active":"2023-08-11T15:53:09.786031+0000","last_became_peered":"2023-08-11T15:53:09.786031+0000","last_unstale":"2023-08-11T15:53:11.181772+0000","last_undegraded":"2023-08-11T15:53:11.181772+0000","last_fullsized":"2023-08-11T15:53:11.181772+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:58:29.111103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.181041+0000","last_change":"2023-08-11T15:53:08.750894+0000","last_active":"2023-08-11T15:53:11.181041+0000","last_peered":"2023-08-11T15:53:11.181041+0000","last_clean":"2023-08-11T15:53:11.181041+0000","last_became_active":"2023-08-11T15:53:08.750562+0000","last_became_peered":"2023-08-11T15:53:08.750562+0000","last_unstale":"2023-08-11T15:53:11.181041+0000","last_undegraded":"2023-08-11T15:53:11.181041+0000","last_fullsized":"2023-08-11T15:53:11.181041+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:37:21.858059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180151+0000","last_change":"2023-08-11T15:50:54.660575+0000","last_active":"2023-08-11T15:53:11.180151+0000","last_peered":"2023-08-11T15:53:11.180151+0000","last_clean":"2023-08-11T15:53:11.180151+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:53:11.180151+0000","last_undegraded":"2023-08-11T15:53:11.180151+0000","last_fullsized":"2023-08-11T15:53:11.180151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:13:57.717733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180705+0000","last_change":"2023-08-11T15:53:09.768911+0000","last_active":"2023-08-11T15:53:11.180705+0000","last_peered":"2023-08-11T15:53:11.180705+0000","last_clean":"2023-08-11T15:53:11.180705+0000","last_became_active":"2023-08-11T15:53:09.768610+0000","last_became_peered":"2023-08-11T15:53:09.768610+0000","last_unstale":"2023-08-11T15:53:11.180705+0000","last_undegraded":"2023-08-11T15:53:11.180705+0000","last_fullsized":"2023-08-11T15:53:11.180705+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:49.810015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.178189+0000","last_change":"2023-08-11T15:53:08.743107+0000","last_active":"2023-08-11T15:53:11.178189+0000","last_peered":"2023-08-11T15:53:11.178189+0000","last_clean":"2023-08-11T15:53:11.178189+0000","last_became_active":"2023-08-11T15:53:08.742738+0000","last_became_peered":"2023-08-11T15:53:08.742738+0000","last_unstale":"2023-08-11T15:53:11.178189+0000","last_undegraded":"2023-08-11T15:53:11.178189+0000","last_fullsized":"2023-08-11T15:53:11.178189+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:53:03.427605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.179866+0000","last_change":"2023-08-11T15:50:54.669675+0000","last_active":"2023-08-11T15:53:11.179866+0000","last_peered":"2023-08-11T15:53:11.179866+0000","last_clean":"2023-08-11T15:53:11.179866+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:53:11.179866+0000","last_undegraded":"2023-08-11T15:53:11.179866+0000","last_fullsized":"2023-08-11T15:53:11.179866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:02:59.893275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.182688+0000","last_change":"2023-08-11T15:53:09.788950+0000","last_active":"2023-08-11T15:53:11.182688+0000","last_peered":"2023-08-11T15:53:11.182688+0000","last_clean":"2023-08-11T15:53:11.182688+0000","last_became_active":"2023-08-11T15:53:09.785438+0000","last_became_peered":"2023-08-11T15:53:09.785438+0000","last_unstale":"2023-08-11T15:53:11.182688+0000","last_undegraded":"2023-08-11T15:53:11.182688+0000","last_fullsized":"2023-08-11T15:53:11.182688+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:08.702209+0000","last_clean_scrub_stamp":"2023-08-11T15:53:08.702209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:09:24.668922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.754046+0000","last_change":"2023-08-11T15:53:08.743904+0000","last_active":"2023-08-11T15:53:10.754046+0000","last_peered":"2023-08-11T15:53:10.754046+0000","last_clean":"2023-08-11T15:53:10.754046+0000","last_became_active":"2023-08-11T15:53:08.743585+0000","last_became_peered":"2023-08-11T15:53:08.743585+0000","last_unstale":"2023-08-11T15:53:10.754046+0000","last_undegraded":"2023-08-11T15:53:10.754046+0000","last_fullsized":"2023-08-11T15:53:10.754046+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:07.697090+0000","last_clean_scrub_stamp":"2023-08-11T15:53:07.697090+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:43:35.240762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:10.751222+0000","last_change":"2023-08-11T15:50:54.663730+0000","last_active":"2023-08-11T15:53:10.751222+0000","last_peered":"2023-08-11T15:53:10.751222+0000","last_clean":"2023-08-11T15:53:10.751222+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:53:10.751222+0000","last_undegraded":"2023-08-11T15:53:10.751222+0000","last_fullsized":"2023-08-11T15:53:10.751222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:54:19.510684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-08-11T15:53:11.180453+0000","last_change":"2023-08-11T15:50:48.426393+0000","last_active":"2023-08-11T15:53:11.180453+0000","last_peered":"2023-08-11T15:53:11.180453+0000","last_clean":"2023-08-11T15:53:11.180453+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:53:11.180453+0000","last_undegraded":"2023-08-11T15:53:11.180453+0000","last_fullsized":"2023-08-11T15:53:11.180453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_clean_scrub_stamp":"2023-08-11T15:50:47.133560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:47:49.518872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1081344,"data_stored":805556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.093,"15min":1.1160000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.042},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.216,"15min":1.222},"min":{"1min":0.80900000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5509999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.98699999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3129999999999999,"15min":1.327},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.7370000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.333,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.091}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.177,"15min":1.175},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.232,"5min":1.254,"15min":1.258},"min":{"1min":0.87,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5169999999999999,"5min":2.02,"15min":2.02},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2909999999999999,"15min":1.3069999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.671,"5min":1.869,"15min":1.869},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4890000000000001,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.86199999999999999}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4219999999999999,"15min":1.4279999999999999},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.885,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.181},{"interface":"front","average":{"1min":1.282,"5min":1.395,"15min":1.4139999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6910000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.2410000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.373,"15min":1.381},"min":{"1min":0.93500000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5449999999999999,"15min":1.5509999999999999},"min":{"1min":1.2170000000000001,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":1.9330000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.286}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.5029999999999999,"15min":1.5209999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.768,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.544,"15min":1.5580000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.861,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.3999999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542177,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":222743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0609999999999999,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1339999999999999,"15min":1.1359999999999999},"min":{"1min":0.751,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.115,"15min":1.1140000000000001},"min":{"1min":0.73599999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.591,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.878},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.66100000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.0820000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2390000000000001,"15min":1.24},"min":{"1min":0.85599999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.3180000000000001,"15min":1.3400000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5600000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.327}]},{"osd":3,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.369,"15min":1.377},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.77,"5min":2.774,"15min":2.774},"last":1.145},{"interface":"front","average":{"1min":1.282,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7330000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.575}]},{"osd":4,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.319,"15min":1.327},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8879999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.468,"15min":1.4910000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.665,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.518}]},{"osd":5,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.496,"15min":1.5069999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8149999999999999,"5min":3.371,"15min":3.371},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.486,"15min":1.504},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0009999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.2789999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.615,"15min":1.6479999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.762,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.6739999999999999,"15min":1.704},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.921,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.752}]}]},{"osd":5,"up_from":14,"seq":60129542178,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":222141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0389999999999999,"15min":1.0269999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.077},{"interface":"front","average":{"1min":1.073,"5min":1.1499999999999999,"15min":1.163},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.133}]},{"osd":1,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1419999999999999,"15min":1.1399999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.93999999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.621,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.516}]},{"osd":2,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.262,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.615,"5min":1.827,"15min":1.827},"last":1.482},{"interface":"front","average":{"1min":1.194,"5min":1.351,"15min":1.377},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8300000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.3919999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.353,"15min":1.353},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.702,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.292},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.444},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.782,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.339}]},{"osd":4,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4379999999999999,"15min":1.466},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8779999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5880000000000001,"15min":1.6080000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.847,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.96499999999999997}]},{"osd":6,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.4690000000000001,"15min":1.492},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.796,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.6120000000000001,"15min":1.6379999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.2160000000000002,"5min":3.9870000000000001,"15min":3.9870000000000001},"last":1.6759999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.5820000000000001,"15min":1.605},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9990000000000001,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.637,"15min":1.677},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.855,"5min":3.367,"15min":3.367},"last":1.6020000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":223683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0669999999999999,"15min":1.0609999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5700000000000001,"5min":1.776,"15min":1.776},"last":1.444},{"interface":"front","average":{"1min":1.05,"5min":1.115,"15min":1.1259999999999999},"min":{"1min":0.64900000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.4930000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.91500000000000004}]},{"osd":1,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1200000000000001,"15min":1.129},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5089999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1879999999999999,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6659999999999999,"5min":1.756,"15min":1.756},"last":1.1419999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.26,"15min":1.2769999999999999},"min":{"1min":0.89700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3720000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.395},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2310000000000001,"15min":1.2230000000000001},"min":{"1min":0.88900000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.5609999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.82199999999999995,"5min":0.75,"15min":0.75},"max":{"1min":1.524,"5min":1.736,"15min":1.736},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.357,"15min":1.3640000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.738,"5min":2.121,"15min":2.121},"last":1.6559999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2929999999999999,"15min":1.2949999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8759999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.746,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6419999999999999,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.3}]},{"osd":6,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3819999999999999,"15min":1.3839999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.825,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.9450000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.349}]},{"osd":7,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.405,"15min":1.415},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.617,"5min":2.286,"15min":2.286},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.577,"15min":1.5940000000000001},"min":{"1min":1.008,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.9139999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.9359999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542177,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0589999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.075},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97999999999999998},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.96799999999999997}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0249999999999999,"15min":1.0449999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.458,"5min":1.78,"15min":1.78},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.97599999999999998,"15min":0.99199999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.24,"5min":1.623,"15min":1.623},"last":0.90700000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.55,"5min":3.7799999999999998,"15min":3.7799999999999998},"last":1.339},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.202,"15min":1.2290000000000001},"min":{"1min":0.628,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5920000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.2589999999999999}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3620000000000001,"15min":1.3799999999999999},"min":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.369,"15min":1.383},"min":{"1min":0.88500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.7190000000000001,"5min":2.573,"15min":2.573},"last":1.671}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.5629999999999999,"15min":1.6000000000000001},"min":{"1min":0.99399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":1.851},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.514,"15min":1.5700000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6299999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":1.3140000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.534,"15min":1.5580000000000001},"min":{"1min":1.0569999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.873,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.6060000000000001,"15min":1.6459999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.758,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":1.389}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.6499999999999999,"15min":1.6899999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.825,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.6379999999999999,"15min":1.6879999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.738,"5min":3.4340000000000002,"15min":3.4340000000000002},"last":1.4430000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542176,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":223705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89900000000000002,"15min":0.90300000000000002},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3009999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.121},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.024,"15min":1.036},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.405,"5min":1.865,"15min":1.865},"last":0.92000000000000004}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.383,"5min":1.468,"15min":1.468},"last":1.333},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.079,"15min":1.091},"min":{"1min":0.64900000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.1850000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4570000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.2},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.879,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.4119999999999999}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4139999999999999,"15min":1.4179999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.8759999999999999,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.93100000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.847,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.4770000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.508,"15min":1.5229999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9730000000000001,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.4399999999999999,"15min":1.462},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6990000000000001,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.4339999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7829999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.5069999999999999,"15min":1.5129999999999999},"min":{"1min":0.97799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.0289999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.6040000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.6060000000000001,"15min":1.6279999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9279999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.593,"15min":1.6140000000000001},"min":{"1min":1.054,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.907,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":1.55}]}]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":499712,"data_stored":222141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.575,"15min":1.575},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0089999999999999,"15min":1.0149999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.69499999999999995}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.026,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.782,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.034,"5min":1.0569999999999999,"15min":1.0609999999999999},"min":{"1min":0.64000000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.377,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.1879999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.282,"15min":1.292},"min":{"1min":0.70999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6200000000000001,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.2709999999999999,"15min":1.266},"min":{"1min":0.62,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.962,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":0.745}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.411,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.7410000000000001,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4199999999999999,"15min":1.4359999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.994,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":1.1160000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1059999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.018},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5009999999999999,"15min":1.51},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.8700000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.304}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.524,"15min":1.518},"min":{"1min":1.0529999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.3500000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.458,"15min":1.4670000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.919,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.2230000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.5209999999999999,"15min":1.524},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.0449999999999999,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.617,"15min":1.617},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":2.4529999999999998,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.3999999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542178,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1093632,"data_stored":812473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.83799999999999997},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.109,"5min":1.288,"15min":1.288},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":0.92100000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.379}]},{"osd":2,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.052},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0249999999999999,"15min":1.0129999999999999},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.113}]},{"osd":3,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.51200000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3080000000000001,"5min":1.522,"15min":1.522},"last":1.857},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.137,"15min":1.149},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6879999999999999,"5min":1.728,"15min":1.728},"last":1.329}]},{"osd":4,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.323,"15min":1.3360000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.724,"5min":2.23,"15min":2.23},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2869999999999999,"15min":1.298},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.738}]},{"osd":5,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.333,"15min":1.337},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.889,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.524},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.4330000000000001,"15min":1.4490000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.7549999999999999,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":1.587}]},{"osd":6,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.3460000000000001,"15min":1.3700000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.677,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.22},{"interface":"front","average":{"1min":1.347,"5min":1.413,"15min":1.4239999999999999},"min":{"1min":0.877,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.6579999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4990000000000001,"15min":1.5069999999999999},"min":{"1min":0.86199999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.99,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.7},{"interface":"front","average":{"1min":1.337,"5min":1.468,"15min":1.49},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.7729999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4239999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:53:14.854 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-11T15:53:14.854 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-11T15:53:15.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.136+0000 7fe4dbe46640 1 Processor -- start 2023-08-11T15:53:15.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.137+0000 7fe4dbe46640 1 -- start start 2023-08-11T15:53:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.138+0000 7fe4dbe46640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4d4151220 0x7fe4d406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.138+0000 7fe4dbe46640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d415e680 0x7fe4d40ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.138+0000 7fe4dbe46640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d406c850 con 0x7fe4d406bdf0 2023-08-11T15:53:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.138+0000 7fe4dbe46640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d406c290 con 0x7fe4d4151220 2023-08-11T15:53:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.138+0000 7fe4dbe46640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d406c570 con 0x7fe4d415e680 2023-08-11T15:53:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.139+0000 7fe4d9be2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4d4151220 0x7fe4d406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.139+0000 7fe4d9be2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4d4151220 0x7fe4d406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40318/0 (socket says 172.21.15.27:40318) 2023-08-11T15:53:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.139+0000 7fe4d9be2640 1 -- 172.21.15.27:0/1444407465 learned_addr learned my addr 172.21.15.27:0/1444407465 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.139+0000 7fe4da3e3640 1 --2- 172.21.15.27:0/1444407465 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d415e680 0x7fe4d40ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.139+0000 7fe4d9be2640 1 -- 172.21.15.27:0/1444407465 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d415e680 msgr2=0x7fe4d40ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.139+0000 7fe4d9be2640 1 --2- 172.21.15.27:0/1444407465 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d415e680 0x7fe4d40ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.139+0000 7fe4d9be2640 1 -- 172.21.15.27:0/1444407465 >> v1:172.21.15.119:6789/0 conn(0x7fe4d406bdf0 legacy=0x7fe4d415df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:15.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.139+0000 7fe4d9be2640 1 -- 172.21.15.27:0/1444407465 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d40ca050 con 0x7fe4d4151220 2023-08-11T15:53:15.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.139+0000 7fe4da3e3640 1 --2- 172.21.15.27:0/1444407465 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d415e680 0x7fe4d40ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:15.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.141+0000 7fe4d9be2640 1 --2- 172.21.15.27:0/1444407465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d4151220 0x7fe4d406b8b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009e50 tx=0x7fe4d002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2209a362c5a001b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.141+0000 7fe4d8be0640 1 -- 172.21.15.27:0/1444407465 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d002c960 con 0x7fe4d4151220 2023-08-11T15:53:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.141+0000 7fe4d8be0640 1 -- 172.21.15.27:0/1444407465 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d002cac0 con 0x7fe4d4151220 2023-08-11T15:53:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.141+0000 7fe4d8be0640 1 -- 172.21.15.27:0/1444407465 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d002ce50 con 0x7fe4d4151220 2023-08-11T15:53:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.142+0000 7fe4dbe46640 1 -- 172.21.15.27:0/1444407465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d4151220 msgr2=0x7fe4d406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.142+0000 7fe4dbe46640 1 --2- 172.21.15.27:0/1444407465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d4151220 0x7fe4d406b8b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009e50 tx=0x7fe4d002bdc0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.142+0000 7fe4dbe46640 1 -- 172.21.15.27:0/1444407465 shutdown_connections 2023-08-11T15:53:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.142+0000 7fe4dbe46640 1 --2- 172.21.15.27:0/1444407465 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d415e680 0x7fe4d40ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.142+0000 7fe4dbe46640 1 --2- 172.21.15.27:0/1444407465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d4151220 0x7fe4d406b8b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.142+0000 7fe4dbe46640 1 -- 172.21.15.27:0/1444407465 >> 172.21.15.27:0/1444407465 conn(0x7fe4d40afbe0 msgr2=0x7fe4d40affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.142+0000 7fe4dbe46640 1 -- 172.21.15.27:0/1444407465 shutdown_connections 2023-08-11T15:53:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.142+0000 7fe4dbe46640 1 -- 172.21.15.27:0/1444407465 wait complete. 2023-08-11T15:53:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.144+0000 7fe4dbe46640 1 Processor -- start 2023-08-11T15:53:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.145+0000 7fe4dbe46640 1 -- start start 2023-08-11T15:53:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.146+0000 7fe4dbe46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d406bdf0 0x7fe4d414d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.146+0000 7fe4dbe46640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d4151220 0x7fe4d4152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.146+0000 7fe4dbe46640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d415e680 0x7fe4d4153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.146+0000 7fe4dbe46640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d4153f30 con 0x7fe4d415e680 2023-08-11T15:53:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.146+0000 7fe4dbe46640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d4153690 con 0x7fe4d406bdf0 2023-08-11T15:53:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.146+0000 7fe4dbe46640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d4153970 con 0x7fe4d4151220 2023-08-11T15:53:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.146+0000 7fe4da3e3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d415e680 0x7fe4d4153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.146+0000 7fe4da3e3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d415e680 0x7fe4d4153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40322/0 (socket says 172.21.15.27:40322) 2023-08-11T15:53:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.146+0000 7fe4da3e3640 1 -- 172.21.15.27:0/990044803 learned_addr learned my addr 172.21.15.27:0/990044803 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.147+0000 7fe4d9be2640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d406bdf0 0x7fe4d414d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.147+0000 7fe4d93e1640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d4151220 0x7fe4d4152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.147+0000 7fe4da3e3640 1 -- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d4151220 msgr2=0x7fe4d4152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.147+0000 7fe4da3e3640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d4151220 0x7fe4d4152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.147+0000 7fe4da3e3640 1 -- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d406bdf0 msgr2=0x7fe4d414d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.147+0000 7fe4da3e3640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d406bdf0 0x7fe4d414d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.147+0000 7fe4da3e3640 1 -- 172.21.15.27:0/990044803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d0009b30 con 0x7fe4d415e680 2023-08-11T15:53:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.147+0000 7fe4d9be2640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d406bdf0 0x7fe4d414d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.147+0000 7fe4da3e3640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d415e680 0x7fe4d4153080 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c7c0 tx=0x7fe4cc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.148+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/990044803 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc004230 con 0x7fe4d415e680 2023-08-11T15:53:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.148+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/990044803 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc004390 con 0x7fe4d415e680 2023-08-11T15:53:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.148+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/990044803 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc007860 con 0x7fe4d415e680 2023-08-11T15:53:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.148+0000 7fe4dbe46640 1 -- 172.21.15.27:0/990044803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d4153c50 con 0x7fe4d415e680 2023-08-11T15:53:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.148+0000 7fe4dbe46640 1 -- 172.21.15.27:0/990044803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d40cce10 con 0x7fe4d415e680 2023-08-11T15:53:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.149+0000 7fe4dbe46640 1 -- 172.21.15.27:0/990044803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d40b4040 con 0x7fe4d415e680 2023-08-11T15:53:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.150+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/990044803 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe4cc0079c0 con 0x7fe4d415e680 2023-08-11T15:53:15.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.151+0000 7fe4ca7fc640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4b806a050 0x7fe4b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.151+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/990044803 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe4cc0888d0 con 0x7fe4d415e680 2023-08-11T15:53:15.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.152+0000 7fe4d9be2640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4b806a050 0x7fe4b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.153+0000 7fe4d9be2640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4b806a050 0x7fe4b806c510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0035000 tx=0x7fe4d0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:15.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.156+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/990044803 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe4cc054080 con 0x7fe4d415e680 2023-08-11T15:53:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.422+0000 7fe4dbe46640 1 -- 172.21.15.27:0/990044803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe4d40b2050 con 0x7fe4d415e680 2023-08-11T15:53:15.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.429+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/990044803 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe4d40b2050 con 0x7fe4d415e680 2023-08-11T15:53:15.432 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2023-08-11T15:53:15.432 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCgWNZkD5/iEhAAZ4I6N46HjdmeGnn6p1vIvg== 2023-08-11T15:53:15.432 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2023-08-11T15:53:15.432 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2023-08-11T15:53:15.433 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-11T15:53:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-08-11T15:53:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 -- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4b806a050 msgr2=0x7fe4b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4b806a050 0x7fe4b806c510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0035000 tx=0x7fe4d0030000 comp rx=0 tx=0).stop 2023-08-11T15:53:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 -- 172.21.15.27:0/990044803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d415e680 msgr2=0x7fe4d4153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d415e680 0x7fe4d4153080 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c7c0 tx=0x7fe4cc00cc80 comp rx=0 tx=0).stop 2023-08-11T15:53:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 -- 172.21.15.27:0/990044803 shutdown_connections 2023-08-11T15:53:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe4b806a050 0x7fe4b806c510 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d4151220 0x7fe4d4152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d415e680 0x7fe4d4153080 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 --2- 172.21.15.27:0/990044803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d406bdf0 0x7fe4d414d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 -- 172.21.15.27:0/990044803 >> 172.21.15.27:0/990044803 conn(0x7fe4d40afbe0 msgr2=0x7fe4d40c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:15.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 -- 172.21.15.27:0/990044803 shutdown_connections 2023-08-11T15:53:15.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.432+0000 7fe4dbe46640 1 -- 172.21.15.27:0/990044803 wait complete. 2023-08-11T15:53:15.455 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:53:15.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.680+0000 7faa9abfd640 1 Processor -- start 2023-08-11T15:53:15.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.680+0000 7faa9abfd640 1 -- start start 2023-08-11T15:53:15.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.681+0000 7faa9abfd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa940cb430 0x7faa940b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.681+0000 7faa9abfd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9406f5d0 0x7faa9406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.682+0000 7faa9abfd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa940c8a30 con 0x7faa9415a140 2023-08-11T15:53:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.682+0000 7faa9abfd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa9406ffa0 con 0x7faa940cb430 2023-08-11T15:53:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.682+0000 7faa9abfd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9406c0c0 con 0x7faa9406f5d0 2023-08-11T15:53:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.682+0000 7faa98999640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faa9415a140 0x7faa9406b870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49042/0 (socket says 172.21.15.27:49042) 2023-08-11T15:53:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.682+0000 7faa98999640 1 -- 172.21.15.27:0/368171674 learned_addr learned my addr 172.21.15.27:0/368171674 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.682+0000 7faa93fff640 1 --2- 172.21.15.27:0/368171674 >> v2:172.21.15.119:3300/0 conn(0x7faa940cb430 0x7faa940b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.682+0000 7faa9919a640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9406f5d0 0x7faa9406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.682+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607877858 0 0) 0x7faa940c8a30 con 0x7faa9415a140 2023-08-11T15:53:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa84003520 con 0x7faa9415a140 2023-08-11T15:53:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 287741605 0 0) 0x7faa84003520 con 0x7faa9415a140 2023-08-11T15:53:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9406f5d0 msgr2=0x7faa9406fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9406f5d0 0x7faa9406fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 >> v2:172.21.15.119:3300/0 conn(0x7faa940cb430 msgr2=0x7faa940b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 --2- 172.21.15.27:0/368171674 >> v2:172.21.15.119:3300/0 conn(0x7faa940cb430 0x7faa940b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9406c3a0 con 0x7faa9415a140 2023-08-11T15:53:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7faa880017f0 con 0x7faa9415a140 2023-08-11T15:53:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 >> v1:172.21.15.27:6789/0 conn(0x7faa9415a140 legacy=0x7faa9406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9406f5d0 0x7faa84003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa84003ef0 0x7faa840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 0x7faa84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9406ffa0 con 0x7faa84004d00 2023-08-11T15:53:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9406c0c0 con 0x7faa84003ef0 2023-08-11T15:53:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8400b980 con 0x7faa9406f5d0 2023-08-11T15:53:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.683+0000 7faa9919a640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 0x7faa84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.684+0000 7faa93fff640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa84003ef0 0x7faa840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.684+0000 7faa98999640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9406f5d0 0x7faa84003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.684+0000 7faa9919a640 1 -- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9406f5d0 msgr2=0x7faa84003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.684+0000 7faa9919a640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9406f5d0 0x7faa84003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.684+0000 7faa9919a640 1 -- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa84003ef0 msgr2=0x7faa840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.684+0000 7faa9919a640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa84003ef0 0x7faa840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.684+0000 7faa9919a640 1 -- 172.21.15.27:0/368171674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa84003520 con 0x7faa84004d00 2023-08-11T15:53:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.684+0000 7faa98999640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9406f5d0 0x7faa84003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.685+0000 7faa9919a640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 0x7faa84005180 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009fc0 tx=0x7faa840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b6c2384c3765c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.685+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c02cbe0 con 0x7faa84004d00 2023-08-11T15:53:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.685+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8c00f040 con 0x7faa84004d00 2023-08-11T15:53:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.685+0000 7faa937fe640 1 -- 172.21.15.27:0/368171674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c034440 con 0x7faa84004d00 2023-08-11T15:53:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.686+0000 7faa9abfd640 1 -- 172.21.15.27:0/368171674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 msgr2=0x7faa84005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.686+0000 7faa9abfd640 1 --2- 172.21.15.27:0/368171674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 0x7faa84005180 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009fc0 tx=0x7faa840057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.686+0000 7faa9919a640 1 -- 172.21.15.27:0/368171674 reap_dead start 2023-08-11T15:53:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.686+0000 7faa9abfd640 1 -- 172.21.15.27:0/368171674 shutdown_connections 2023-08-11T15:53:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.686+0000 7faa9abfd640 1 -- 172.21.15.27:0/368171674 >> 172.21.15.27:0/368171674 conn(0x7faa940afb70 msgr2=0x7faa940aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.686+0000 7faa9abfd640 1 -- 172.21.15.27:0/368171674 shutdown_connections 2023-08-11T15:53:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.686+0000 7faa9abfd640 1 -- 172.21.15.27:0/368171674 wait complete. 2023-08-11T15:53:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.688+0000 7faa9abfd640 1 Processor -- start 2023-08-11T15:53:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.688+0000 7faa9abfd640 1 -- start start 2023-08-11T15:53:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.689+0000 7faa9abfd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 0x7faa940c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.689+0000 7faa9abfd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9415a600 0x7faa940b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.689+0000 7faa9abfd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa940b1de0 0x7faa940b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.689+0000 7faa9abfd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9415d3f0 con 0x7faa84004d00 2023-08-11T15:53:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.689+0000 7faa9abfd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9415ce30 con 0x7faa940b1de0 2023-08-11T15:53:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.689+0000 7faa9abfd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9415d110 con 0x7faa9415a600 2023-08-11T15:53:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.689+0000 7faa98999640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 0x7faa940c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.689+0000 7faa98999640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 0x7faa940c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40344/0 (socket says 172.21.15.27:40344) 2023-08-11T15:53:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.689+0000 7faa98999640 1 -- 172.21.15.27:0/3292061412 learned_addr learned my addr 172.21.15.27:0/3292061412 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.690+0000 7faa98999640 1 -- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9415a600 msgr2=0x7faa940b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.690+0000 7faa93fff640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9415a600 0x7faa940b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.690+0000 7faa9919a640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa940b1de0 0x7faa940b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.690+0000 7faa98999640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9415a600 0x7faa940b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.690+0000 7faa98999640 1 -- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa940b1de0 msgr2=0x7faa940b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.690+0000 7faa98999640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa940b1de0 0x7faa940b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.690+0000 7faa98999640 1 -- 172.21.15.27:0/3292061412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa940c1ed0 con 0x7faa84004d00 2023-08-11T15:53:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.690+0000 7faa9919a640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa940b1de0 0x7faa940b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.690+0000 7faa98999640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 0x7faa940c4260 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faa8800d8e0 tx=0x7faa8800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.690+0000 7faa917fa640 1 -- 172.21.15.27:0/3292061412 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88007480 con 0x7faa84004d00 2023-08-11T15:53:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.691+0000 7faa917fa640 1 -- 172.21.15.27:0/3292061412 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa880075e0 con 0x7faa84004d00 2023-08-11T15:53:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.691+0000 7faa917fa640 1 -- 172.21.15.27:0/3292061412 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa880059a0 con 0x7faa84004d00 2023-08-11T15:53:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.691+0000 7faa9abfd640 1 -- 172.21.15.27:0/3292061412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa940c21b0 con 0x7faa84004d00 2023-08-11T15:53:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.691+0000 7faa9abfd640 1 -- 172.21.15.27:0/3292061412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa940b2980 con 0x7faa84004d00 2023-08-11T15:53:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.692+0000 7faa9abfd640 1 -- 172.21.15.27:0/3292061412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa940c1d10 con 0x7faa84004d00 2023-08-11T15:53:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.693+0000 7faa917fa640 1 -- 172.21.15.27:0/3292061412 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faa88005b00 con 0x7faa84004d00 2023-08-11T15:53:15.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.694+0000 7faa917fa640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa7806a070 0x7faa7806c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:15.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.694+0000 7faa917fa640 1 -- 172.21.15.27:0/3292061412 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7faa880889d0 con 0x7faa84004d00 2023-08-11T15:53:15.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.695+0000 7faa93fff640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa7806a070 0x7faa7806c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:15.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.696+0000 7faa93fff640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa7806a070 0x7faa7806c530 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faa940c1ca0 tx=0x7faa80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:15.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.699+0000 7faa917fa640 1 -- 172.21.15.27:0/3292061412 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7faa88057990 con 0x7faa84004d00 2023-08-11T15:53:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.930+0000 7faa9abfd640 1 -- 172.21.15.27:0/3292061412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa94073990 con 0x7faa84004d00 2023-08-11T15:53:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.931+0000 7faa917fa640 1 -- 172.21.15.27:0/3292061412 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4916 (secure 0 0 0) 0x7faa88005db0 con 0x7faa84004d00 2023-08-11T15:53:15.935 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:15.936 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2294518038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2294518038},{"type":"v1","addr":"172.21.15.119:6836","nonce":2294518038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5188,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2104666724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2104666724},{"type":"v1","addr":"172.21.15.119:6837","nonce":2104666724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2379252997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2379252997},{"type":"v1","addr":"172.21.15.27:6833","nonce":2379252997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:53:09.220611+0000","modified":"2023-08-11T15:53:12.792531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5643},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5643":{"gid":5643,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/899405615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899405615},{"type":"v1","addr":"172.21.15.27:6835","nonce":899405615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-11T15:53:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 18 2023-08-11T15:53:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.933+0000 7faa9abfd640 1 -- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa7806a070 msgr2=0x7faa7806c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.933+0000 7faa9abfd640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa7806a070 0x7faa7806c530 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faa940c1ca0 tx=0x7faa80008040 comp rx=0 tx=0).stop 2023-08-11T15:53:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.933+0000 7faa9abfd640 1 -- 172.21.15.27:0/3292061412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 msgr2=0x7faa940c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.933+0000 7faa9abfd640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 0x7faa940c4260 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faa8800d8e0 tx=0x7faa8800dda0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.934+0000 7faa9abfd640 1 -- 172.21.15.27:0/3292061412 shutdown_connections 2023-08-11T15:53:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.934+0000 7faa9abfd640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7faa7806a070 0x7faa7806c530 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.934+0000 7faa9abfd640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa9415a600 0x7faa940b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.934+0000 7faa9abfd640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa940b1de0 0x7faa940b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.934+0000 7faa9abfd640 1 --2- 172.21.15.27:0/3292061412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa84004d00 0x7faa940c4260 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.934+0000 7faa9abfd640 1 -- 172.21.15.27:0/3292061412 >> 172.21.15.27:0/3292061412 conn(0x7faa940afb70 msgr2=0x7faa940cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.934+0000 7faa9abfd640 1 -- 172.21.15.27:0/3292061412 shutdown_connections 2023-08-11T15:53:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:15.934+0000 7faa9abfd640 1 -- 172.21.15.27:0/3292061412 wait complete. 2023-08-11T15:53:16.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-11T15:53:09.220611+0000', 'modified': '2023-08-11T15:53:12.792531+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5643}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5643': {'gid': 5643, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.27:6835/899405615', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 899405615}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 899405615}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-11T15:53:16.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-11T15:53:16.359 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-11T15:53:16.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.602+0000 7fe199b29640 1 Processor -- start 2023-08-11T15:53:16.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.602+0000 7fe199b29640 1 -- start start 2023-08-11T15:53:16.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe199b29640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940cd500 0x7fe19415feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe199b29640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1940cf670 con 0x7fe1940b7ab0 2023-08-11T15:53:16.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe199b29640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe194160e50 con 0x7fe19415c3c0 2023-08-11T15:53:16.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe199b29640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19415c7a0 con 0x7fe1940cd500 2023-08-11T15:53:16.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe199328640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe1940b7ab0 0x7fe1940d0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49054/0 (socket says 172.21.15.27:49054) 2023-08-11T15:53:16.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe199328640 1 -- 172.21.15.27:0/2540678605 learned_addr learned my addr 172.21.15.27:0/2540678605 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:16.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe193fff640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940cd500 0x7fe19415feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429909096 0 0) 0x7fe1940cf670 con 0x7fe1940b7ab0 2023-08-11T15:53:16.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17c003520 con 0x7fe1940b7ab0 2023-08-11T15:53:16.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2520016014 0 0) 0x7fe17c003520 con 0x7fe1940b7ab0 2023-08-11T15:53:16.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940cd500 msgr2=0x7fe19415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940cd500 0x7fe19415feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 >> v1:172.21.15.119:6789/0 conn(0x7fe19415c3c0 legacy=0x7fe1940ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1941605b0 con 0x7fe1940b7ab0 2023-08-11T15:53:16.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fe18c001670 con 0x7fe1940b7ab0 2023-08-11T15:53:16.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 >> v1:172.21.15.27:6789/0 conn(0x7fe1940b7ab0 legacy=0x7fe1940d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c0039a0 0x7fe17c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940cd500 0x7fe17c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe17c004d00 0x7fe17c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19415c7a0 con 0x7fe17c004d00 2023-08-11T15:53:16.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c00ba60 con 0x7fe17c0039a0 2023-08-11T15:53:16.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.604+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c00bc20 con 0x7fe1940cd500 2023-08-11T15:53:16.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.605+0000 7fe199328640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe17c004d00 0x7fe17c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.605+0000 7fe198b27640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c0039a0 0x7fe17c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.605+0000 7fe193fff640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940cd500 0x7fe17c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.605+0000 7fe199328640 1 -- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940cd500 msgr2=0x7fe17c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.605+0000 7fe199328640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940cd500 0x7fe17c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.605+0000 7fe199328640 1 -- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c0039a0 msgr2=0x7fe17c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.605+0000 7fe199328640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c0039a0 0x7fe17c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.605+0000 7fe199328640 1 -- 172.21.15.27:0/2540678605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c003520 con 0x7fe17c004d00 2023-08-11T15:53:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.605+0000 7fe198b27640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c0039a0 0x7fe17c003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.606+0000 7fe199328640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe17c004d00 0x7fe17c005180 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c190 tx=0x7fe17c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62199b967606683a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.607+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c005ad0 con 0x7fe17c004d00 2023-08-11T15:53:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.607+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c005c50 con 0x7fe17c004d00 2023-08-11T15:53:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.607+0000 7fe1937fe640 1 -- 172.21.15.27:0/2540678605 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c02c830 con 0x7fe17c004d00 2023-08-11T15:53:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.607+0000 7fe199b29640 1 -- 172.21.15.27:0/2540678605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe17c004d00 msgr2=0x7fe17c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.607+0000 7fe199b29640 1 --2- 172.21.15.27:0/2540678605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe17c004d00 0x7fe17c005180 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c190 tx=0x7fe17c0057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.607+0000 7fe199328640 1 -- 172.21.15.27:0/2540678605 reap_dead start 2023-08-11T15:53:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.607+0000 7fe199b29640 1 -- 172.21.15.27:0/2540678605 shutdown_connections 2023-08-11T15:53:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.607+0000 7fe199b29640 1 -- 172.21.15.27:0/2540678605 >> 172.21.15.27:0/2540678605 conn(0x7fe19406a190 msgr2=0x7fe19406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.607+0000 7fe199b29640 1 -- 172.21.15.27:0/2540678605 shutdown_connections 2023-08-11T15:53:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.607+0000 7fe199b29640 1 -- 172.21.15.27:0/2540678605 wait complete. 2023-08-11T15:53:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.609+0000 7fe199b29640 1 Processor -- start 2023-08-11T15:53:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.609+0000 7fe199b29640 1 -- start start 2023-08-11T15:53:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.610+0000 7fe199b29640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c004d00 0x7fe1941f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.610+0000 7fe199b29640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940b3f40 0x7fe1940cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.610+0000 7fe199b29640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1940b2fe0 0x7fe19415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.610+0000 7fe199b29640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940b4500 con 0x7fe1940b2fe0 2023-08-11T15:53:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.610+0000 7fe199b29640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940b2220 con 0x7fe17c004d00 2023-08-11T15:53:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.610+0000 7fe199b29640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940b2500 con 0x7fe1940b3f40 2023-08-11T15:53:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe199328640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1940b2fe0 0x7fe19415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe199328640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1940b2fe0 0x7fe19415d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40362/0 (socket says 172.21.15.27:40362) 2023-08-11T15:53:16.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe199328640 1 -- 172.21.15.27:0/1793937846 learned_addr learned my addr 172.21.15.27:0/1793937846 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:16.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe193fff640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940b3f40 0x7fe1940cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe198b27640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c004d00 0x7fe1941f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe199328640 1 -- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940b3f40 msgr2=0x7fe1940cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe199328640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940b3f40 0x7fe1940cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe199328640 1 -- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c004d00 msgr2=0x7fe1941f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe199328640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c004d00 0x7fe1941f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe199328640 1 -- 172.21.15.27:0/1793937846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1940b27e0 con 0x7fe1940b2fe0 2023-08-11T15:53:16.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe198b27640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c004d00 0x7fe1941f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:16.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.611+0000 7fe199328640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1940b2fe0 0x7fe19415d9c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c190 tx=0x7fe18c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:16.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.612+0000 7fe1917fa640 1 -- 172.21.15.27:0/1793937846 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c03c040 con 0x7fe1940b2fe0 2023-08-11T15:53:16.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.612+0000 7fe1917fa640 1 -- 172.21.15.27:0/1793937846 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c02cc40 con 0x7fe1940b2fe0 2023-08-11T15:53:16.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.612+0000 7fe1917fa640 1 -- 172.21.15.27:0/1793937846 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c007070 con 0x7fe1940b2fe0 2023-08-11T15:53:16.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.612+0000 7fe199b29640 1 -- 172.21.15.27:0/1793937846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1940b2ac0 con 0x7fe1940b2fe0 2023-08-11T15:53:16.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.612+0000 7fe199b29640 1 -- 172.21.15.27:0/1793937846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1940cbe40 con 0x7fe1940b2fe0 2023-08-11T15:53:16.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.614+0000 7fe1917fa640 1 -- 172.21.15.27:0/1793937846 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe18c032070 con 0x7fe1940b2fe0 2023-08-11T15:53:16.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.615+0000 7fe1917fa640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe18006a050 0x7fe18006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.616+0000 7fe1917fa640 1 -- 172.21.15.27:0/1793937846 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe18c0a98f0 con 0x7fe1940b2fe0 2023-08-11T15:53:16.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.616+0000 7fe19d9db740 1 Processor -- start 2023-08-11T15:53:16.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.616+0000 7fe198b27640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe18006a050 0x7fe18006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.616+0000 7fe19d9db740 1 -- start start 2023-08-11T15:53:16.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.617+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca20e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.617+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f09f0 0x557cbca1dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.617+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbc9f45f0 0x557cbca1e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.617+0000 7fe19d9db740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbc962ef0 con 0x557cbc9f45f0 2023-08-11T15:53:16.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.617+0000 7fe19d9db740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbc9605e0 con 0x557cbca20a70 2023-08-11T15:53:16.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.617+0000 7fe19d9db740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cbc95bed0 con 0x557cbc9f09f0 2023-08-11T15:53:16.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.617+0000 7fe198b27640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe18006a050 0x7fe18006c510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe184007fd0 tx=0x7fe184009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:16.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe199328640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbc9f45f0 0x557cbca1e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe199328640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbc9f45f0 0x557cbca1e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40376/0 (socket says 172.21.15.27:40376) 2023-08-11T15:53:16.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe199328640 1 -- 172.21.15.27:0/45346480 learned_addr learned my addr 172.21.15.27:0/45346480 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:16.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe198b27640 1 --2- 172.21.15.27:0/45346480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca20e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe199328640 1 -- 172.21.15.27:0/45346480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f09f0 msgr2=0x557cbca1dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe199328640 1 --2- 172.21.15.27:0/45346480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f09f0 0x557cbca1dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe199328640 1 -- 172.21.15.27:0/45346480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 msgr2=0x557cbca20e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe199328640 1 --2- 172.21.15.27:0/45346480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca20e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe199328640 1 -- 172.21.15.27:0/45346480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cbc960b40 con 0x557cbc9f45f0 2023-08-11T15:53:16.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe198b27640 1 --2- 172.21.15.27:0/45346480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca20e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:16.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.618+0000 7fe199328640 1 --2- 172.21.15.27:0/45346480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbc9f45f0 0x557cbca1e220 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe18c075050 tx=0x7fe18c078a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa9cdc3eb642c28 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:16.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe17affd640 1 -- 172.21.15.27:0/45346480 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c07c910 con 0x557cbc9f45f0 2023-08-11T15:53:16.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe17affd640 1 -- 172.21.15.27:0/45346480 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c080860 con 0x557cbc9f45f0 2023-08-11T15:53:16.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe17affd640 1 -- 172.21.15.27:0/45346480 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c007c80 con 0x557cbc9f45f0 2023-08-11T15:53:16.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe19d9db740 1 -- 172.21.15.27:0/45346480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbc9f45f0 msgr2=0x557cbca1e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe19d9db740 1 --2- 172.21.15.27:0/45346480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbc9f45f0 0x557cbca1e220 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe18c075050 tx=0x7fe18c078a20 comp rx=0 tx=0).stop 2023-08-11T15:53:16.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe19d9db740 1 -- 172.21.15.27:0/45346480 shutdown_connections 2023-08-11T15:53:16.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe19d9db740 1 --2- 172.21.15.27:0/45346480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f09f0 0x557cbca1dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe19d9db740 1 --2- 172.21.15.27:0/45346480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbc9f45f0 0x557cbca1e220 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe19d9db740 1 --2- 172.21.15.27:0/45346480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca20e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe19d9db740 1 -- 172.21.15.27:0/45346480 >> 172.21.15.27:0/45346480 conn(0x557cbc9f2d00 msgr2=0x557cbca3b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:16.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe19d9db740 1 -- 172.21.15.27:0/45346480 shutdown_connections 2023-08-11T15:53:16.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.619+0000 7fe19d9db740 1 -- 172.21.15.27:0/45346480 wait complete. 2023-08-11T15:53:16.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.620+0000 7fe19d9db740 1 -- start start 2023-08-11T15:53:16.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.620+0000 7fe19d9db740 1 Processor -- start 2023-08-11T15:53:16.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.621+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f45f0 0x557cbca0ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.621+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca1b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.621+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca08080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.621+0000 7fe19d9db740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbc8fbdc0 con 0x557cbca20e50 2023-08-11T15:53:16.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.621+0000 7fe19d9db740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbc8fba60 con 0x557cbca20a70 2023-08-11T15:53:16.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.621+0000 7fe19d9db740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cbc6c1f20 con 0x557cbc9f45f0 2023-08-11T15:53:16.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe19d9db740 10 client.0 _ll_register_callbacks cb 0x557cbc953d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:16.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe199328640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca08080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe199328640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca08080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40386/0 (socket says 172.21.15.27:40386) 2023-08-11T15:53:16.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe199328640 1 -- 172.21.15.27:0/112390971 learned_addr learned my addr 172.21.15.27:0/112390971 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:16.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe198b27640 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f45f0 0x557cbca0ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe193fff640 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca1b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe199328640 1 -- 172.21.15.27:0/112390971 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f45f0 msgr2=0x557cbca0ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe199328640 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f45f0 0x557cbca0ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe199328640 1 -- 172.21.15.27:0/112390971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 msgr2=0x557cbca1b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe199328640 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca1b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe199328640 1 -- 172.21.15.27:0/112390971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1940cbe40 con 0x557cbca20e50 2023-08-11T15:53:16.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.622+0000 7fe193fff640 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca1b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:16.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe199328640 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca08080 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe18c078ae0 tx=0x7fe18c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121432f30fe96770 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:16.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe1737fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-08-11T15:53:16.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe1737fe640 1 -- 172.21.15.27:0/112390971 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c07c910 con 0x557cbca20e50 2023-08-11T15:53:16.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe19d9db740 1 -- 172.21.15.27:0/112390971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cbc933cf0 con 0x557cbca20e50 2023-08-11T15:53:16.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe1737fe640 1 -- 172.21.15.27:0/112390971 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe18c07cb70 con 0x557cbca20e50 2023-08-11T15:53:16.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe1737fe640 1 -- 172.21.15.27:0/112390971 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe18c07cd30 con 0x557cbca20e50 2023-08-11T15:53:16.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe19d9db740 10 client.5823 fetch_fsmap learned FSMap version 18 2023-08-11T15:53:16.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe19d9db740 1 -- 172.21.15.27:0/112390971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x557cbc95a550 con 0x557cbca20e50 2023-08-11T15:53:16.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe1737fe640 1 -- 172.21.15.27:0/112390971 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557cbc933cf0 con 0x557cbca20e50 2023-08-11T15:53:16.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe1737fe640 1 -- 172.21.15.27:0/112390971 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe18c048060 con 0x557cbca20e50 2023-08-11T15:53:16.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe19d9db740 10 client.5823 fetch_fsmap finished waiting for FSMap version 18 2023-08-11T15:53:16.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.623+0000 7fe19d9db740 10 client.5823 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-11T15:53:16.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.624+0000 7fe19d9db740 20 client.5823 populate_metadata read hostname 'smithi027' 2023-08-11T15:53:16.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.624+0000 7fe19d9db740 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.27:6834/899405615,v1:172.21.15.27:6835/899405615] conn(0x557cbca078e0 0x557cbca1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.624+0000 7fe19d9db740 4 client.5823 mds_command: new command op to 5643 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:53:16.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.624+0000 7fe19d9db740 1 -- 172.21.15.27:0/112390971 --> [v2:172.21.15.27:6834/899405615,v1:172.21.15.27:6835/899405615] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cbc930c60 con 0x557cbca078e0 2023-08-11T15:53:16.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.624+0000 7fe198b27640 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.27:6834/899405615,v1:172.21.15.27:6835/899405615] conn(0x557cbca078e0 0x557cbca1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.624+0000 7fe198b27640 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.27:6834/899405615,v1:172.21.15.27:6835/899405615] conn(0x557cbca078e0 0x557cbca1eb00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b5fcaa5ae644630 server_cookie=dfef7e5a48e64bda in_seq=0 out_seq=0 2023-08-11T15:53:16.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.624+0000 7fe1737fe640 10 client.5823 ms_handle_connect on v2:172.21.15.27:6834/899405615 2023-08-11T15:53:16.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.625+0000 7fe1737fe640 1 -- 172.21.15.27:0/112390971 <== mds.0 v2:172.21.15.27:6834/899405615 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x557cbc930c60 con 0x557cbca078e0 2023-08-11T15:53:16.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.625+0000 7fe1737fe640 10 client.5823 handle_command_reply: tid=0 2023-08-11T15:53:16.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.625+0000 7fe19d9db740 1 client.5823 shutdown 2023-08-11T15:53:16.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.625+0000 7fe19d9db740 1 -- 172.21.15.27:0/112390971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 msgr2=0x557cbca08080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.625+0000 7fe19d9db740 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca08080 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe18c078ae0 tx=0x7fe18c000bc0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.626+0000 7fe19d9db740 1 -- 172.21.15.27:0/112390971 shutdown_connections 2023-08-11T15:53:16.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.626+0000 7fe19d9db740 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.27:6834/899405615,v1:172.21.15.27:6835/899405615] conn(0x557cbca078e0 0x557cbca1eb00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.626+0000 7fe1737fe640 0 client.5823 ms_handle_reset on v2:172.21.15.27:6834/899405615 2023-08-11T15:53:16.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.626+0000 7fe19d9db740 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca08080 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.626+0000 7fe19d9db740 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca1b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.626+0000 7fe19d9db740 1 --2- 172.21.15.27:0/112390971 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f45f0 0x557cbca0ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.626+0000 7fe19d9db740 1 -- 172.21.15.27:0/112390971 >> 172.21.15.27:0/112390971 conn(0x557cbc9f09f0 msgr2=0x557cbca1be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:16.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.626+0000 7fe19d9db740 1 -- 172.21.15.27:0/112390971 shutdown_connections 2023-08-11T15:53:16.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.626+0000 7fe19d9db740 1 -- 172.21.15.27:0/112390971 wait complete. 2023-08-11T15:53:16.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.626+0000 7fe19d9db740 20 client.5823 trim_cache size 0 max 16384 2023-08-11T15:53:16.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.643+0000 7fe19d9db740 1 Processor -- start 2023-08-11T15:53:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.643+0000 7fe19d9db740 1 -- start start 2023-08-11T15:53:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.643+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbc9f45f0 0x557cbca1ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20a70 0x557cbca3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbca20e50 0x557cbca3b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe19d9db740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbca4a5d0 con 0x557cbca20a70 2023-08-11T15:53:16.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe19d9db740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbca4a080 con 0x557cbc9f45f0 2023-08-11T15:53:16.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe19d9db740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cbca21670 con 0x557cbca20e50 2023-08-11T15:53:16.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe193fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20a70 0x557cbca3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe193fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20a70 0x557cbca3b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40392/0 (socket says 172.21.15.27:40392) 2023-08-11T15:53:16.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe193fff640 1 -- 172.21.15.27:0/2787894230 learned_addr learned my addr 172.21.15.27:0/2787894230 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:16.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe199328640 1 --2- 172.21.15.27:0/2787894230 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbca20e50 0x557cbca3b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe198b27640 1 --2- 172.21.15.27:0/2787894230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbc9f45f0 0x557cbca1ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe193fff640 1 -- 172.21.15.27:0/2787894230 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbca20e50 msgr2=0x557cbca3b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe193fff640 1 --2- 172.21.15.27:0/2787894230 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbca20e50 0x557cbca3b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe193fff640 1 -- 172.21.15.27:0/2787894230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbc9f45f0 msgr2=0x557cbca1ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe193fff640 1 --2- 172.21.15.27:0/2787894230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbc9f45f0 0x557cbca1ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe193fff640 1 -- 172.21.15.27:0/2787894230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cbc8faa90 con 0x557cbca20a70 2023-08-11T15:53:16.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.644+0000 7fe198b27640 1 --2- 172.21.15.27:0/2787894230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbc9f45f0 0x557cbca1ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.645+0000 7fe193fff640 1 --2- 172.21.15.27:0/2787894230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20a70 0x557cbca3b260 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe188009950 tx=0x7fe188009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad95e2cd60657726 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.645+0000 7fe17b7fe640 1 -- 172.21.15.27:0/2787894230 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188004220 con 0x557cbca20a70 2023-08-11T15:53:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.645+0000 7fe17b7fe640 1 -- 172.21.15.27:0/2787894230 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1880043a0 con 0x557cbca20a70 2023-08-11T15:53:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.645+0000 7fe17b7fe640 1 -- 172.21.15.27:0/2787894230 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188013ae0 con 0x557cbca20a70 2023-08-11T15:53:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.646+0000 7fe19d9db740 1 -- 172.21.15.27:0/2787894230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20a70 msgr2=0x557cbca3b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.646+0000 7fe19d9db740 1 --2- 172.21.15.27:0/2787894230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20a70 0x557cbca3b260 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe188009950 tx=0x7fe188009e10 comp rx=0 tx=0).stop 2023-08-11T15:53:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.646+0000 7fe19d9db740 1 -- 172.21.15.27:0/2787894230 shutdown_connections 2023-08-11T15:53:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.646+0000 7fe19d9db740 1 --2- 172.21.15.27:0/2787894230 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbca20e50 0x557cbca3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.646+0000 7fe19d9db740 1 --2- 172.21.15.27:0/2787894230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20a70 0x557cbca3b260 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.646+0000 7fe19d9db740 1 --2- 172.21.15.27:0/2787894230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbc9f45f0 0x557cbca1ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.646+0000 7fe19d9db740 1 -- 172.21.15.27:0/2787894230 >> 172.21.15.27:0/2787894230 conn(0x557cbc9f09f0 msgr2=0x557cbca22190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.646+0000 7fe19d9db740 1 -- 172.21.15.27:0/2787894230 shutdown_connections 2023-08-11T15:53:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.646+0000 7fe19d9db740 1 -- 172.21.15.27:0/2787894230 wait complete. 2023-08-11T15:53:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.647+0000 7fe19d9db740 1 -- start start 2023-08-11T15:53:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.647+0000 7fe19d9db740 1 Processor -- start 2023-08-11T15:53:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.648+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f45f0 0x557cbca19470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.648+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.648+0000 7fe19d9db740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca05ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.648+0000 7fe19d9db740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbcaa7a80 con 0x557cbca20e50 2023-08-11T15:53:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.648+0000 7fe19d9db740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cbca3bac0 con 0x557cbca20a70 2023-08-11T15:53:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.648+0000 7fe19d9db740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cbcaa7f70 con 0x557cbc9f45f0 2023-08-11T15:53:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.648+0000 7fe19d9db740 10 client.0 _ll_register_callbacks cb 0x7fe18c001bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.648+0000 7fe199328640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca05ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.648+0000 7fe199328640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca05ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40408/0 (socket says 172.21.15.27:40408) 2023-08-11T15:53:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.648+0000 7fe199328640 1 -- 172.21.15.27:0/797129047 learned_addr learned my addr 172.21.15.27:0/797129047 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe199328640 1 -- 172.21.15.27:0/797129047 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f45f0 msgr2=0x557cbca19470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe199328640 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f45f0 0x557cbca19470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe199328640 1 -- 172.21.15.27:0/797129047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 msgr2=0x557cbca199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe199328640 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe199328640 1 -- 172.21.15.27:0/797129047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe188009630 con 0x557cbca20e50 2023-08-11T15:53:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe199328640 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca05ce0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0018b0 tx=0x7fe18c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad58c37287c62ba server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe1737fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-08-11T15:53:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe1737fe640 1 -- 172.21.15.27:0/797129047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c074b40 con 0x557cbca20e50 2023-08-11T15:53:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe1737fe640 1 -- 172.21.15.27:0/797129047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe18c074ca0 con 0x557cbca20e50 2023-08-11T15:53:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe19d9db740 1 -- 172.21.15.27:0/797129047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cbc983eb0 con 0x557cbca20e50 2023-08-11T15:53:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe1737fe640 1 -- 172.21.15.27:0/797129047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe18c079d00 con 0x557cbca20e50 2023-08-11T15:53:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe19d9db740 10 client.5829 fetch_fsmap learned FSMap version 18 2023-08-11T15:53:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.649+0000 7fe19d9db740 1 -- 172.21.15.27:0/797129047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x557cbc91fe60 con 0x557cbca20e50 2023-08-11T15:53:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.650+0000 7fe1737fe640 1 -- 172.21.15.27:0/797129047 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557cbc983eb0 con 0x557cbca20e50 2023-08-11T15:53:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.650+0000 7fe1737fe640 1 -- 172.21.15.27:0/797129047 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe18c077ab0 con 0x557cbca20e50 2023-08-11T15:53:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.650+0000 7fe19d9db740 10 client.5829 fetch_fsmap finished waiting for FSMap version 18 2023-08-11T15:53:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.650+0000 7fe19d9db740 10 client.5829 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-11T15:53:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.650+0000 7fe19d9db740 20 client.5829 populate_metadata read hostname 'smithi027' 2023-08-11T15:53:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.650+0000 7fe19d9db740 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.27:6834/899405615,v1:172.21.15.27:6835/899405615] conn(0x557cbca06220 0x557cbca06600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.650+0000 7fe19d9db740 4 client.5829 mds_command: new command op to 5643 tid=0[{"prefix": "status"}] 2023-08-11T15:53:16.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.650+0000 7fe19d9db740 1 -- 172.21.15.27:0/797129047 --> [v2:172.21.15.27:6834/899405615,v1:172.21.15.27:6835/899405615] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557cbc954eb0 con 0x557cbca06220 2023-08-11T15:53:16.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.650+0000 7fe198b27640 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.27:6834/899405615,v1:172.21.15.27:6835/899405615] conn(0x557cbca06220 0x557cbca06600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe198b27640 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.27:6834/899405615,v1:172.21.15.27:6835/899405615] conn(0x557cbca06220 0x557cbca06600 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d49b0672c92541d server_cookie=a63be880b54f79f in_seq=0 out_seq=0 2023-08-11T15:53:16.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe1737fe640 10 client.5829 ms_handle_connect on v2:172.21.15.27:6834/899405615 2023-08-11T15:53:16.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe1737fe640 1 -- 172.21.15.27:0/797129047 <== mds.0 v2:172.21.15.27:6834/899405615 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557cbc954eb0 con 0x557cbca06220 2023-08-11T15:53:16.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe1737fe640 10 client.5829 handle_command_reply: tid=0 2023-08-11T15:53:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 client.5829 shutdown 2023-08-11T15:53:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 -- 172.21.15.27:0/797129047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 msgr2=0x557cbca05ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca05ce0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0018b0 tx=0x7fe18c042000 comp rx=0 tx=0).stop 2023-08-11T15:53:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 -- 172.21.15.27:0/797129047 shutdown_connections 2023-08-11T15:53:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.27:6834/899405615,v1:172.21.15.27:6835/899405615] conn(0x557cbca06220 0x557cbca06600 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557cbca20e50 0x557cbca05ce0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557cbca20a70 0x557cbca199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 --2- 172.21.15.27:0/797129047 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557cbc9f45f0 0x557cbca19470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe1737fe640 0 client.5829 ms_handle_reset on v2:172.21.15.27:6834/899405615 2023-08-11T15:53:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 -- 172.21.15.27:0/797129047 >> 172.21.15.27:0/797129047 conn(0x557cbc9f09f0 msgr2=0x557cbca3ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 -- 172.21.15.27:0/797129047 shutdown_connections 2023-08-11T15:53:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.651+0000 7fe19d9db740 1 -- 172.21.15.27:0/797129047 wait complete. 2023-08-11T15:53:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe19d9db740 20 client.5829 trim_cache size 0 max 16384 2023-08-11T15:53:16.668 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:16.669 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "bee60691-4c46-4397-a560-d178aa4db950", 2023-08-11T15:53:16.669 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-08-11T15:53:16.669 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5643, 2023-08-11T15:53:16.670 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-08-11T15:53:16.670 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-08-11T15:53:16.670 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-08-11T15:53:16.670 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 6.8830888979999996, 2023-08-11T15:53:16.671 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 18, 2023-08-11T15:53:16.671 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 43, 2023-08-11T15:53:16.671 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 43, 2023-08-11T15:53:16.672 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 17.619326236999999 2023-08-11T15:53:16.672 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:16.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 -- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe18006a050 msgr2=0x7fe18006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe18006a050 0x7fe18006c510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe184007fd0 tx=0x7fe184009040 comp rx=0 tx=0).stop 2023-08-11T15:53:16.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 -- 172.21.15.27:0/1793937846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1940b2fe0 msgr2=0x7fe19415d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1940b2fe0 0x7fe19415d9c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c190 tx=0x7fe18c02e070 comp rx=0 tx=0).stop 2023-08-11T15:53:16.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 -- 172.21.15.27:0/1793937846 shutdown_connections 2023-08-11T15:53:16.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe18006a050 0x7fe18006c510 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe1940b3f40 0x7fe1940cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1940b2fe0 0x7fe19415d9c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 --2- 172.21.15.27:0/1793937846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe17c004d00 0x7fe1941f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 -- 172.21.15.27:0/1793937846 >> 172.21.15.27:0/1793937846 conn(0x7fe19406a190 msgr2=0x7fe19415b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:16.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 -- 172.21.15.27:0/1793937846 shutdown_connections 2023-08-11T15:53:16.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.652+0000 7fe199b29640 1 -- 172.21.15.27:0/1793937846 wait complete. 2023-08-11T15:53:16.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-11T15:53:16.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.897+0000 7f4ee3fff640 1 Processor -- start 2023-08-11T15:53:16.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.897+0000 7f4ee3fff640 1 -- start start 2023-08-11T15:53:16.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.899+0000 7f4ee3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ee406fce0 0x7f4ee415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.899+0000 7f4ee3fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee40c85f0 con 0x7f4ee415a020 2023-08-11T15:53:16.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.899+0000 7f4ee3fff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee40c8840 con 0x7f4ee40c9600 2023-08-11T15:53:16.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.899+0000 7f4ee3fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee415eae0 con 0x7f4ee406fce0 2023-08-11T15:53:16.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.899+0000 7f4ee2ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4ee415a020 0x7f4ee4071440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49060/0 (socket says 172.21.15.27:49060) 2023-08-11T15:53:16.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.899+0000 7f4ee2ffd640 1 -- 172.21.15.27:0/3390988384 learned_addr learned my addr 172.21.15.27:0/3390988384 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:16.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.899+0000 7f4ee37fe640 1 --2- 172.21.15.27:0/3390988384 >> v2:172.21.15.27:3300/0 conn(0x7f4ee406fce0 0x7f4ee415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.900+0000 7f4ee1ffb640 1 -- 172.21.15.27:0/3390988384 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706472847 0 0) 0x7f4ee40c85f0 con 0x7f4ee415a020 2023-08-11T15:53:16.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.900+0000 7f4ee1ffb640 1 -- 172.21.15.27:0/3390988384 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ecc003520 con 0x7f4ee415a020 2023-08-11T15:53:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.900+0000 7f4ee37fe640 1 -- 172.21.15.27:0/3390988384 >> v1:172.21.15.119:6789/0 conn(0x7f4ee40c9600 legacy=0x7f4ee406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.900+0000 7f4ee37fe640 1 -- 172.21.15.27:0/3390988384 >> v1:172.21.15.27:6789/0 conn(0x7f4ee415a020 legacy=0x7f4ee4071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.900+0000 7f4ee37fe640 1 -- 172.21.15.27:0/3390988384 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee40c8b20 con 0x7f4ee406fce0 2023-08-11T15:53:16.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.901+0000 7f4ee37fe640 1 --2- 172.21.15.27:0/3390988384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 0x7f4ee415e5a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009d90 tx=0x7f4edc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2838d2d2b44f54e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.901+0000 7f4ee1ffb640 1 -- 172.21.15.27:0/3390988384 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc02cbd0 con 0x7f4ee406fce0 2023-08-11T15:53:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.901+0000 7f4ee1ffb640 1 -- 172.21.15.27:0/3390988384 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4edc02cd30 con 0x7f4ee406fce0 2023-08-11T15:53:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.901+0000 7f4ee1ffb640 1 -- 172.21.15.27:0/3390988384 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc02cea0 con 0x7f4ee406fce0 2023-08-11T15:53:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.902+0000 7f4ee3fff640 1 -- 172.21.15.27:0/3390988384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 msgr2=0x7f4ee415e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.902+0000 7f4ee3fff640 1 --2- 172.21.15.27:0/3390988384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 0x7f4ee415e5a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009d90 tx=0x7f4edc02be20 comp rx=0 tx=0).stop 2023-08-11T15:53:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.902+0000 7f4ee3fff640 1 -- 172.21.15.27:0/3390988384 shutdown_connections 2023-08-11T15:53:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.902+0000 7f4ee3fff640 1 --2- 172.21.15.27:0/3390988384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 0x7f4ee415e5a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.902+0000 7f4ee3fff640 1 -- 172.21.15.27:0/3390988384 >> 172.21.15.27:0/3390988384 conn(0x7f4ee40afb70 msgr2=0x7f4ee40aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.902+0000 7f4ee3fff640 1 -- 172.21.15.27:0/3390988384 shutdown_connections 2023-08-11T15:53:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.902+0000 7f4ee3fff640 1 -- 172.21.15.27:0/3390988384 wait complete. 2023-08-11T15:53:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.904+0000 7f4ee3fff640 1 Processor -- start 2023-08-11T15:53:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.904+0000 7f4ee3fff640 1 -- start start 2023-08-11T15:53:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 0x7f4ee4156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ee40c9600 0x7f4ee40c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ee415a020 0x7f4ee40cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee3fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee4157390 con 0x7f4ee406fce0 2023-08-11T15:53:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee3fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee41578a0 con 0x7f4ee415a020 2023-08-11T15:53:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee3fff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40c62d0 con 0x7f4ee40c9600 2023-08-11T15:53:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 0x7f4ee4156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 0x7f4ee4156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40430/0 (socket says 172.21.15.27:40430) 2023-08-11T15:53:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee2ffd640 1 -- 172.21.15.27:0/874866083 learned_addr learned my addr 172.21.15.27:0/874866083 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee2ffd640 1 -- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ee40c9600 msgr2=0x7f4ee40c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee2ffd640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ee40c9600 0x7f4ee40c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee2ffd640 1 -- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ee415a020 msgr2=0x7f4ee40cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee2ffd640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ee415a020 0x7f4ee40cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.905+0000 7f4ee2ffd640 1 -- 172.21.15.27:0/874866083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc009a70 con 0x7f4ee406fce0 2023-08-11T15:53:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.906+0000 7f4ee2ffd640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 0x7f4ee4156e50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c410 tx=0x7f4ed400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.906+0000 7f4ecb7fe640 1 -- 172.21.15.27:0/874866083 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4005ce0 con 0x7f4ee406fce0 2023-08-11T15:53:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.906+0000 7f4ecb7fe640 1 -- 172.21.15.27:0/874866083 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed4005e40 con 0x7f4ee406fce0 2023-08-11T15:53:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.906+0000 7f4ecb7fe640 1 -- 172.21.15.27:0/874866083 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40072f0 con 0x7f4ee406fce0 2023-08-11T15:53:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.906+0000 7f4ee3fff640 1 -- 172.21.15.27:0/874866083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee40c6570 con 0x7f4ee406fce0 2023-08-11T15:53:16.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.906+0000 7f4ee3fff640 1 -- 172.21.15.27:0/874866083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee40c6b30 con 0x7f4ee406fce0 2023-08-11T15:53:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.907+0000 7f4ee3fff640 1 -- 172.21.15.27:0/874866083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee40ab8e0 con 0x7f4ee406fce0 2023-08-11T15:53:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.908+0000 7f4ecb7fe640 1 -- 172.21.15.27:0/874866083 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4ed4007510 con 0x7f4ee406fce0 2023-08-11T15:53:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.909+0000 7f4ecb7fe640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ed006a050 0x7f4ed006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.909+0000 7f4ecb7fe640 1 -- 172.21.15.27:0/874866083 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4ed4087da0 con 0x7f4ee406fce0 2023-08-11T15:53:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.910+0000 7f4ee27fc640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ed006a050 0x7f4ed006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.911+0000 7f4ee27fc640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ed006a050 0x7f4ed006c510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40ad4a0 tx=0x7f4ed80079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:16.913+0000 7f4ecb7fe640 1 -- 172.21.15.27:0/874866083 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f4ee40ab8e0 con 0x7f4ee406fce0 2023-08-11T15:53:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.140+0000 7f4ee3fff640 1 -- 172.21.15.27:0/874866083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4ee40ab8e0 con 0x7f4ee406fce0 2023-08-11T15:53:17.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.140+0000 7f4ecb7fe640 1 -- 172.21.15.27:0/874866083 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4ee40ab8e0 con 0x7f4ee406fce0 2023-08-11T15:53:17.142 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-11T15:53:17.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 -- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ed006a050 msgr2=0x7f4ed006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ed006a050 0x7f4ed006c510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40ad4a0 tx=0x7f4ed80079b0 comp rx=0 tx=0).stop 2023-08-11T15:53:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 -- 172.21.15.27:0/874866083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 msgr2=0x7f4ee4156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 0x7f4ee4156e50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c410 tx=0x7f4ed400c8d0 comp rx=0 tx=0).stop 2023-08-11T15:53:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 -- 172.21.15.27:0/874866083 shutdown_connections 2023-08-11T15:53:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4ed006a050 0x7f4ed006c510 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ee40c9600 0x7f4ee40c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ee415a020 0x7f4ee40cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 --2- 172.21.15.27:0/874866083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee406fce0 0x7f4ee4156e50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 -- 172.21.15.27:0/874866083 >> 172.21.15.27:0/874866083 conn(0x7f4ee40afb70 msgr2=0x7f4ee4072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 -- 172.21.15.27:0/874866083 shutdown_connections 2023-08-11T15:53:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:17.143+0000 7f4ee3fff640 1 -- 172.21.15.27:0/874866083 wait complete. 2023-08-11T15:53:17.164 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-11T15:53:17.164 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T15:53:17.165 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-08-11T15:53:17.165 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-11T15:53:17.165 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-11T15:53:17.165 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-11T15:53:17.166 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T15:53:17.166 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T15:53:17.166 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-11T15:53:17.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:17.167 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-08-11T15:53:17.225 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:53:17.225 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:53:17.225 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:53:17.225 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:17.226 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-08-11T15:53:17.226 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:17.226 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:53:17.226 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:53:17.226 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-08-11T15:53:17.226 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:53:17.226 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:53:17.227 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-08-11T15:53:17.227 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T15:53:17.227 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-08-11T15:53:17.227 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-08-11T15:53:17.227 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-11T15:53:17.227 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41903sec preferred_lft 41903sec 2023-08-11T15:53:17.227 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2023-08-11T15:53:17.227 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:17.228 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:53:17.228 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-08-11T15:53:17.228 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-08-11T15:53:17.228 INFO:teuthology.orchestra.run.smithi027.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-11T15:53:17.228 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 3e:d3:b3:c9:79:4a brd ff:ff:ff:ff:ff:ff 2023-08-11T15:53:17.228 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-11T15:53:17.228 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:17.229 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::b471:35ff:fe61:7a71/64 scope link 2023-08-11T15:53:17.229 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:17.229 INFO:teuthology.orchestra.run.smithi027.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-11T15:53:17.229 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 3e:d3:b3:c9:79:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-11T15:53:17.229 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::3cd3:b3ff:fec9:794a/64 scope link 2023-08-11T15:53:17.229 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:17.229 INFO:teuthology.orchestra.run.smithi027.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-11T15:53:17.230 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether de:54:ad:4f:35:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-11T15:53:17.230 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::dc54:adff:fe4f:35e4/64 scope link 2023-08-11T15:53:17.230 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:17.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:17.231 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T15:53:17.327 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-08-11T15:53:17.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:17.342 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-08-11T15:53:17.404 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-08-11T15:53:17.404 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T15:53:17.404 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-11T15:53:17.404 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-11T15:53:17.405 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T15:53:17.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:17.405 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-08-11T15:53:17.406 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-08-11T15:53:17.406 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-11T15:53:17.406 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-11T15:53:17.407 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-11T15:53:17.407 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-08-11T15:53:17.450 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:53:17.462 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:53:17.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:17.624 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-08-11T15:53:17.729 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-11T15:53:17.730 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-11T15:53:17.730 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-11T15:53:17.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:17.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-08-11T15:53:17.849 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-11T15:53:17.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:53:17.850 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-11T15:53:17.908 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-11T15:53:17.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:53:17.909 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-11T15:53:17.966 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-08-11T15:53:18.076 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:53:18.077 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:53:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:53:18.192 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:53:18.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:18.193 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-08-11T15:53:18.250 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-11T15:53:18.251 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-11T15:53:18.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-08-11T15:53:18.346 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:53:18.347 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:53:18.431 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:53:18.433 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:53:18.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:18.435 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-08-11T15:53:18.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-08-11T15:53:18.463+0000 7f209263e480 -1 init, newargv = 0x5589e23fd5d0 newargc=15 2023-08-11T15:53:18.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[106006]: starting ceph client 2023-08-11T15:53:18.475 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[106006]: starting fuse 2023-08-11T15:53:18.483 INFO:teuthology.orchestra.run.smithi027.stdout:39 2023-08-11T15:53:18.484 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-11T15:53:18.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2023-08-11T15:53:18.485 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-08-11T15:53:18.485 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-08-11T15:53:18.485 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-08-11T15:53:18.485 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-08-11T15:53:18.486 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:53:18.486 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-08-11T15:53:18.486 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-11T15:53:18.487 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-08-11T15:53:18.487 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-11T15:53:18.487 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:53:18.488 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-08-11T15:53:18.488 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-08-11T15:53:18.488 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-08-11T15:53:18.489 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-08-11T15:53:18.489 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:53:18.489 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-08-11T15:53:18.489 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-11T15:53:18.490 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-11T15:53:18.490 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-11T15:53:18.490 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-08-11T15:53:18.491 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-08-11T15:53:18.491 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-08-11T15:53:18.491 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-11T15:53:18.492 DEBUG:teuthology.orchestra.run.smithi027:> 2023-08-11T15:53:18.492 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-08-11T15:53:18.492 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-08-11T15:53:18.711 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.106006.asok 2023-08-11T15:53:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-11T15:53:18.730 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106006.asok 2023-08-11T15:53:18.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:18.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106006.asok status 2023-08-11T15:53:18.994 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:18.995 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-08-11T15:53:18.995 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "92684807f016ce045ac66e37fdde6f8c47857bcc", 2023-08-11T15:53:18.995 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:53:18.995 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-08-11T15:53:18.995 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-08-11T15:53:18.995 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-11T15:53:18.995 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "106006", 2023-08-11T15:53:18.996 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-08-11T15:53:18.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:18.996 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-08-11T15:53:18.996 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-08-11T15:53:18.996 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5847, 2023-08-11T15:53:18.996 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-08-11T15:53:18.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-08-11T15:53:18.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-08-11T15:53:18.997 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 5847 2023-08-11T15:53:18.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:18.997 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-08-11T15:53:18.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-08-11T15:53:18.997 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:53:18.997 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3082730633 2023-08-11T15:53:18.998 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-08-11T15:53:18.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:18.998 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-08-11T15:53:18.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-08-11T15:53:18.998 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:53:18.998 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3082730633 2023-08-11T15:53:18.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-08-11T15:53:18.999 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.5847 192.168.0.1:0/3082730633", 2023-08-11T15:53:18.999 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3082730633", 2023-08-11T15:53:18.999 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-08-11T15:53:18.999 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 18, 2023-08-11T15:53:18.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 43, 2023-08-11T15:53:18.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-08-11T15:53:18.999 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-08-11T15:53:18.999 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-08-11T15:53:19.000 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:19.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:19.011 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-08-11T15:53:19.011 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-11T15:53:19.070 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-08-11T15:53:19.071 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-11T15:53:19.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:19.072 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-11T15:53:19.180 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-11T15:53:19.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.418+0000 7f75c2871640 1 Processor -- start 2023-08-11T15:53:19.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.418+0000 7f75c2871640 1 -- start start 2023-08-11T15:53:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.420+0000 7f75c2871640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75bc15a140 0x7f75bc06b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.420+0000 7f75c2871640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc06f5d0 0x7f75bc06fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.420+0000 7f75c2871640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75bc0c8a30 con 0x7f75bc0cb430 2023-08-11T15:53:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.420+0000 7f75c2871640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75bc06ffa0 con 0x7f75bc15a140 2023-08-11T15:53:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.420+0000 7f75c2871640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0b1580 con 0x7f75bc06f5d0 2023-08-11T15:53:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.420+0000 7f75bbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75bc15a140 0x7f75bc06b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:19.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.420+0000 7f75bbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75bc15a140 0x7f75bc06b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54100/0 (socket says 172.21.15.27:54100) 2023-08-11T15:53:19.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.420+0000 7f75bbfff640 1 -- 172.21.15.27:0/2759454224 learned_addr learned my addr 172.21.15.27:0/2759454224 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:19.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.420+0000 7f75bb7fe640 1 --1- 172.21.15.27:0/2759454224 >> v1:172.21.15.27:6789/0 conn(0x7f75bc0cb430 0x7f75bc0b0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35182/0 (socket says 172.21.15.27:35182) 2023-08-11T15:53:19.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.421+0000 7f75c0e0e640 1 --2- 172.21.15.27:0/2759454224 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc06f5d0 0x7f75bc06fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.421+0000 7f75bbfff640 1 -- 172.21.15.27:0/2759454224 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc06f5d0 msgr2=0x7f75bc06fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.421+0000 7f75bbfff640 1 --2- 172.21.15.27:0/2759454224 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc06f5d0 0x7f75bc06fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.421+0000 7f75bbfff640 1 -- 172.21.15.27:0/2759454224 >> v1:172.21.15.27:6789/0 conn(0x7f75bc0cb430 legacy=0x7f75bc0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.421+0000 7f75bbfff640 1 -- 172.21.15.27:0/2759454224 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc06c0c0 con 0x7f75bc15a140 2023-08-11T15:53:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.421+0000 7f75baffd640 1 -- 172.21.15.27:0/2759454224 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329370480 0 0) 0x7f75bc0c8a30 con 0x7f75bc0cb430 2023-08-11T15:53:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.422+0000 7f75bbfff640 1 --2- 172.21.15.27:0/2759454224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc15a140 0x7f75bc06b870 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009880 tx=0x7f75ac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8142ed339cbc3b85 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.422+0000 7f75baffd640 1 -- 172.21.15.27:0/2759454224 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac004230 con 0x7f75bc15a140 2023-08-11T15:53:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.422+0000 7f75baffd640 1 -- 172.21.15.27:0/2759454224 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ac004390 con 0x7f75bc15a140 2023-08-11T15:53:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.422+0000 7f75baffd640 1 -- 172.21.15.27:0/2759454224 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac003810 con 0x7f75bc15a140 2023-08-11T15:53:19.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.423+0000 7f75c2871640 1 -- 172.21.15.27:0/2759454224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc15a140 msgr2=0x7f75bc06b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:19.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.423+0000 7f75c2871640 1 --2- 172.21.15.27:0/2759454224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc15a140 0x7f75bc06b870 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009880 tx=0x7f75ac02b960 comp rx=0 tx=0).stop 2023-08-11T15:53:19.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.423+0000 7f75c2871640 1 -- 172.21.15.27:0/2759454224 shutdown_connections 2023-08-11T15:53:19.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.423+0000 7f75c2871640 1 --2- 172.21.15.27:0/2759454224 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc06f5d0 0x7f75bc06fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:19.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.423+0000 7f75c2871640 1 --2- 172.21.15.27:0/2759454224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc15a140 0x7f75bc06b870 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:19.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.423+0000 7f75c2871640 1 -- 172.21.15.27:0/2759454224 >> 172.21.15.27:0/2759454224 conn(0x7f75bc0afb70 msgr2=0x7f75bc0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:19.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.423+0000 7f75c2871640 1 -- 172.21.15.27:0/2759454224 shutdown_connections 2023-08-11T15:53:19.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.423+0000 7f75c2871640 1 -- 172.21.15.27:0/2759454224 wait complete. 2023-08-11T15:53:19.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.425+0000 7f75c2871640 1 Processor -- start 2023-08-11T15:53:19.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.425+0000 7f75c2871640 1 -- start start 2023-08-11T15:53:19.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.426+0000 7f75c2871640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc06f5d0 0x7f75bc1480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:19.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.426+0000 7f75c2871640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc0cb430 0x7f75bc1444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:19.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.426+0000 7f75c2871640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc15a140 0x7f75bc15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:19.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.426+0000 7f75c2871640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc1457d0 con 0x7f75bc0cb430 2023-08-11T15:53:19.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.426+0000 7f75c2871640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc144bc0 con 0x7f75bc06f5d0 2023-08-11T15:53:19.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.426+0000 7f75c2871640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc144ea0 con 0x7f75bc15a140 2023-08-11T15:53:19.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.426+0000 7f75bb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc0cb430 0x7f75bc1444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:19.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75bb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc0cb430 0x7f75bc1444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54112/0 (socket says 172.21.15.27:54112) 2023-08-11T15:53:19.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75bb7fe640 1 -- 172.21.15.27:0/3594641873 learned_addr learned my addr 172.21.15.27:0/3594641873 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:19.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75c0e0e640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc15a140 0x7f75bc15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:19.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75bb7fe640 1 -- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc15a140 msgr2=0x7f75bc15a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:19.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75bb7fe640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc15a140 0x7f75bc15a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:19.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75bb7fe640 1 -- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc06f5d0 msgr2=0x7f75bc1480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:19.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75bbfff640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc06f5d0 0x7f75bc1480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:19.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75bb7fe640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc06f5d0 0x7f75bc1480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:19.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75bb7fe640 1 -- 172.21.15.27:0/3594641873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac009460 con 0x7f75bc0cb430 2023-08-11T15:53:19.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75c0e0e640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc15a140 0x7f75bc15a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:19.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.427+0000 7f75bb7fe640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc0cb430 0x7f75bc1444a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c4d0 tx=0x7f75b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:19.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.428+0000 7f75b8ff9640 1 -- 172.21.15.27:0/3594641873 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00074e0 con 0x7f75bc0cb430 2023-08-11T15:53:19.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.428+0000 7f75b8ff9640 1 -- 172.21.15.27:0/3594641873 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b0007660 con 0x7f75bc0cb430 2023-08-11T15:53:19.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.428+0000 7f75b8ff9640 1 -- 172.21.15.27:0/3594641873 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0004a20 con 0x7f75bc0cb430 2023-08-11T15:53:19.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.428+0000 7f75c2871640 1 -- 172.21.15.27:0/3594641873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75bc145180 con 0x7f75bc0cb430 2023-08-11T15:53:19.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.428+0000 7f75c2871640 1 -- 172.21.15.27:0/3594641873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75bc15aa60 con 0x7f75bc0cb430 2023-08-11T15:53:19.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.429+0000 7f75c2871640 1 -- 172.21.15.27:0/3594641873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75bc145940 con 0x7f75bc0cb430 2023-08-11T15:53:19.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.430+0000 7f75b8ff9640 1 -- 172.21.15.27:0/3594641873 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f75b0004b80 con 0x7f75bc0cb430 2023-08-11T15:53:19.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.431+0000 7f75b8ff9640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75a806a050 0x7f75a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:19.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.432+0000 7f75b8ff9640 1 -- 172.21.15.27:0/3594641873 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f75b0087980 con 0x7f75bc0cb430 2023-08-11T15:53:19.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.432+0000 7f75bbfff640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75a806a050 0x7f75a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:19.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.433+0000 7f75bbfff640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75a806a050 0x7f75a806c510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f75ac035000 tx=0x7f75ac008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.436+0000 7f75b8ff9640 1 -- 172.21.15.27:0/3594641873 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f75b0052e90 con 0x7f75bc0cb430 2023-08-11T15:53:19.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.674+0000 7f75c2871640 1 -- 172.21.15.27:0/3594641873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f75bc073990 con 0x7f75bc0cb430 2023-08-11T15:53:19.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.675+0000 7f75b8ff9640 1 -- 172.21.15.27:0/3594641873 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f75b00564b0 con 0x7f75bc0cb430 2023-08-11T15:53:19.677 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-11T15:53:19.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 -- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75a806a050 msgr2=0x7f75a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:19.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75a806a050 0x7f75a806c510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f75ac035000 tx=0x7f75ac008030 comp rx=0 tx=0).stop 2023-08-11T15:53:19.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 -- 172.21.15.27:0/3594641873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc0cb430 msgr2=0x7f75bc1444a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:19.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc0cb430 0x7f75bc1444a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c4d0 tx=0x7f75b000c990 comp rx=0 tx=0).stop 2023-08-11T15:53:19.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 -- 172.21.15.27:0/3594641873 shutdown_connections 2023-08-11T15:53:19.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f75a806a050 0x7f75a806c510 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:19.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f75bc15a140 0x7f75bc15a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:19.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75bc0cb430 0x7f75bc1444a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:19.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 --2- 172.21.15.27:0/3594641873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75bc06f5d0 0x7f75bc1480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:19.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 -- 172.21.15.27:0/3594641873 >> 172.21.15.27:0/3594641873 conn(0x7f75bc0afb70 msgr2=0x7f75bc0b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:19.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 -- 172.21.15.27:0/3594641873 shutdown_connections 2023-08-11T15:53:19.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:19.678+0000 7f75c2871640 1 -- 172.21.15.27:0/3594641873 wait complete. 2023-08-11T15:53:19.700 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-11T15:53:19.700 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T15:53:19.701 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-08-11T15:53:19.701 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-11T15:53:19.701 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-11T15:53:19.701 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-11T15:53:19.702 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T15:53:19.702 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T15:53:19.702 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-11T15:53:19.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:19.703 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-08-11T15:53:19.764 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:53:19.764 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:53:19.764 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:53:19.765 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:19.765 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-08-11T15:53:19.765 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:19.765 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:53:19.765 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:53:19.765 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-08-11T15:53:19.765 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:53:19.766 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:53:19.766 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-08-11T15:53:19.766 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:53:19.766 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-08-11T15:53:19.766 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T15:53:19.767 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-08-11T15:53:19.767 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-11T15:53:19.767 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41874sec preferred_lft 41874sec 2023-08-11T15:53:19.767 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link noprefixroute 2023-08-11T15:53:19.767 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:19.767 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-11T15:53:19.767 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 6a:1a:57:8c:46:a1 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:53:19.768 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-11T15:53:19.768 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:19.768 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c0f5:5eff:fe99:60b/64 scope link 2023-08-11T15:53:19.768 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:19.768 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-11T15:53:19.768 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 6a:1a:57:8c:46:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-11T15:53:19.768 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::681a:57ff:fe8c:46a1/64 scope link 2023-08-11T15:53:19.768 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:19.769 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-11T15:53:19.769 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether ee:7b:d3:49:4d:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-11T15:53:19.769 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec7b:d3ff:fe49:4d9b/64 scope link 2023-08-11T15:53:19.769 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:53:19.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:19.771 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T15:53:19.868 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-08-11T15:53:19.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:19.873 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-08-11T15:53:19.926 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-08-11T15:53:19.926 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T15:53:19.926 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-11T15:53:19.926 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-11T15:53:19.927 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T15:53:19.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:19.927 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-08-11T15:53:19.928 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-08-11T15:53:19.928 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-11T15:53:19.928 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-11T15:53:19.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-11T15:53:19.929 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-08-11T15:53:20.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:20.117 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-08-11T15:53:20.166 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-11T15:53:20.167 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-11T15:53:20.167 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-11T15:53:20.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:20.168 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-08-11T15:53:20.246 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-11T15:53:20.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:53:20.246 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-11T15:53:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-11T15:53:20.291 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:53:20.292 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-11T15:53:20.387 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-08-11T15:53:20.481 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:53:20.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:53:20.576 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:53:20.579 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:53:20.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:20.580 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-08-11T15:53:20.635 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-11T15:53:20.635 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-11T15:53:20.640 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-08-11T15:53:20.720 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:53:20.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:53:20.801 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:53:20.804 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:53:20.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:20.805 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-08-11T15:53:20.831 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:2023-08-11T15:53:20.830+0000 7fdff5e39480 -1 init, newargv = 0x55e260e5e3b0 newargc=15 2023-08-11T15:53:20.832 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:ceph-fuse[96551]: starting ceph client 2023-08-11T15:53:20.843 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:ceph-fuse[96551]: starting fuse 2023-08-11T15:53:20.851 INFO:teuthology.orchestra.run.smithi119.stdout:39 2023-08-11T15:53:20.851 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-11T15:53:20.852 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer -- python3 -c ' 2023-08-11T15:53:20.852 DEBUG:teuthology.orchestra.run.smithi119:> import glob 2023-08-11T15:53:20.852 DEBUG:teuthology.orchestra.run.smithi119:> import re 2023-08-11T15:53:20.853 DEBUG:teuthology.orchestra.run.smithi119:> import os 2023-08-11T15:53:20.853 DEBUG:teuthology.orchestra.run.smithi119:> import subprocess 2023-08-11T15:53:20.853 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:53:20.854 DEBUG:teuthology.orchestra.run.smithi119:> def _find_admin_socket(client_name): 2023-08-11T15:53:20.854 DEBUG:teuthology.orchestra.run.smithi119:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-11T15:53:20.854 DEBUG:teuthology.orchestra.run.smithi119:> files = glob.glob(asok_path) 2023-08-11T15:53:20.854 DEBUG:teuthology.orchestra.run.smithi119:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-11T15:53:20.855 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:53:20.855 DEBUG:teuthology.orchestra.run.smithi119:> # Given a non-glob path, it better be there 2023-08-11T15:53:20.855 DEBUG:teuthology.orchestra.run.smithi119:> if "*" not in asok_path: 2023-08-11T15:53:20.856 DEBUG:teuthology.orchestra.run.smithi119:> assert(len(files) == 1) 2023-08-11T15:53:20.856 DEBUG:teuthology.orchestra.run.smithi119:> return files[0] 2023-08-11T15:53:20.856 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:53:20.857 DEBUG:teuthology.orchestra.run.smithi119:> for f in files: 2023-08-11T15:53:20.857 DEBUG:teuthology.orchestra.run.smithi119:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-11T15:53:20.857 DEBUG:teuthology.orchestra.run.smithi119:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-11T15:53:20.857 DEBUG:teuthology.orchestra.run.smithi119:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-11T15:53:20.858 DEBUG:teuthology.orchestra.run.smithi119:> contents = proc_f.read() 2023-08-11T15:53:20.858 DEBUG:teuthology.orchestra.run.smithi119:> if mountpoint in contents: 2023-08-11T15:53:20.858 DEBUG:teuthology.orchestra.run.smithi119:> return f 2023-08-11T15:53:20.859 DEBUG:teuthology.orchestra.run.smithi119:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-11T15:53:20.859 DEBUG:teuthology.orchestra.run.smithi119:> 2023-08-11T15:53:20.859 DEBUG:teuthology.orchestra.run.smithi119:> print(_find_admin_socket("client.1")) 2023-08-11T15:53:20.859 DEBUG:teuthology.orchestra.run.smithi119:> ' 2023-08-11T15:53:21.059 INFO:teuthology.orchestra.run.smithi119.stdout:/var/run/ceph/ceph-client.1.96551.asok 2023-08-11T15:53:21.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-08-11T15:53:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-11T15:53:21.076 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.96551.asok 2023-08-11T15:53:21.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:21.077 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.96551.asok status 2023-08-11T15:53:21.319 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-08-11T15:53:21.319 INFO:teuthology.orchestra.run.smithi119.stdout: "metadata": { 2023-08-11T15:53:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_sha1": "92684807f016ce045ac66e37fdde6f8c47857bcc", 2023-08-11T15:53:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:53:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "entity_id": "1", 2023-08-11T15:53:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": "smithi119", 2023-08-11T15:53:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-11T15:53:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "pid": "96551", 2023-08-11T15:53:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "root": "/" 2023-08-11T15:53:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:53:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_count": 0, 2023-08-11T15:53:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_pinned_count": 0, 2023-08-11T15:53:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 5296, 2023-08-11T15:53:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "inst": { 2023-08-11T15:53:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "name": { 2023-08-11T15:53:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "client", 2023-08-11T15:53:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "num": 5296 2023-08-11T15:53:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:53:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-08-11T15:53:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-08-11T15:53:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:53:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 2738574416 2023-08-11T15:53:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-08-11T15:53:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:53:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-08-11T15:53:21.323 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-08-11T15:53:21.323 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:53:21.323 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 2738574416 2023-08-11T15:53:21.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-08-11T15:53:21.323 INFO:teuthology.orchestra.run.smithi119.stdout: "inst_str": "client.5296 192.168.0.1:0/2738574416", 2023-08-11T15:53:21.323 INFO:teuthology.orchestra.run.smithi119.stdout: "addr_str": "192.168.0.1:0/2738574416", 2023-08-11T15:53:21.323 INFO:teuthology.orchestra.run.smithi119.stdout: "inode_count": 1, 2023-08-11T15:53:21.324 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_epoch": 18, 2023-08-11T15:53:21.324 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch": 43, 2023-08-11T15:53:21.324 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch_barrier": 0, 2023-08-11T15:53:21.324 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklisted": false, 2023-08-11T15:53:21.324 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs" 2023-08-11T15:53:21.324 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-08-11T15:53:21.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:21.335 DEBUG:teuthology.orchestra.run.smithi119:> stat --file-system '--printf=%T 2023-08-11T15:53:21.335 DEBUG:teuthology.orchestra.run.smithi119:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-11T15:53:21.391 INFO:teuthology.orchestra.run.smithi119.stdout:fuseblk 2023-08-11T15:53:21.391 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-11T15:53:21.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:21.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-11T15:53:21.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:21.488 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-08-11T15:53:21.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:21.568 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 0 ./subdir) 2023-08-11T15:53:21.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:21.632 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes ./subdir) 2023-08-11T15:53:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:./subdir: ceph.quota.max_bytes: No such attribute 2023-08-11T15:53:21.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:53:21.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:21.693 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir) 2023-08-11T15:53:21.747 INFO:teuthology.orchestra.run.smithi027.stderr:setfattr: ./subdir: Invalid argument 2023-08-11T15:53:21.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:53:21.748 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-11T15:53:21.748 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-11T15:53:21.872 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-11T15:53:21.998 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-11T15:53:21.999 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:53:22.000 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:53:22.066 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:53:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.306+0000 7f90aaba4640 1 Processor -- start 2023-08-11T15:53:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.306+0000 7f90aaba4640 1 -- start start 2023-08-11T15:53:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.308+0000 7f90aaba4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90a415c520 0x7f90a40caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.308+0000 7f90aaba4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f90a40ad2a0 0x7f90a40cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.308+0000 7f90aaba4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a40cc4e0 con 0x7f90a40cc0b0 2023-08-11T15:53:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.308+0000 7f90aaba4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a406b910 con 0x7f90a40ad2a0 2023-08-11T15:53:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.308+0000 7f90aaba4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a406bbf0 con 0x7f90a415c520 2023-08-11T15:53:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.309+0000 7f90a93a1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f90a40ad2a0 0x7f90a40cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.309+0000 7f90a93a1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f90a40ad2a0 0x7f90a40cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54122/0 (socket says 172.21.15.27:54122) 2023-08-11T15:53:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.309+0000 7f90a93a1640 1 -- 172.21.15.27:0/3849648213 learned_addr learned my addr 172.21.15.27:0/3849648213 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.309+0000 7f90a9ba2640 1 --2- 172.21.15.27:0/3849648213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90a415c520 0x7f90a40caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.309+0000 7f90a93a1640 1 -- 172.21.15.27:0/3849648213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90a415c520 msgr2=0x7f90a40caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.309+0000 7f90a93a1640 1 --2- 172.21.15.27:0/3849648213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90a415c520 0x7f90a40caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.309+0000 7f90a93a1640 1 -- 172.21.15.27:0/3849648213 >> v1:172.21.15.27:6789/0 conn(0x7f90a40cc0b0 legacy=0x7f90a40abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.309+0000 7f90a93a1640 1 -- 172.21.15.27:0/3849648213 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a406bed0 con 0x7f90a40ad2a0 2023-08-11T15:53:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.309+0000 7f90a8ba0640 1 -- 172.21.15.27:0/3849648213 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000574957 0 0) 0x7f90a40cc4e0 con 0x7f90a40cc0b0 2023-08-11T15:53:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.311+0000 7f90a93a1640 1 --2- 172.21.15.27:0/3849648213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40ad2a0 0x7f90a40cbb70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9094009870 tx=0x7f909402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6d2384e4bc23f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.311+0000 7f90a8ba0640 1 -- 172.21.15.27:0/3849648213 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094031640 con 0x7f90a40ad2a0 2023-08-11T15:53:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.311+0000 7f90a8ba0640 1 -- 172.21.15.27:0/3849648213 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90940317a0 con 0x7f90a40ad2a0 2023-08-11T15:53:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.311+0000 7f90a8ba0640 1 -- 172.21.15.27:0/3849648213 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094031b30 con 0x7f90a40ad2a0 2023-08-11T15:53:22.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.311+0000 7f90aaba4640 1 -- 172.21.15.27:0/3849648213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40ad2a0 msgr2=0x7f90a40cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.312+0000 7f90aaba4640 1 --2- 172.21.15.27:0/3849648213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40ad2a0 0x7f90a40cbb70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9094009870 tx=0x7f909402b8f0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.312+0000 7f90aaba4640 1 -- 172.21.15.27:0/3849648213 shutdown_connections 2023-08-11T15:53:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.312+0000 7f90aaba4640 1 --2- 172.21.15.27:0/3849648213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40ad2a0 0x7f90a40cbb70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.312+0000 7f90aaba4640 1 --2- 172.21.15.27:0/3849648213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90a415c520 0x7f90a40caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.312+0000 7f90aaba4640 1 -- 172.21.15.27:0/3849648213 >> 172.21.15.27:0/3849648213 conn(0x7f90a40afb70 msgr2=0x7f90a40aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:22.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.312+0000 7f90aaba4640 1 -- 172.21.15.27:0/3849648213 shutdown_connections 2023-08-11T15:53:22.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.312+0000 7f90aaba4640 1 -- 172.21.15.27:0/3849648213 wait complete. 2023-08-11T15:53:22.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.314+0000 7f90aaba4640 1 Processor -- start 2023-08-11T15:53:22.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.314+0000 7f90aaba4640 1 -- start start 2023-08-11T15:53:22.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90aaba4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90a40ad2a0 0x7f90a40c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90aaba4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40cc0b0 0x7f90a40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90aaba4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90a415c520 0x7f90a40c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90aaba4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40c6380 con 0x7f90a40cc0b0 2023-08-11T15:53:22.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90aaba4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40c6660 con 0x7f90a415c520 2023-08-11T15:53:22.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90aaba4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40c2c40 con 0x7f90a40ad2a0 2023-08-11T15:53:22.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90a93a1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40cc0b0 0x7f90a40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90a93a1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40cc0b0 0x7f90a40c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54124/0 (socket says 172.21.15.27:54124) 2023-08-11T15:53:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90a93a1640 1 -- 172.21.15.27:0/3129303344 learned_addr learned my addr 172.21.15.27:0/3129303344 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90a9ba2640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90a40ad2a0 0x7f90a40c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90a93a1640 1 -- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90a40ad2a0 msgr2=0x7f90a40c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90a93a1640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90a40ad2a0 0x7f90a40c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90a93a1640 1 -- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90a415c520 msgr2=0x7f90a40c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90a93a1640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90a415c520 0x7f90a40c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.315+0000 7f90a93a1640 1 -- 172.21.15.27:0/3129303344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940094d0 con 0x7f90a40cc0b0 2023-08-11T15:53:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.316+0000 7f90a93a1640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40cc0b0 0x7f90a40c6dd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9094000c00 tx=0x7f90940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.316+0000 7f909a7fc640 1 -- 172.21.15.27:0/3129303344 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909403ac60 con 0x7f90a40cc0b0 2023-08-11T15:53:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.316+0000 7f909a7fc640 1 -- 172.21.15.27:0/3129303344 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909403adc0 con 0x7f90a40cc0b0 2023-08-11T15:53:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.316+0000 7f909a7fc640 1 -- 172.21.15.27:0/3129303344 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094044630 con 0x7f90a40cc0b0 2023-08-11T15:53:22.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.316+0000 7f90aaba4640 1 -- 172.21.15.27:0/3129303344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a40c2f20 con 0x7f90a40cc0b0 2023-08-11T15:53:22.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.316+0000 7f90aaba4640 1 -- 172.21.15.27:0/3129303344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a40c7310 con 0x7f90a40cc0b0 2023-08-11T15:53:22.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.318+0000 7f90aaba4640 1 -- 172.21.15.27:0/3129303344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a404e420 con 0x7f90a40cc0b0 2023-08-11T15:53:22.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.319+0000 7f909a7fc640 1 -- 172.21.15.27:0/3129303344 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f909402f050 con 0x7f90a40cc0b0 2023-08-11T15:53:22.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.320+0000 7f909a7fc640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f908806a050 0x7f908806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.320+0000 7f909a7fc640 1 -- 172.21.15.27:0/3129303344 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f90940aa5e0 con 0x7f90a40cc0b0 2023-08-11T15:53:22.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.320+0000 7f90a9ba2640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f908806a050 0x7f908806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.322+0000 7f90a9ba2640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f908806a050 0x7f908806c510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f90a0006fd0 tx=0x7f90a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:22.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.325+0000 7f909a7fc640 1 -- 172.21.15.27:0/3129303344 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9094075af0 con 0x7f90a40cc0b0 2023-08-11T15:53:22.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.520+0000 7f90aaba4640 1 -- 172.21.15.27:0/3129303344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90a40ceb00 con 0x7f90a40cc0b0 2023-08-11T15:53:22.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.524+0000 7f909a7fc640 1 -- 172.21.15.27:0/3129303344 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18500 (secure 0 0 0) 0x7f9094031e60 con 0x7f90a40cc0b0 2023-08-11T15:53:22.526 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:22.526 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":43,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:53:10.738370+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:53:06.898951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-11T15:53:08.258996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:53:22.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 -- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f908806a050 msgr2=0x7f908806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f908806a050 0x7f908806c510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f90a0006fd0 tx=0x7f90a0008040 comp rx=0 tx=0).stop 2023-08-11T15:53:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 -- 172.21.15.27:0/3129303344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40cc0b0 msgr2=0x7f90a40c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40cc0b0 0x7f90a40c6dd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9094000c00 tx=0x7f90940046c0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 -- 172.21.15.27:0/3129303344 shutdown_connections 2023-08-11T15:53:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f908806a050 0x7f908806c510 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90a415c520 0x7f90a40c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90a40cc0b0 0x7f90a40c6dd0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 --2- 172.21.15.27:0/3129303344 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f90a40ad2a0 0x7f90a40c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 -- 172.21.15.27:0/3129303344 >> 172.21.15.27:0/3129303344 conn(0x7f90a40afb70 msgr2=0x7f90a40af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:22.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 -- 172.21.15.27:0/3129303344 shutdown_connections 2023-08-11T15:53:22.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.527+0000 7f90aaba4640 1 -- 172.21.15.27:0/3129303344 wait complete. 2023-08-11T15:53:22.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:53:06.898951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:53:08.258996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T15:53:22.548 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:53:22.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.792+0000 7f8ae9d39640 1 Processor -- start 2023-08-11T15:53:22.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.792+0000 7f8ae9d39640 1 -- start start 2023-08-11T15:53:22.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.794+0000 7f8ae9d39640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8ae415ceb0 0x7f8ae40caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.794+0000 7f8ae9d39640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae40cd040 con 0x7f8ae40cb530 2023-08-11T15:53:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.794+0000 7f8ae9d39640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae40cb980 con 0x7f8ae40ca080 2023-08-11T15:53:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.794+0000 7f8ae9d39640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae4070600 con 0x7f8ae415ceb0 2023-08-11T15:53:22.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.794+0000 7f8ae3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8ae40cb530 0x7f8ae4070ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35200/0 (socket says 172.21.15.27:35200) 2023-08-11T15:53:22.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.794+0000 7f8ae3fff640 1 -- 172.21.15.27:0/1288347750 learned_addr learned my addr 172.21.15.27:0/1288347750 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:22.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335467994 0 0) 0x7f8ae40cd040 con 0x7f8ae40cb530 2023-08-11T15:53:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad0003520 con 0x7f8ae40cb530 2023-08-11T15:53:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8adbfff640 1 --2- 172.21.15.27:0/1288347750 >> v2:172.21.15.119:3300/0 conn(0x7f8ae415ceb0 0x7f8ae40caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445951470 0 0) 0x7f8ad0003520 con 0x7f8ae40cb530 2023-08-11T15:53:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 >> v2:172.21.15.119:3300/0 conn(0x7f8ae415ceb0 msgr2=0x7f8ae40caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 --2- 172.21.15.27:0/1288347750 >> v2:172.21.15.119:3300/0 conn(0x7f8ae415ceb0 0x7f8ae40caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 >> v1:172.21.15.119:6789/0 conn(0x7f8ae40ca080 legacy=0x7f8ae415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae415d370 con 0x7f8ae40cb530 2023-08-11T15:53:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f8adc0019f0 con 0x7f8ae40cb530 2023-08-11T15:53:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 >> v1:172.21.15.27:6789/0 conn(0x7f8ae40cb530 legacy=0x7f8ae4070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ad0003970 0x7f8ad0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ad0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ad00052a0 0x7f8ad000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4070600 con 0x7f8ad00047c0 2023-08-11T15:53:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad000e380 con 0x7f8ad0003970 2023-08-11T15:53:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.795+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad000e540 con 0x7f8ad00052a0 2023-08-11T15:53:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.796+0000 7f8adbfff640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ad0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.796+0000 7f8ae37fe640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ad0003970 0x7f8ad0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.796+0000 7f8adbfff640 1 -- 172.21.15.27:0/1288347750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ad00052a0 msgr2=0x7f8ad000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:22.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.796+0000 7f8ae3fff640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ad00052a0 0x7f8ad000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.796+0000 7f8adbfff640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ad00052a0 0x7f8ad000db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.796+0000 7f8adbfff640 1 -- 172.21.15.27:0/1288347750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ad0003970 msgr2=0x7f8ad0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:22.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.796+0000 7f8adbfff640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ad0003970 0x7f8ad0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.796+0000 7f8adbfff640 1 -- 172.21.15.27:0/1288347750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad0003520 con 0x7f8ad00047c0 2023-08-11T15:53:22.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.796+0000 7f8ae3fff640 1 -- 172.21.15.27:0/1288347750 reap_dead start 2023-08-11T15:53:22.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.796+0000 7f8ae37fe640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ad0003970 0x7f8ad0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:22.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.798+0000 7f8adbfff640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ad0004c40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00ba00 tx=0x7f8ad00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed37c26d9aa0019d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:22.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.798+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc032620 con 0x7f8ad00047c0 2023-08-11T15:53:22.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.798+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc032780 con 0x7f8ad00047c0 2023-08-11T15:53:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.798+0000 7f8ae2ffd640 1 -- 172.21.15.27:0/1288347750 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc032b10 con 0x7f8ad00047c0 2023-08-11T15:53:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.798+0000 7f8ae9d39640 1 -- 172.21.15.27:0/1288347750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 msgr2=0x7f8ad0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.798+0000 7f8ae9d39640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ad0004c40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00ba00 tx=0x7f8ad00057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.799+0000 7f8ae9d39640 1 -- 172.21.15.27:0/1288347750 shutdown_connections 2023-08-11T15:53:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.799+0000 7f8ae9d39640 1 --2- 172.21.15.27:0/1288347750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ad0004c40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.799+0000 7f8ae9d39640 1 -- 172.21.15.27:0/1288347750 >> 172.21.15.27:0/1288347750 conn(0x7f8ae4069bf0 msgr2=0x7f8ae406a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.799+0000 7f8ae9d39640 1 -- 172.21.15.27:0/1288347750 shutdown_connections 2023-08-11T15:53:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.799+0000 7f8ae9d39640 1 -- 172.21.15.27:0/1288347750 wait complete. 2023-08-11T15:53:22.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.801+0000 7f8ae9d39640 1 Processor -- start 2023-08-11T15:53:22.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.801+0000 7f8ae9d39640 1 -- start start 2023-08-11T15:53:22.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.802+0000 7f8ae9d39640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ae4150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.802+0000 7f8ae9d39640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae40c1bf0 0x7f8ae4150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.802+0000 7f8ae9d39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae40c2070 0x7f8ae41512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.802+0000 7f8ae9d39640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40c33b0 con 0x7f8ad00047c0 2023-08-11T15:53:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.802+0000 7f8ae9d39640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40c2560 con 0x7f8ae40c2070 2023-08-11T15:53:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.802+0000 7f8ae9d39640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40c2840 con 0x7f8ae40c1bf0 2023-08-11T15:53:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.802+0000 7f8ae37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ae4150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.802+0000 7f8ae37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ae4150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54142/0 (socket says 172.21.15.27:54142) 2023-08-11T15:53:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.802+0000 7f8ae37fe640 1 -- 172.21.15.27:0/864366692 learned_addr learned my addr 172.21.15.27:0/864366692 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae37fe640 1 -- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae40c1bf0 msgr2=0x7f8ae4150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae37fe640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae40c1bf0 0x7f8ae4150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae37fe640 1 -- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae40c2070 msgr2=0x7f8ae41512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae37fe640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae40c2070 0x7f8ae41512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae37fe640 1 -- 172.21.15.27:0/864366692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae40c2b20 con 0x7f8ad00047c0 2023-08-11T15:53:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae37fe640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ae4150740 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400e480 tx=0x7f8ad400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae0ff9640 1 -- 172.21.15.27:0/864366692 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40052e0 con 0x7f8ad00047c0 2023-08-11T15:53:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae0ff9640 1 -- 172.21.15.27:0/864366692 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad4005440 con 0x7f8ad00047c0 2023-08-11T15:53:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae0ff9640 1 -- 172.21.15.27:0/864366692 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4003810 con 0x7f8ad00047c0 2023-08-11T15:53:22.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae9d39640 1 -- 172.21.15.27:0/864366692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae40c2e00 con 0x7f8ad00047c0 2023-08-11T15:53:22.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.803+0000 7f8ae9d39640 1 -- 172.21.15.27:0/864366692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae415b940 con 0x7f8ad00047c0 2023-08-11T15:53:22.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.805+0000 7f8ae9d39640 1 -- 172.21.15.27:0/864366692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae415bb60 con 0x7f8ad00047c0 2023-08-11T15:53:22.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.806+0000 7f8ae0ff9640 1 -- 172.21.15.27:0/864366692 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8ad4003970 con 0x7f8ad00047c0 2023-08-11T15:53:22.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.807+0000 7f8ae0ff9640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8ac406a080 0x7f8ac406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:22.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.807+0000 7f8ae0ff9640 1 -- 172.21.15.27:0/864366692 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8ad4088300 con 0x7f8ad00047c0 2023-08-11T15:53:22.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.807+0000 7f8adbfff640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8ac406a080 0x7f8ac406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:22.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.809+0000 7f8adbfff640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8ac406a080 0x7f8ac406c540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0004740 tx=0x7f8acc032000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:22.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:22.811+0000 7f8ae0ff9640 1 -- 172.21.15.27:0/864366692 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f8ad40538a0 con 0x7f8ad00047c0 2023-08-11T15:53:23.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.009+0000 7f8ae9d39640 1 -- 172.21.15.27:0/864366692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ae40cb980 con 0x7f8ad00047c0 2023-08-11T15:53:23.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.010+0000 7f8ae0ff9640 1 -- 172.21.15.27:0/864366692 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ae40cb980 con 0x7f8ad00047c0 2023-08-11T15:53:23.012 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:53:23.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 -- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8ac406a080 msgr2=0x7f8ac406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8ac406a080 0x7f8ac406c540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0004740 tx=0x7f8acc032000 comp rx=0 tx=0).stop 2023-08-11T15:53:23.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 -- 172.21.15.27:0/864366692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 msgr2=0x7f8ae4150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ae4150740 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400e480 tx=0x7f8ad400e940 comp rx=0 tx=0).stop 2023-08-11T15:53:23.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 -- 172.21.15.27:0/864366692 shutdown_connections 2023-08-11T15:53:23.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8ac406a080 0x7f8ac406c540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae40c1bf0 0x7f8ae4150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae40c2070 0x7f8ae41512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 --2- 172.21.15.27:0/864366692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ad00047c0 0x7f8ae4150740 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 -- 172.21.15.27:0/864366692 >> 172.21.15.27:0/864366692 conn(0x7f8ae4069bf0 msgr2=0x7f8ae40c9450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:23.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 -- 172.21.15.27:0/864366692 shutdown_connections 2023-08-11T15:53:23.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.013+0000 7f8ae9d39640 1 -- 172.21.15.27:0/864366692 wait complete. 2023-08-11T15:53:23.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:53:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.269+0000 7f773e883640 1 Processor -- start 2023-08-11T15:53:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.270+0000 7f773e883640 1 -- start start 2023-08-11T15:53:23.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.271+0000 7f773e883640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f77380cb700 0x7f773813fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.271+0000 7f773e883640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f773815b6b0 0x7f77381401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.271+0000 7f773e883640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7738140ba0 con 0x7f7738154ac0 2023-08-11T15:53:23.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.271+0000 7f773e883640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7738071b10 con 0x7f77380cb700 2023-08-11T15:53:23.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.271+0000 7f773e883640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773815b220 con 0x7f773815b6b0 2023-08-11T15:53:23.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.271+0000 7f77377fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f77380cb700 0x7f773813fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.271+0000 7f77377fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f77380cb700 0x7f773813fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54148/0 (socket says 172.21.15.27:54148) 2023-08-11T15:53:23.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.271+0000 7f77377fe640 1 -- 172.21.15.27:0/3604315539 learned_addr learned my addr 172.21.15.27:0/3604315539 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:23.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.272+0000 7f773ce20640 1 --2- 172.21.15.27:0/3604315539 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f773815b6b0 0x7f77381401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.272+0000 7f77377fe640 1 -- 172.21.15.27:0/3604315539 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f773815b6b0 msgr2=0x7f77381401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:23.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.272+0000 7f77377fe640 1 --2- 172.21.15.27:0/3604315539 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f773815b6b0 0x7f77381401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.272+0000 7f77377fe640 1 -- 172.21.15.27:0/3604315539 >> v1:172.21.15.119:6789/0 conn(0x7f7738154ac0 legacy=0x7f77380caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.272+0000 7f77377fe640 1 -- 172.21.15.27:0/3604315539 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77381408c0 con 0x7f77380cb700 2023-08-11T15:53:23.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.274+0000 7f77377fe640 1 --2- 172.21.15.27:0/3604315539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77380cb700 0x7f773813fc90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f772c009870 tx=0x7f772c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef8b8a6fa66acf2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:23.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.274+0000 7f7736ffd640 1 -- 172.21.15.27:0/3604315539 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c031640 con 0x7f77380cb700 2023-08-11T15:53:23.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.274+0000 7f7736ffd640 1 -- 172.21.15.27:0/3604315539 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c0317a0 con 0x7f77380cb700 2023-08-11T15:53:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.274+0000 7f7736ffd640 1 -- 172.21.15.27:0/3604315539 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c031b30 con 0x7f77380cb700 2023-08-11T15:53:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.275+0000 7f773e883640 1 -- 172.21.15.27:0/3604315539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77380cb700 msgr2=0x7f773813fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.275+0000 7f773e883640 1 --2- 172.21.15.27:0/3604315539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77380cb700 0x7f773813fc90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f772c009870 tx=0x7f772c02b8f0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.275+0000 7f773e883640 1 -- 172.21.15.27:0/3604315539 shutdown_connections 2023-08-11T15:53:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.275+0000 7f773e883640 1 --2- 172.21.15.27:0/3604315539 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f773815b6b0 0x7f77381401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.275+0000 7f773e883640 1 --2- 172.21.15.27:0/3604315539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77380cb700 0x7f773813fc90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.275+0000 7f773e883640 1 -- 172.21.15.27:0/3604315539 >> 172.21.15.27:0/3604315539 conn(0x7f7738069bf0 msgr2=0x7f773806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.275+0000 7f773e883640 1 -- 172.21.15.27:0/3604315539 shutdown_connections 2023-08-11T15:53:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.275+0000 7f773e883640 1 -- 172.21.15.27:0/3604315539 wait complete. 2023-08-11T15:53:23.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.277+0000 7f773e883640 1 Processor -- start 2023-08-11T15:53:23.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.277+0000 7f773e883640 1 -- start start 2023-08-11T15:53:23.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.278+0000 7f773e883640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77380cb700 0x7f77381f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.278+0000 7f773e883640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7738154ac0 0x7f77380c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.278+0000 7f773e883640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f773815b6b0 0x7f7738159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.278+0000 7f773e883640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77381f2cf0 con 0x7f7738154ac0 2023-08-11T15:53:23.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.278+0000 7f773e883640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77380c24d0 con 0x7f77380cb700 2023-08-11T15:53:23.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.278+0000 7f773e883640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77380c2700 con 0x7f773815b6b0 2023-08-11T15:53:23.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.278+0000 7f77377fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7738154ac0 0x7f77380c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.278+0000 7f77377fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7738154ac0 0x7f77380c1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54152/0 (socket says 172.21.15.27:54152) 2023-08-11T15:53:23.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.278+0000 7f77377fe640 1 -- 172.21.15.27:0/105172357 learned_addr learned my addr 172.21.15.27:0/105172357 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:23.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.279+0000 7f7737fff640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77380cb700 0x7f77381f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.279+0000 7f773ce20640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f773815b6b0 0x7f7738159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.279+0000 7f77377fe640 1 -- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f773815b6b0 msgr2=0x7f7738159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.279+0000 7f77377fe640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f773815b6b0 0x7f7738159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.279+0000 7f77377fe640 1 -- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77380cb700 msgr2=0x7f77381f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.279+0000 7f77377fe640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77380cb700 0x7f77381f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.279+0000 7f77377fe640 1 -- 172.21.15.27:0/105172357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0094d0 con 0x7f7738154ac0 2023-08-11T15:53:23.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.279+0000 7f77377fe640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7738154ac0 0x7f77380c1f90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f772c000c00 tx=0x7f772c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:23.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.280+0000 7f7734ff9640 1 -- 172.21.15.27:0/105172357 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c03ac60 con 0x7f7738154ac0 2023-08-11T15:53:23.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.280+0000 7f7734ff9640 1 -- 172.21.15.27:0/105172357 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c03adc0 con 0x7f7738154ac0 2023-08-11T15:53:23.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.280+0000 7f7734ff9640 1 -- 172.21.15.27:0/105172357 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c044630 con 0x7f7738154ac0 2023-08-11T15:53:23.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.280+0000 7f773e883640 1 -- 172.21.15.27:0/105172357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77380c29e0 con 0x7f7738154ac0 2023-08-11T15:53:23.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.280+0000 7f773e883640 1 -- 172.21.15.27:0/105172357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77381f2f20 con 0x7f7738154ac0 2023-08-11T15:53:23.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.281+0000 7f773e883640 1 -- 172.21.15.27:0/105172357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77381f3110 con 0x7f7738154ac0 2023-08-11T15:53:23.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.282+0000 7f7734ff9640 1 -- 172.21.15.27:0/105172357 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f772c02f050 con 0x7f7738154ac0 2023-08-11T15:53:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.283+0000 7f7734ff9640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f772406a050 0x7f772406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.283+0000 7f7734ff9640 1 -- 172.21.15.27:0/105172357 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f772c0aa5e0 con 0x7f7738154ac0 2023-08-11T15:53:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.284+0000 7f7737fff640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f772406a050 0x7f772406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.285+0000 7f7737fff640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f772406a050 0x7f772406c510 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7738140770 tx=0x7f7728009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:23.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.287+0000 7f7734ff9640 1 -- 172.21.15.27:0/105172357 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f772c075b90 con 0x7f7738154ac0 2023-08-11T15:53:23.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.479+0000 7f773e883640 1 -- 172.21.15.27:0/105172357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f773804e420 con 0x7f7738154ac0 2023-08-11T15:53:23.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.480+0000 7f7734ff9640 1 -- 172.21.15.27:0/105172357 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7f772c031e60 con 0x7f7738154ac0 2023-08-11T15:53:23.482 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:53:23.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 -- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f772406a050 msgr2=0x7f772406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f772406a050 0x7f772406c510 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7738140770 tx=0x7f7728009040 comp rx=0 tx=0).stop 2023-08-11T15:53:23.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 -- 172.21.15.27:0/105172357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7738154ac0 msgr2=0x7f77380c1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7738154ac0 0x7f77380c1f90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f772c000c00 tx=0x7f772c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 -- 172.21.15.27:0/105172357 shutdown_connections 2023-08-11T15:53:23.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f772406a050 0x7f772406c510 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f773815b6b0 0x7f7738159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7738154ac0 0x7f77380c1f90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 --2- 172.21.15.27:0/105172357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77380cb700 0x7f77381f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 -- 172.21.15.27:0/105172357 >> 172.21.15.27:0/105172357 conn(0x7f7738069bf0 msgr2=0x7f77380b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:23.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.483+0000 7f773e883640 1 -- 172.21.15.27:0/105172357 shutdown_connections 2023-08-11T15:53:23.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.484+0000 7f773e883640 1 -- 172.21.15.27:0/105172357 wait complete. 2023-08-11T15:53:23.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:53:23.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.755+0000 7f7e2205d640 1 Processor -- start 2023-08-11T15:53:23.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.755+0000 7f7e2205d640 1 -- start start 2023-08-11T15:53:23.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.757+0000 7f7e2205d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e1c0cec00 0x7f7e1c0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.757+0000 7f7e2205d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c0b51f0 0x7f7e1c0b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.757+0000 7f7e2205d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e1c0d0dc0 con 0x7f7e1c15c5b0 2023-08-11T15:53:23.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.757+0000 7f7e2205d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e1c0cf5d0 con 0x7f7e1c0cec00 2023-08-11T15:53:23.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.757+0000 7f7e2205d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c0d10a0 con 0x7f7e1c0b51f0 2023-08-11T15:53:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.757+0000 7f7e2105b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7e1c15c5b0 0x7f7e1c0d0250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35208/0 (socket says 172.21.15.27:35208) 2023-08-11T15:53:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.757+0000 7f7e2105b640 1 -- 172.21.15.27:0/1664266602 learned_addr learned my addr 172.21.15.27:0/1664266602 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.757+0000 7f7e2085a640 1 --2- 172.21.15.27:0/1664266602 >> v2:172.21.15.119:3300/0 conn(0x7f7e1c0cec00 0x7f7e1c0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.757+0000 7f7e2185c640 1 --2- 172.21.15.27:0/1664266602 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c0b51f0 0x7f7e1c0b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.757+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929290398 0 0) 0x7f7e1c0d0dc0 con 0x7f7e1c15c5b0 2023-08-11T15:53:23.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e08003520 con 0x7f7e1c15c5b0 2023-08-11T15:53:23.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1817811014 0 0) 0x7f7e08003520 con 0x7f7e1c15c5b0 2023-08-11T15:53:23.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c0b51f0 msgr2=0x7f7e1c0b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:23.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 --2- 172.21.15.27:0/1664266602 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c0b51f0 0x7f7e1c0b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 >> v2:172.21.15.119:3300/0 conn(0x7f7e1c0cec00 msgr2=0x7f7e1c0cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 --2- 172.21.15.27:0/1664266602 >> v2:172.21.15.119:3300/0 conn(0x7f7e1c0cec00 0x7f7e1c0cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1c0d1380 con 0x7f7e1c15c5b0 2023-08-11T15:53:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f7e18001670 con 0x7f7e1c15c5b0 2023-08-11T15:53:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 >> v1:172.21.15.27:6789/0 conn(0x7f7e1c15c5b0 legacy=0x7f7e1c0d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 --2- 172.21.15.27:0/1664266602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 0x7f7e08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 --2- 172.21.15.27:0/1664266602 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c0b51f0 0x7f7e080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 --2- 172.21.15.27:0/1664266602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e08004d00 0x7f7e08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c0cf5d0 con 0x7f7e08003970 2023-08-11T15:53:23.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c0d10a0 con 0x7f7e08004d00 2023-08-11T15:53:23.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0800ba40 con 0x7f7e1c0b51f0 2023-08-11T15:53:23.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e2085a640 1 --2- 172.21.15.27:0/1664266602 >> v2:172.21.15.119:3300/0 conn(0x7f7e1c0cec00 0x7f7e1c0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:23.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.758+0000 7f7e2105b640 1 --2- 172.21.15.27:0/1664266602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 0x7f7e08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.759+0000 7f7e2105b640 1 -- 172.21.15.27:0/1664266602 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c0b51f0 msgr2=0x7f7e080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.759+0000 7f7e2105b640 1 --2- 172.21.15.27:0/1664266602 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c0b51f0 0x7f7e080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.759+0000 7f7e2105b640 1 -- 172.21.15.27:0/1664266602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e08004d00 msgr2=0x7f7e08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.759+0000 7f7e2105b640 1 --2- 172.21.15.27:0/1664266602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e08004d00 0x7f7e08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.759+0000 7f7e2105b640 1 -- 172.21.15.27:0/1664266602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e08003520 con 0x7f7e08003970 2023-08-11T15:53:23.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.760+0000 7f7e2105b640 1 --2- 172.21.15.27:0/1664266602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 0x7f7e08003d90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c240 tx=0x7f7e080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534873a2a61ef07f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:23.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.760+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1802cb00 con 0x7f7e08003970 2023-08-11T15:53:23.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.760+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1802cc60 con 0x7f7e08003970 2023-08-11T15:53:23.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.760+0000 7f7e13fff640 1 -- 172.21.15.27:0/1664266602 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18005af0 con 0x7f7e08003970 2023-08-11T15:53:23.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.761+0000 7f7e2205d640 1 -- 172.21.15.27:0/1664266602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 msgr2=0x7f7e08003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.761+0000 7f7e2205d640 1 --2- 172.21.15.27:0/1664266602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 0x7f7e08003d90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c240 tx=0x7f7e080057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.761+0000 7f7e2185c640 1 -- 172.21.15.27:0/1664266602 reap_dead start 2023-08-11T15:53:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.761+0000 7f7e2205d640 1 -- 172.21.15.27:0/1664266602 shutdown_connections 2023-08-11T15:53:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.761+0000 7f7e2205d640 1 -- 172.21.15.27:0/1664266602 >> 172.21.15.27:0/1664266602 conn(0x7f7e1c06a270 msgr2=0x7f7e1c06a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.761+0000 7f7e2205d640 1 -- 172.21.15.27:0/1664266602 shutdown_connections 2023-08-11T15:53:23.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.761+0000 7f7e2205d640 1 -- 172.21.15.27:0/1664266602 wait complete. 2023-08-11T15:53:23.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.763+0000 7f7e2205d640 1 Processor -- start 2023-08-11T15:53:23.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.764+0000 7f7e2205d640 1 -- start start 2023-08-11T15:53:23.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2205d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 0x7f7e1c0c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2205d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c150310 0x7f7e1c150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2205d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e1c151ef0 0x7f7e1c1523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2205d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1528e0 con 0x7f7e08003970 2023-08-11T15:53:23.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2205d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1f1d40 con 0x7f7e1c151ef0 2023-08-11T15:53:23.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2205d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1f2020 con 0x7f7e1c150310 2023-08-11T15:53:23.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2105b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 0x7f7e1c0c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2105b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 0x7f7e1c0c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54166/0 (socket says 172.21.15.27:54166) 2023-08-11T15:53:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2105b640 1 -- 172.21.15.27:0/1526827786 learned_addr learned my addr 172.21.15.27:0/1526827786 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2085a640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c150310 0x7f7e1c150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.765+0000 7f7e2185c640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e1c151ef0 0x7f7e1c1523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.766+0000 7f7e2105b640 1 -- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c150310 msgr2=0x7f7e1c150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.766+0000 7f7e2105b640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c150310 0x7f7e1c150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.766+0000 7f7e2105b640 1 -- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e1c151ef0 msgr2=0x7f7e1c1523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.766+0000 7f7e2105b640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e1c151ef0 0x7f7e1c1523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.766+0000 7f7e2105b640 1 -- 172.21.15.27:0/1526827786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1c1f2300 con 0x7f7e08003970 2023-08-11T15:53:23.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.766+0000 7f7e2105b640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 0x7f7e1c0c7c60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c240 tx=0x7f7e18001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:23.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.766+0000 7f7e11ffb640 1 -- 172.21.15.27:0/1526827786 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18039070 con 0x7f7e08003970 2023-08-11T15:53:23.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.766+0000 7f7e11ffb640 1 -- 172.21.15.27:0/1526827786 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e18001630 con 0x7f7e08003970 2023-08-11T15:53:23.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.766+0000 7f7e11ffb640 1 -- 172.21.15.27:0/1526827786 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1803c040 con 0x7f7e08003970 2023-08-11T15:53:23.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.766+0000 7f7e2205d640 1 -- 172.21.15.27:0/1526827786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1c1f25e0 con 0x7f7e08003970 2023-08-11T15:53:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.767+0000 7f7e2205d640 1 -- 172.21.15.27:0/1526827786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1c0c81a0 con 0x7f7e08003970 2023-08-11T15:53:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.768+0000 7f7e2205d640 1 -- 172.21.15.27:0/1526827786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e1c04e420 con 0x7f7e08003970 2023-08-11T15:53:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.769+0000 7f7e11ffb640 1 -- 172.21.15.27:0/1526827786 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7e18032070 con 0x7f7e08003970 2023-08-11T15:53:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.770+0000 7f7e11ffb640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e0006a050 0x7f7e0006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.770+0000 7f7e11ffb640 1 -- 172.21.15.27:0/1526827786 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7e180a9d50 con 0x7f7e08003970 2023-08-11T15:53:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.771+0000 7f7e2085a640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e0006a050 0x7f7e0006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.772+0000 7f7e2085a640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e0006a050 0x7f7e0006c510 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c006a90 tx=0x7f7e0c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.775+0000 7f7e11ffb640 1 -- 172.21.15.27:0/1526827786 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f7e180dc7c0 con 0x7f7e08003970 2023-08-11T15:53:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.978+0000 7f7e2205d640 1 -- 172.21.15.27:0/1526827786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e1c159cd0 con 0x7f7e08003970 2023-08-11T15:53:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.980+0000 7f7e11ffb640 1 -- 172.21.15.27:0/1526827786 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e18046020 con 0x7f7e08003970 2023-08-11T15:53:23.981 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:53:23.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 -- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e0006a050 msgr2=0x7f7e0006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e0006a050 0x7f7e0006c510 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c006a90 tx=0x7f7e0c008040 comp rx=0 tx=0).stop 2023-08-11T15:53:23.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 -- 172.21.15.27:0/1526827786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 msgr2=0x7f7e1c0c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:23.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 0x7f7e1c0c7c60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c240 tx=0x7f7e18001a00 comp rx=0 tx=0).stop 2023-08-11T15:53:23.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 -- 172.21.15.27:0/1526827786 shutdown_connections 2023-08-11T15:53:23.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7e0006a050 0x7f7e0006c510 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7e1c150310 0x7f7e1c150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e1c151ef0 0x7f7e1c1523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 --2- 172.21.15.27:0/1526827786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e08003970 0x7f7e1c0c7c60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 -- 172.21.15.27:0/1526827786 >> 172.21.15.27:0/1526827786 conn(0x7f7e1c06a270 msgr2=0x7f7e1c153830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 -- 172.21.15.27:0/1526827786 shutdown_connections 2023-08-11T15:53:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:23.983+0000 7f7e2205d640 1 -- 172.21.15.27:0/1526827786 wait complete. 2023-08-11T15:53:24.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:53:24.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.240+0000 7fa908c71640 1 Processor -- start 2023-08-11T15:53:24.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.241+0000 7fa908c71640 1 -- start start 2023-08-11T15:53:24.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.242+0000 7fa908c71640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9040cf340 0x7fa9040b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.242+0000 7fa908c71640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa9040d1d30 0x7fa9040d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.242+0000 7fa908c71640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9040d0c90 con 0x7fa90415c430 2023-08-11T15:53:24.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.242+0000 7fa908c71640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9040d1910 con 0x7fa9040d1d30 2023-08-11T15:53:24.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.242+0000 7fa908c71640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040b8280 con 0x7fa9040cf340 2023-08-11T15:53:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.242+0000 7fa903fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa9040d1d30 0x7fa9040d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.242+0000 7fa903fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa9040d1d30 0x7fa9040d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54170/0 (socket says 172.21.15.27:54170) 2023-08-11T15:53:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.242+0000 7fa903fff640 1 -- 172.21.15.27:0/3595270876 learned_addr learned my addr 172.21.15.27:0/3595270876 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.243+0000 7fa902ffd640 1 --2- 172.21.15.27:0/3595270876 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9040cf340 0x7fa9040b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.243+0000 7fa903fff640 1 -- 172.21.15.27:0/3595270876 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9040cf340 msgr2=0x7fa9040b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.243+0000 7fa903fff640 1 --2- 172.21.15.27:0/3595270876 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9040cf340 0x7fa9040b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.243+0000 7fa903fff640 1 -- 172.21.15.27:0/3595270876 >> v1:172.21.15.27:6789/0 conn(0x7fa90415c430 legacy=0x7fa9040cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.243+0000 7fa903fff640 1 -- 172.21.15.27:0/3595270876 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040d0f70 con 0x7fa9040d1d30 2023-08-11T15:53:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.244+0000 7fa903fff640 1 --2- 172.21.15.27:0/3595270876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 0x7fa9040d0560 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009870 tx=0x7fa8fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6393892f47da62f8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.244+0000 7fa9027fc640 1 -- 172.21.15.27:0/3595270876 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc031640 con 0x7fa9040d1d30 2023-08-11T15:53:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.244+0000 7fa9027fc640 1 -- 172.21.15.27:0/3595270876 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc0317a0 con 0x7fa9040d1d30 2023-08-11T15:53:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.244+0000 7fa9027fc640 1 -- 172.21.15.27:0/3595270876 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc031b30 con 0x7fa9040d1d30 2023-08-11T15:53:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.245+0000 7fa908c71640 1 -- 172.21.15.27:0/3595270876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 msgr2=0x7fa9040d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.245+0000 7fa908c71640 1 --2- 172.21.15.27:0/3595270876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 0x7fa9040d0560 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009870 tx=0x7fa8fc02b8f0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.245+0000 7fa908c71640 1 -- 172.21.15.27:0/3595270876 shutdown_connections 2023-08-11T15:53:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.245+0000 7fa908c71640 1 --2- 172.21.15.27:0/3595270876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 0x7fa9040d0560 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.245+0000 7fa908c71640 1 --2- 172.21.15.27:0/3595270876 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9040cf340 0x7fa9040b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.245+0000 7fa908c71640 1 -- 172.21.15.27:0/3595270876 >> 172.21.15.27:0/3595270876 conn(0x7fa90406a2b0 msgr2=0x7fa90406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.245+0000 7fa908c71640 1 -- 172.21.15.27:0/3595270876 shutdown_connections 2023-08-11T15:53:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.245+0000 7fa908c71640 1 -- 172.21.15.27:0/3595270876 wait complete. 2023-08-11T15:53:24.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.247+0000 7fa908c71640 1 Processor -- start 2023-08-11T15:53:24.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.247+0000 7fa908c71640 1 -- start start 2023-08-11T15:53:24.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.248+0000 7fa908c71640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf340 0x7fa90415e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.248+0000 7fa908c71640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 0x7fa9040cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.248+0000 7fa908c71640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa90415c430 0x7fa9040cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.248+0000 7fa908c71640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040cc4a0 con 0x7fa9040d1d30 2023-08-11T15:53:24.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.248+0000 7fa908c71640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90415fc60 con 0x7fa9040cf340 2023-08-11T15:53:24.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.248+0000 7fa908c71640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90415ed90 con 0x7fa90415c430 2023-08-11T15:53:24.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa902ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 0x7fa9040cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa902ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 0x7fa9040cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54174/0 (socket says 172.21.15.27:54174) 2023-08-11T15:53:24.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa902ffd640 1 -- 172.21.15.27:0/224986332 learned_addr learned my addr 172.21.15.27:0/224986332 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:24.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa9037fe640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf340 0x7fa90415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa902ffd640 1 -- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa90415c430 msgr2=0x7fa9040cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa903fff640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa90415c430 0x7fa9040cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa902ffd640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa90415c430 0x7fa9040cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa902ffd640 1 -- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf340 msgr2=0x7fa90415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa902ffd640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf340 0x7fa90415e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa902ffd640 1 -- 172.21.15.27:0/224986332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc0094d0 con 0x7fa9040d1d30 2023-08-11T15:53:24.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa9037fe640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf340 0x7fa90415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:24.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.249+0000 7fa902ffd640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 0x7fa9040cba20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00d800 tx=0x7fa8ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:24.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.250+0000 7fa8fbfff640 1 -- 172.21.15.27:0/224986332 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00b6c0 con 0x7fa9040d1d30 2023-08-11T15:53:24.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.250+0000 7fa8fbfff640 1 -- 172.21.15.27:0/224986332 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec00b820 con 0x7fa9040d1d30 2023-08-11T15:53:24.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.250+0000 7fa8fbfff640 1 -- 172.21.15.27:0/224986332 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00bbb0 con 0x7fa9040d1d30 2023-08-11T15:53:24.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.250+0000 7fa908c71640 1 -- 172.21.15.27:0/224986332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90415f050 con 0x7fa9040d1d30 2023-08-11T15:53:24.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.250+0000 7fa908c71640 1 -- 172.21.15.27:0/224986332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90415f610 con 0x7fa9040d1d30 2023-08-11T15:53:24.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.251+0000 7fa908c71640 1 -- 172.21.15.27:0/224986332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90404e420 con 0x7fa9040d1d30 2023-08-11T15:53:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.252+0000 7fa8fbfff640 1 -- 172.21.15.27:0/224986332 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa8ec00bd10 con 0x7fa9040d1d30 2023-08-11T15:53:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.253+0000 7fa8fbfff640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa8e406a050 0x7fa8e406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.253+0000 7fa8fbfff640 1 -- 172.21.15.27:0/224986332 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa8ec088bc0 con 0x7fa9040d1d30 2023-08-11T15:53:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.254+0000 7fa9037fe640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa8e406a050 0x7fa8e406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.255+0000 7fa9037fe640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa8e406a050 0x7fa8e406c510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40012e0 tx=0x7fa8f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:24.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.257+0000 7fa8fbfff640 1 -- 172.21.15.27:0/224986332 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fa8ec0541e0 con 0x7fa9040d1d30 2023-08-11T15:53:24.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.457+0000 7fa908c71640 1 -- 172.21.15.27:0/224986332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9040ce950 con 0x7fa9040d1d30 2023-08-11T15:53:24.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.458+0000 7fa8fbfff640 1 -- 172.21.15.27:0/224986332 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa8ec057800 con 0x7fa9040d1d30 2023-08-11T15:53:24.459 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:53:24.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.460+0000 7fa908c71640 1 -- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa8e406a050 msgr2=0x7fa8e406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.460+0000 7fa908c71640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa8e406a050 0x7fa8e406c510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40012e0 tx=0x7fa8f4009040 comp rx=0 tx=0).stop 2023-08-11T15:53:24.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.460+0000 7fa908c71640 1 -- 172.21.15.27:0/224986332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 msgr2=0x7fa9040cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.460+0000 7fa908c71640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 0x7fa9040cba20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00d800 tx=0x7fa8ec00dcc0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.460+0000 7fa908c71640 1 -- 172.21.15.27:0/224986332 shutdown_connections 2023-08-11T15:53:24.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.460+0000 7fa908c71640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa8e406a050 0x7fa8e406c510 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.460+0000 7fa908c71640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa90415c430 0x7fa9040cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.460+0000 7fa908c71640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9040d1d30 0x7fa9040cba20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.460+0000 7fa908c71640 1 --2- 172.21.15.27:0/224986332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf340 0x7fa90415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.460+0000 7fa908c71640 1 -- 172.21.15.27:0/224986332 >> 172.21.15.27:0/224986332 conn(0x7fa90406a2b0 msgr2=0x7fa9040c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:24.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.461+0000 7fa908c71640 1 -- 172.21.15.27:0/224986332 shutdown_connections 2023-08-11T15:53:24.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.461+0000 7fa908c71640 1 -- 172.21.15.27:0/224986332 wait complete. 2023-08-11T15:53:24.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:53:24.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.700+0000 7fb0a79b5640 1 Processor -- start 2023-08-11T15:53:24.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.700+0000 7fb0a79b5640 1 -- start start 2023-08-11T15:53:24.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.702+0000 7fb0a79b5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb0a015a3b0 0x7fb0a0071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.702+0000 7fb0a79b5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a00cb520 con 0x7fb0a006f6e0 2023-08-11T15:53:24.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.702+0000 7fb0a79b5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a015a790 con 0x7fb0a015df00 2023-08-11T15:53:24.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.702+0000 7fb0a79b5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a015eea0 con 0x7fb0a015a3b0 2023-08-11T15:53:24.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.702+0000 7fb0a5f52640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb0a006f6e0 0x7fb0a00c8f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35228/0 (socket says 172.21.15.27:35228) 2023-08-11T15:53:24.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.702+0000 7fb0a5f52640 1 -- 172.21.15.27:0/1301625116 learned_addr learned my addr 172.21.15.27:0/1301625116 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb0a5751640 1 --2- 172.21.15.27:0/1301625116 >> v2:172.21.15.119:3300/0 conn(0x7fb0a015a3b0 0x7fb0a0071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241086271 0 0) 0x7fb0a00cb520 con 0x7fb0a006f6e0 2023-08-11T15:53:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb08c003520 con 0x7fb0a006f6e0 2023-08-11T15:53:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2083488483 0 0) 0x7fb08c003520 con 0x7fb0a006f6e0 2023-08-11T15:53:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 >> v2:172.21.15.119:3300/0 conn(0x7fb0a015a3b0 msgr2=0x7fb0a0071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 --2- 172.21.15.27:0/1301625116 >> v2:172.21.15.119:3300/0 conn(0x7fb0a015a3b0 0x7fb0a0071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 >> v1:172.21.15.119:6789/0 conn(0x7fb0a015df00 legacy=0x7fb0a00b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a015e360 con 0x7fb0a006f6e0 2023-08-11T15:53:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fb0980019f0 con 0x7fb0a006f6e0 2023-08-11T15:53:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 >> v1:172.21.15.27:6789/0 conn(0x7fb0a006f6e0 legacy=0x7fb0a00c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c003970 0x7fb08c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb08c0047c0 0x7fb08c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb08c005180 0x7fb08c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a015eea0 con 0x7fb08c003970 2023-08-11T15:53:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c00c020 con 0x7fb08c0047c0 2023-08-11T15:53:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.703+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c00c1e0 con 0x7fb08c005180 2023-08-11T15:53:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.704+0000 7fb0a5751640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c003970 0x7fb08c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.704+0000 7fb0a4f50640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb08c0047c0 0x7fb08c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.704+0000 7fb0a5f52640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb08c005180 0x7fb08c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.704+0000 7fb0a5751640 1 -- 172.21.15.27:0/1301625116 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb08c005180 msgr2=0x7fb08c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.704+0000 7fb0a5751640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb08c005180 0x7fb08c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.704+0000 7fb0a5751640 1 -- 172.21.15.27:0/1301625116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb08c0047c0 msgr2=0x7fb08c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.704+0000 7fb0a5751640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb08c0047c0 0x7fb08c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.704+0000 7fb0a5751640 1 -- 172.21.15.27:0/1301625116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c003520 con 0x7fb08c003970 2023-08-11T15:53:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.704+0000 7fb0a5f52640 1 -- 172.21.15.27:0/1301625116 reap_dead start 2023-08-11T15:53:24.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.706+0000 7fb0a5751640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c003970 0x7fb08c003d90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009ac0 tx=0x7fb08c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8767f53d56b42a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.706+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c02c6b0 con 0x7fb08c003970 2023-08-11T15:53:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.706+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c02c810 con 0x7fb08c003970 2023-08-11T15:53:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.706+0000 7fb097fff640 1 -- 172.21.15.27:0/1301625116 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c02cba0 con 0x7fb08c003970 2023-08-11T15:53:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.707+0000 7fb0a79b5640 1 -- 172.21.15.27:0/1301625116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c003970 msgr2=0x7fb08c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.707+0000 7fb0a79b5640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c003970 0x7fb08c003d90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009ac0 tx=0x7fb08c0057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.707+0000 7fb0a79b5640 1 -- 172.21.15.27:0/1301625116 shutdown_connections 2023-08-11T15:53:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.707+0000 7fb0a79b5640 1 --2- 172.21.15.27:0/1301625116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c003970 0x7fb08c003d90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.707+0000 7fb0a79b5640 1 -- 172.21.15.27:0/1301625116 >> 172.21.15.27:0/1301625116 conn(0x7fb0a0069e90 msgr2=0x7fb0a00afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.707+0000 7fb0a79b5640 1 -- 172.21.15.27:0/1301625116 shutdown_connections 2023-08-11T15:53:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.707+0000 7fb0a79b5640 1 -- 172.21.15.27:0/1301625116 wait complete. 2023-08-11T15:53:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.709+0000 7fb0a79b5640 1 Processor -- start 2023-08-11T15:53:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.709+0000 7fb0a79b5640 1 -- start start 2023-08-11T15:53:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.710+0000 7fb0a79b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb08c003970 0x7fb0a00ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.710+0000 7fb0a79b5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0a015c620 0x7fb0a015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.710+0000 7fb0a79b5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a015cfc0 0x7fb0a00ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.710+0000 7fb0a79b5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00cc210 con 0x7fb0a015cfc0 2023-08-11T15:53:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.710+0000 7fb0a79b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00cbbe0 con 0x7fb08c003970 2023-08-11T15:53:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.710+0000 7fb0a79b5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a015d490 con 0x7fb0a015c620 2023-08-11T15:53:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.710+0000 7fb0a5f52640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a015cfc0 0x7fb0a00ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.710+0000 7fb0a5f52640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a015cfc0 0x7fb0a00ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54190/0 (socket says 172.21.15.27:54190) 2023-08-11T15:53:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.711+0000 7fb0a5f52640 1 -- 172.21.15.27:0/3364913235 learned_addr learned my addr 172.21.15.27:0/3364913235 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.711+0000 7fb0a5751640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb08c003970 0x7fb0a00ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.711+0000 7fb0a4f50640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0a015c620 0x7fb0a015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.711+0000 7fb0a5f52640 1 -- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0a015c620 msgr2=0x7fb0a015ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.711+0000 7fb0a5f52640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0a015c620 0x7fb0a015ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.711+0000 7fb0a5f52640 1 -- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb08c003970 msgr2=0x7fb0a00ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.711+0000 7fb0a5f52640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb08c003970 0x7fb0a00ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.711+0000 7fb0a5f52640 1 -- 172.21.15.27:0/3364913235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a015d770 con 0x7fb0a015cfc0 2023-08-11T15:53:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.711+0000 7fb0a5751640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb08c003970 0x7fb0a00ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.711+0000 7fb0a5f52640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a015cfc0 0x7fb0a00ab8e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb098000f80 tx=0x7fb09800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.712+0000 7fb095ffb640 1 -- 172.21.15.27:0/3364913235 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09800b800 con 0x7fb0a015cfc0 2023-08-11T15:53:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.712+0000 7fb095ffb640 1 -- 172.21.15.27:0/3364913235 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09800b960 con 0x7fb0a015cfc0 2023-08-11T15:53:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.712+0000 7fb095ffb640 1 -- 172.21.15.27:0/3364913235 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09800bcf0 con 0x7fb0a015cfc0 2023-08-11T15:53:24.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.712+0000 7fb0a79b5640 1 -- 172.21.15.27:0/3364913235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a00cbea0 con 0x7fb0a015cfc0 2023-08-11T15:53:24.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.712+0000 7fb0a79b5640 1 -- 172.21.15.27:0/3364913235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a00ae7c0 con 0x7fb0a015cfc0 2023-08-11T15:53:24.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.713+0000 7fb0a79b5640 1 -- 172.21.15.27:0/3364913235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a00cc3c0 con 0x7fb0a015cfc0 2023-08-11T15:53:24.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.714+0000 7fb095ffb640 1 -- 172.21.15.27:0/3364913235 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb098007070 con 0x7fb0a015cfc0 2023-08-11T15:53:24.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.715+0000 7fb095ffb640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb08406a050 0x7fb08406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:24.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.715+0000 7fb095ffb640 1 -- 172.21.15.27:0/3364913235 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb098088710 con 0x7fb0a015cfc0 2023-08-11T15:53:24.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.716+0000 7fb0a5751640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb08406a050 0x7fb08406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:24.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.717+0000 7fb0a5751640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb08406a050 0x7fb08406c510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb09c036000 tx=0x7fb09c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:24.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.719+0000 7fb095ffb640 1 -- 172.21.15.27:0/3364913235 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fb0a00cc3c0 con 0x7fb0a015cfc0 2023-08-11T15:53:24.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.930+0000 7fb0a79b5640 1 -- 172.21.15.27:0/3364913235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb0a00ae7c0 con 0x7fb0a015cfc0 2023-08-11T15:53:24.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.931+0000 7fb095ffb640 1 -- 172.21.15.27:0/3364913235 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb0a00ae7c0 con 0x7fb0a015cfc0 2023-08-11T15:53:24.932 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:53:24.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 -- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb08406a050 msgr2=0x7fb08406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb08406a050 0x7fb08406c510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb09c036000 tx=0x7fb09c030000 comp rx=0 tx=0).stop 2023-08-11T15:53:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 -- 172.21.15.27:0/3364913235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a015cfc0 msgr2=0x7fb0a00ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a015cfc0 0x7fb0a00ab8e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb098000f80 tx=0x7fb09800df70 comp rx=0 tx=0).stop 2023-08-11T15:53:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 -- 172.21.15.27:0/3364913235 shutdown_connections 2023-08-11T15:53:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb08406a050 0x7fb08406c510 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0a015c620 0x7fb0a015ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a015cfc0 0x7fb0a00ab8e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 --2- 172.21.15.27:0/3364913235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb08c003970 0x7fb0a00ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:24.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 -- 172.21.15.27:0/3364913235 >> 172.21.15.27:0/3364913235 conn(0x7fb0a0069e90 msgr2=0x7fb0a00aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:24.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 -- 172.21.15.27:0/3364913235 shutdown_connections 2023-08-11T15:53:24.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:24.934+0000 7fb0a79b5640 1 -- 172.21.15.27:0/3364913235 wait complete. 2023-08-11T15:53:24.954 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-08-11T15:53:24.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:24.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-11T15:53:25.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[106006]: fuse finished with error 0 and tester_r 0 2023-08-11T15:53:25.058 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:53:25.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:25.059 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-11T15:53:25.243 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:53:25.243 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:53:25.302 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:53:25.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.539+0000 7fa156d96640 1 Processor -- start 2023-08-11T15:53:25.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.539+0000 7fa156d96640 1 -- start start 2023-08-11T15:53:25.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.540+0000 7fa156d96640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150156e50 0x7fa15015a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:25.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.540+0000 7fa156d96640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1500cbfb0 con 0x7fa1500cbb70 2023-08-11T15:53:25.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.540+0000 7fa156d96640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1500cc270 con 0x7fa1500cb0c0 2023-08-11T15:53:25.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.540+0000 7fa156d96640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500ca460 con 0x7fa150156e50 2023-08-11T15:53:25.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.541+0000 7fa14ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa1500cbb70 0x7fa1500ca9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35244/0 (socket says 172.21.15.27:35244) 2023-08-11T15:53:25.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.541+0000 7fa14ffff640 1 -- 172.21.15.27:0/2709452918 learned_addr learned my addr 172.21.15.27:0/2709452918 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:25.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.541+0000 7fa154b32640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150156e50 0x7fa15015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:25.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.541+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955636416 0 0) 0x7fa1500cbfb0 con 0x7fa1500cbb70 2023-08-11T15:53:25.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.541+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa140005730 con 0x7fa1500cbb70 2023-08-11T15:53:25.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2882614956 0 0) 0x7fa140005730 con 0x7fa1500cbb70 2023-08-11T15:53:25.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150156e50 msgr2=0x7fa15015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:25.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150156e50 0x7fa15015a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:25.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 >> v1:172.21.15.119:6789/0 conn(0x7fa1500cb0c0 legacy=0x7fa1500708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:25.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500cc550 con 0x7fa1500cbb70 2023-08-11T15:53:25.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fa13c001670 con 0x7fa1500cbb70 2023-08-11T15:53:25.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 >> v1:172.21.15.27:6789/0 conn(0x7fa1500cbb70 legacy=0x7fa1500ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:25.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150156e50 0x7fa140005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:25.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa140006130 0x7fa1400069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:25.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 0x7fa140007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:25.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500ca460 con 0x7fa140006f10 2023-08-11T15:53:25.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14000dc70 con 0x7fa140006130 2023-08-11T15:53:25.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14000de30 con 0x7fa150156e50 2023-08-11T15:53:25.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.542+0000 7fa155333640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 0x7fa140007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:25.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.543+0000 7fa154b32640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150156e50 0x7fa140005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:25.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.543+0000 7fa155333640 1 -- 172.21.15.27:0/2709452918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150156e50 msgr2=0x7fa140005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:25.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.543+0000 7fa155333640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150156e50 0x7fa140005bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:25.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.543+0000 7fa155333640 1 -- 172.21.15.27:0/2709452918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa140006130 msgr2=0x7fa1400069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:25.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.543+0000 7fa155333640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa140006130 0x7fa1400069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:25.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.543+0000 7fa155333640 1 -- 172.21.15.27:0/2709452918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa140005730 con 0x7fa140006f10 2023-08-11T15:53:25.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.544+0000 7fa155333640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 0x7fa140007390 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa148001e50 tx=0x7fa140007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42aa83b206af726f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:25.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.544+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148003b70 con 0x7fa140006f10 2023-08-11T15:53:25.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.544+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa148003cd0 con 0x7fa140006f10 2023-08-11T15:53:25.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.544+0000 7fa14f7fe640 1 -- 172.21.15.27:0/2709452918 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148005310 con 0x7fa140006f10 2023-08-11T15:53:25.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.544+0000 7fa156d96640 1 -- 172.21.15.27:0/2709452918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 msgr2=0x7fa140007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:25.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.544+0000 7fa156d96640 1 --2- 172.21.15.27:0/2709452918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 0x7fa140007390 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa148001e50 tx=0x7fa140007a00 comp rx=0 tx=0).stop 2023-08-11T15:53:25.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.544+0000 7fa155333640 1 -- 172.21.15.27:0/2709452918 reap_dead start 2023-08-11T15:53:25.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.545+0000 7fa156d96640 1 -- 172.21.15.27:0/2709452918 shutdown_connections 2023-08-11T15:53:25.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.545+0000 7fa156d96640 1 -- 172.21.15.27:0/2709452918 >> 172.21.15.27:0/2709452918 conn(0x7fa1500afb70 msgr2=0x7fa1500aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:25.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.545+0000 7fa156d96640 1 -- 172.21.15.27:0/2709452918 shutdown_connections 2023-08-11T15:53:25.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.545+0000 7fa156d96640 1 -- 172.21.15.27:0/2709452918 wait complete. 2023-08-11T15:53:25.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.546+0000 7fa156d96640 1 Processor -- start 2023-08-11T15:53:25.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.547+0000 7fa156d96640 1 -- start start 2023-08-11T15:53:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa156d96640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 0x7fa150155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa156d96640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150155260 0x7fa1500b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa156d96640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1500aeed0 0x7fa1500af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa156d96640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500c4070 con 0x7fa140006f10 2023-08-11T15:53:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa156d96640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500af8c0 con 0x7fa1500aeed0 2023-08-11T15:53:25.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa156d96640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa150156200 con 0x7fa150155260 2023-08-11T15:53:25.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa154b32640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 0x7fa150155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:25.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa154b32640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 0x7fa150155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54210/0 (socket says 172.21.15.27:54210) 2023-08-11T15:53:25.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa154b32640 1 -- 172.21.15.27:0/2774183810 learned_addr learned my addr 172.21.15.27:0/2774183810 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:25.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa154b32640 1 -- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150155260 msgr2=0x7fa1500b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:25.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa154b32640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150155260 0x7fa1500b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:25.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa154b32640 1 -- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1500aeed0 msgr2=0x7fa1500af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa154b32640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1500aeed0 0x7fa1500af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.548+0000 7fa154b32640 1 -- 172.21.15.27:0/2774183810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500c1ad0 con 0x7fa140006f10 2023-08-11T15:53:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.549+0000 7fa154b32640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 0x7fa150155910 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa1440108c0 tx=0x7fa144010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.549+0000 7fa14d7fa640 1 -- 172.21.15.27:0/2774183810 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144009600 con 0x7fa140006f10 2023-08-11T15:53:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.549+0000 7fa14d7fa640 1 -- 172.21.15.27:0/2774183810 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa144005b90 con 0x7fa140006f10 2023-08-11T15:53:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.549+0000 7fa14d7fa640 1 -- 172.21.15.27:0/2774183810 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400e8e0 con 0x7fa140006f10 2023-08-11T15:53:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.549+0000 7fa156d96640 1 -- 172.21.15.27:0/2774183810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1500b3660 con 0x7fa140006f10 2023-08-11T15:53:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.549+0000 7fa156d96640 1 -- 172.21.15.27:0/2774183810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1501556e0 con 0x7fa140006f10 2023-08-11T15:53:25.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.550+0000 7fa156d96640 1 -- 172.21.15.27:0/2774183810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1500c41b0 con 0x7fa140006f10 2023-08-11T15:53:25.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.551+0000 7fa14d7fa640 1 -- 172.21.15.27:0/2774183810 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa14400ea40 con 0x7fa140006f10 2023-08-11T15:53:25.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.552+0000 7fa14d7fa640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa13406a080 0x7fa13406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.552+0000 7fa14d7fa640 1 -- 172.21.15.27:0/2774183810 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa144088940 con 0x7fa140006f10 2023-08-11T15:53:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.553+0000 7fa14ffff640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa13406a080 0x7fa13406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.554+0000 7fa14ffff640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa13406a080 0x7fa13406c540 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa1501569c0 tx=0x7fa13c001560 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:25.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.556+0000 7fa14d7fa640 1 -- 172.21.15.27:0/2774183810 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fa144053ee0 con 0x7fa140006f10 2023-08-11T15:53:25.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.754+0000 7fa156d96640 1 -- 172.21.15.27:0/2774183810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa15015a150 con 0x7fa140006f10 2023-08-11T15:53:25.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.757+0000 7fa14d7fa640 1 -- 172.21.15.27:0/2774183810 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18500 (secure 0 0 0) 0x7fa144057500 con 0x7fa140006f10 2023-08-11T15:53:25.759 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:25.759 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":43,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:53:10.738370+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:53:06.898951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-11T15:53:08.258996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:53:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 -- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa13406a080 msgr2=0x7fa13406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa13406a080 0x7fa13406c540 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa1501569c0 tx=0x7fa13c001560 comp rx=0 tx=0).stop 2023-08-11T15:53:25.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 -- 172.21.15.27:0/2774183810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 msgr2=0x7fa150155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:25.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 0x7fa150155910 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa1440108c0 tx=0x7fa144010d80 comp rx=0 tx=0).stop 2023-08-11T15:53:25.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 -- 172.21.15.27:0/2774183810 shutdown_connections 2023-08-11T15:53:25.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa13406a080 0x7fa13406c540 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa150155260 0x7fa1500b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1500aeed0 0x7fa1500af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 --2- 172.21.15.27:0/2774183810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa140006f10 0x7fa150155910 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 -- 172.21.15.27:0/2774183810 >> 172.21.15.27:0/2774183810 conn(0x7fa1500afb70 msgr2=0x7fa1500ca9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 -- 172.21.15.27:0/2774183810 shutdown_connections 2023-08-11T15:53:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:25.760+0000 7fa156d96640 1 -- 172.21.15.27:0/2774183810 wait complete. 2023-08-11T15:53:25.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:53:06.898951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:53:08.258996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T15:53:25.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:53:26.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.013+0000 7f63605e1640 1 Processor -- start 2023-08-11T15:53:26.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.013+0000 7f63605e1640 1 -- start start 2023-08-11T15:53:26.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.015+0000 7f63605e1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63581609c0 0x7f63580caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.015+0000 7f63605e1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6358165220 0x7f63580cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.015+0000 7f63605e1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6358160da0 con 0x7f6358161020 2023-08-11T15:53:26.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.015+0000 7f63605e1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6358164530 con 0x7f63581609c0 2023-08-11T15:53:26.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.015+0000 7f63605e1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6358161d50 con 0x7f6358165220 2023-08-11T15:53:26.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.016+0000 7f635eb7e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6358161020 0x7f6358161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35260/0 (socket says 172.21.15.27:35260) 2023-08-11T15:53:26.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.016+0000 7f635eb7e640 1 -- 172.21.15.27:0/1429745345 learned_addr learned my addr 172.21.15.27:0/1429745345 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:26.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.016+0000 7f635e37d640 1 --2- 172.21.15.27:0/1429745345 >> v2:172.21.15.27:3300/0 conn(0x7f63581609c0 0x7f63580caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.016+0000 7f635db7c640 1 --2- 172.21.15.27:0/1429745345 >> v2:172.21.15.119:3300/0 conn(0x7f6358165220 0x7f63580cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.016+0000 7f635d37b640 1 -- 172.21.15.27:0/1429745345 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476235261 0 0) 0x7f6358160da0 con 0x7f6358161020 2023-08-11T15:53:26.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.016+0000 7f635e37d640 1 -- 172.21.15.27:0/1429745345 >> v2:172.21.15.119:3300/0 conn(0x7f6358165220 msgr2=0x7f63580cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:26.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.016+0000 7f635e37d640 1 --2- 172.21.15.27:0/1429745345 >> v2:172.21.15.119:3300/0 conn(0x7f6358165220 0x7f63580cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.016+0000 7f635e37d640 1 -- 172.21.15.27:0/1429745345 >> v1:172.21.15.27:6789/0 conn(0x7f6358161020 legacy=0x7f6358161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.016+0000 7f635e37d640 1 -- 172.21.15.27:0/1429745345 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358164740 con 0x7f63581609c0 2023-08-11T15:53:26.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.017+0000 7f635e37d640 1 --2- 172.21.15.27:0/1429745345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 0x7f63580caff0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f63580c1bc0 tx=0x7f635402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6563aeb3d1380aa1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:26.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.018+0000 7f635d37b640 1 -- 172.21.15.27:0/1429745345 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354031830 con 0x7f63581609c0 2023-08-11T15:53:26.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.018+0000 7f635d37b640 1 -- 172.21.15.27:0/1429745345 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6354031990 con 0x7f63581609c0 2023-08-11T15:53:26.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.018+0000 7f635d37b640 1 -- 172.21.15.27:0/1429745345 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354031d20 con 0x7f63581609c0 2023-08-11T15:53:26.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.018+0000 7f63605e1640 1 -- 172.21.15.27:0/1429745345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 msgr2=0x7f63580caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.018+0000 7f63605e1640 1 --2- 172.21.15.27:0/1429745345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 0x7f63580caff0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f63580c1bc0 tx=0x7f635402b8d0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.019+0000 7f63605e1640 1 -- 172.21.15.27:0/1429745345 shutdown_connections 2023-08-11T15:53:26.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.019+0000 7f63605e1640 1 --2- 172.21.15.27:0/1429745345 >> v2:172.21.15.119:3300/0 conn(0x7f6358165220 0x7f63580cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.019+0000 7f63605e1640 1 --2- 172.21.15.27:0/1429745345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 0x7f63580caff0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.019+0000 7f63605e1640 1 -- 172.21.15.27:0/1429745345 >> 172.21.15.27:0/1429745345 conn(0x7f6358069bf0 msgr2=0x7f635806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:26.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.019+0000 7f63605e1640 1 -- 172.21.15.27:0/1429745345 shutdown_connections 2023-08-11T15:53:26.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.019+0000 7f63605e1640 1 -- 172.21.15.27:0/1429745345 wait complete. 2023-08-11T15:53:26.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.021+0000 7f63605e1640 1 Processor -- start 2023-08-11T15:53:26.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.021+0000 7f63605e1640 1 -- start start 2023-08-11T15:53:26.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.022+0000 7f63605e1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 0x7f635815a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.022+0000 7f63605e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6358161020 0x7f635815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.022+0000 7f63605e1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6358165220 0x7f63580b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.022+0000 7f63605e1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635815d0b0 con 0x7f63581609c0 2023-08-11T15:53:26.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.022+0000 7f63605e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635815d390 con 0x7f6358161020 2023-08-11T15:53:26.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.022+0000 7f63605e1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635815d670 con 0x7f6358165220 2023-08-11T15:53:26.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.022+0000 7f635e37d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 0x7f635815a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.022+0000 7f635e37d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 0x7f635815a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54216/0 (socket says 172.21.15.27:54216) 2023-08-11T15:53:26.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.022+0000 7f635e37d640 1 -- 172.21.15.27:0/3241671069 learned_addr learned my addr 172.21.15.27:0/3241671069 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:26.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f635e37d640 1 -- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6358165220 msgr2=0x7f63580b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f635e37d640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6358165220 0x7f63580b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f635e37d640 1 -- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6358161020 msgr2=0x7f635815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f635e37d640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6358161020 0x7f635815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f635e37d640 1 -- 172.21.15.27:0/3241671069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354009400 con 0x7f63581609c0 2023-08-11T15:53:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f635e37d640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 0x7f635815a7b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6354009890 tx=0x7f63540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f634effd640 1 -- 172.21.15.27:0/3241671069 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635403a6f0 con 0x7f63581609c0 2023-08-11T15:53:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f634effd640 1 -- 172.21.15.27:0/3241671069 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635403a870 con 0x7f63581609c0 2023-08-11T15:53:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f634effd640 1 -- 172.21.15.27:0/3241671069 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635403ac20 con 0x7f63581609c0 2023-08-11T15:53:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f63605e1640 1 -- 172.21.15.27:0/3241671069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63580aeca0 con 0x7f63581609c0 2023-08-11T15:53:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.023+0000 7f63605e1640 1 -- 172.21.15.27:0/3241671069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635815cd30 con 0x7f63581609c0 2023-08-11T15:53:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.024+0000 7f63605e1640 1 -- 172.21.15.27:0/3241671069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635804e420 con 0x7f63581609c0 2023-08-11T15:53:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.025+0000 7f634effd640 1 -- 172.21.15.27:0/3241671069 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6354043410 con 0x7f63581609c0 2023-08-11T15:53:26.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.026+0000 7f634effd640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f633c06a080 0x7f633c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.026+0000 7f634effd640 1 -- 172.21.15.27:0/3241671069 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f63540aa2a0 con 0x7f63581609c0 2023-08-11T15:53:26.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.027+0000 7f635db7c640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f633c06a080 0x7f633c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.028+0000 7f635db7c640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f633c06a080 0x7f633c06c540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6348006a90 tx=0x7f6348006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.031+0000 7f634effd640 1 -- 172.21.15.27:0/3241671069 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6354075840 con 0x7f63581609c0 2023-08-11T15:53:26.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.223+0000 7f63605e1640 1 -- 172.21.15.27:0/3241671069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f635806c020 con 0x7f63581609c0 2023-08-11T15:53:26.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.224+0000 7f634effd640 1 -- 172.21.15.27:0/3241671069 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6354078e60 con 0x7f63581609c0 2023-08-11T15:53:26.226 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:53:26.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 -- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f633c06a080 msgr2=0x7f633c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f633c06a080 0x7f633c06c540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6348006a90 tx=0x7f6348006a20 comp rx=0 tx=0).stop 2023-08-11T15:53:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 -- 172.21.15.27:0/3241671069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 msgr2=0x7f635815a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 0x7f635815a7b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6354009890 tx=0x7f63540046c0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 -- 172.21.15.27:0/3241671069 shutdown_connections 2023-08-11T15:53:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f633c06a080 0x7f633c06c540 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6358165220 0x7f63580b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6358161020 0x7f635815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 --2- 172.21.15.27:0/3241671069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63581609c0 0x7f635815a7b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 -- 172.21.15.27:0/3241671069 >> 172.21.15.27:0/3241671069 conn(0x7f6358069bf0 msgr2=0x7f6358072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 -- 172.21.15.27:0/3241671069 shutdown_connections 2023-08-11T15:53:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.227+0000 7f63605e1640 1 -- 172.21.15.27:0/3241671069 wait complete. 2023-08-11T15:53:26.249 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:53:26.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.491+0000 7f94428e5640 1 Processor -- start 2023-08-11T15:53:26.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.492+0000 7f94428e5640 1 -- start start 2023-08-11T15:53:26.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.493+0000 7f94428e5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f943c13ce80 0x7f943c13f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.493+0000 7f94428e5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c13d260 0x7f943c13fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.493+0000 7f94428e5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c0cd470 con 0x7f943c0cc720 2023-08-11T15:53:26.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.493+0000 7f94428e5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c140670 con 0x7f943c13ce80 2023-08-11T15:53:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.493+0000 7f94428e5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0cd750 con 0x7f943c13d260 2023-08-11T15:53:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.494+0000 7f94418e3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f943c13ce80 0x7f943c13f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.494+0000 7f94420e4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f943c0cc720 0x7f943c0ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35266/0 (socket says 172.21.15.27:35266) 2023-08-11T15:53:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.494+0000 7f94420e4640 1 -- 172.21.15.27:0/3440702163 learned_addr learned my addr 172.21.15.27:0/3440702163 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.494+0000 7f94410e2640 1 --2- 172.21.15.27:0/3440702163 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c13d260 0x7f943c13fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.494+0000 7f94418e3640 1 -- 172.21.15.27:0/3440702163 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c13d260 msgr2=0x7f943c13fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.494+0000 7f94418e3640 1 --2- 172.21.15.27:0/3440702163 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c13d260 0x7f943c13fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.494+0000 7f94418e3640 1 -- 172.21.15.27:0/3440702163 >> v1:172.21.15.27:6789/0 conn(0x7f943c0cc720 legacy=0x7f943c0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.494+0000 7f94418e3640 1 -- 172.21.15.27:0/3440702163 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c0ad7d0 con 0x7f943c13ce80 2023-08-11T15:53:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.494+0000 7f94408e1640 1 -- 172.21.15.27:0/3440702163 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312244599 0 0) 0x7f943c0cd470 con 0x7f943c0cc720 2023-08-11T15:53:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.494+0000 7f94410e2640 1 --2- 172.21.15.27:0/3440702163 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c13d260 0x7f943c13fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:26.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.495+0000 7f94418e3640 1 --2- 172.21.15.27:0/3440702163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13ce80 0x7f943c13f810 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f94380097a0 tx=0x7f943802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45760aaa7b95a2a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.496+0000 7f94408e1640 1 -- 172.21.15.27:0/3440702163 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943802d040 con 0x7f943c13ce80 2023-08-11T15:53:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.496+0000 7f94408e1640 1 -- 172.21.15.27:0/3440702163 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9438031810 con 0x7f943c13ce80 2023-08-11T15:53:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.496+0000 7f94408e1640 1 -- 172.21.15.27:0/3440702163 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438031ba0 con 0x7f943c13ce80 2023-08-11T15:53:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.496+0000 7f94428e5640 1 -- 172.21.15.27:0/3440702163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13ce80 msgr2=0x7f943c13f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.496+0000 7f94428e5640 1 --2- 172.21.15.27:0/3440702163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13ce80 0x7f943c13f810 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f94380097a0 tx=0x7f943802b8b0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.496+0000 7f94428e5640 1 -- 172.21.15.27:0/3440702163 shutdown_connections 2023-08-11T15:53:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.496+0000 7f94428e5640 1 --2- 172.21.15.27:0/3440702163 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c13d260 0x7f943c13fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.496+0000 7f94428e5640 1 --2- 172.21.15.27:0/3440702163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13ce80 0x7f943c13f810 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.496+0000 7f94428e5640 1 -- 172.21.15.27:0/3440702163 >> 172.21.15.27:0/3440702163 conn(0x7f943c069800 msgr2=0x7f943c069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.497+0000 7f94428e5640 1 -- 172.21.15.27:0/3440702163 shutdown_connections 2023-08-11T15:53:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.497+0000 7f94428e5640 1 -- 172.21.15.27:0/3440702163 wait complete. 2023-08-11T15:53:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.498+0000 7f94428e5640 1 Processor -- start 2023-08-11T15:53:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.498+0000 7f94428e5640 1 -- start start 2023-08-11T15:53:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.499+0000 7f94428e5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c0cc720 0x7f943c13b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.499+0000 7f94428e5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f943c13ce80 0x7f943c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.499+0000 7f94428e5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13d260 0x7f943c0b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94428e5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c15a790 con 0x7f943c13d260 2023-08-11T15:53:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94428e5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c15aa30 con 0x7f943c13ce80 2023-08-11T15:53:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94428e5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c15ad10 con 0x7f943c0cc720 2023-08-11T15:53:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94420e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13d260 0x7f943c0b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94420e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13d260 0x7f943c0b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54242/0 (socket says 172.21.15.27:54242) 2023-08-11T15:53:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94420e4640 1 -- 172.21.15.27:0/2398058268 learned_addr learned my addr 172.21.15.27:0/2398058268 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:26.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94418e3640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c0cc720 0x7f943c13b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94420e4640 1 -- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c0cc720 msgr2=0x7f943c13b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94420e4640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c0cc720 0x7f943c13b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94420e4640 1 -- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f943c13ce80 msgr2=0x7f943c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94420e4640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f943c13ce80 0x7f943c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.500+0000 7f94420e4640 1 -- 172.21.15.27:0/2398058268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9438009400 con 0x7f943c13d260 2023-08-11T15:53:26.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.501+0000 7f94420e4640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13d260 0x7f943c0b2590 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f943400c7e0 tx=0x7f943400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:26.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.501+0000 7f94327fc640 1 -- 172.21.15.27:0/2398058268 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400a640 con 0x7f943c13d260 2023-08-11T15:53:26.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.501+0000 7f94327fc640 1 -- 172.21.15.27:0/2398058268 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943400a7a0 con 0x7f943c13d260 2023-08-11T15:53:26.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.501+0000 7f94327fc640 1 -- 172.21.15.27:0/2398058268 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400ab30 con 0x7f943c13d260 2023-08-11T15:53:26.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.501+0000 7f94428e5640 1 -- 172.21.15.27:0/2398058268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c15aff0 con 0x7f943c13d260 2023-08-11T15:53:26.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.501+0000 7f94428e5640 1 -- 172.21.15.27:0/2398058268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c13ba10 con 0x7f943c13d260 2023-08-11T15:53:26.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.502+0000 7f94428e5640 1 -- 172.21.15.27:0/2398058268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c15b1e0 con 0x7f943c13d260 2023-08-11T15:53:26.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.503+0000 7f94327fc640 1 -- 172.21.15.27:0/2398058268 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f943400ac90 con 0x7f943c13d260 2023-08-11T15:53:26.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.504+0000 7f94327fc640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f942006e460 0x7f9420070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.504+0000 7f94327fc640 1 -- 172.21.15.27:0/2398058268 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9434087c40 con 0x7f943c13d260 2023-08-11T15:53:26.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.505+0000 7f94418e3640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f942006e460 0x7f9420070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.506+0000 7f94418e3640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f942006e460 0x7f9420070920 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9438036000 tx=0x7f943803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.509+0000 7f94327fc640 1 -- 172.21.15.27:0/2398058268 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f9434053150 con 0x7f943c13d260 2023-08-11T15:53:26.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.706+0000 7f94428e5640 1 -- 172.21.15.27:0/2398058268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f943c066e80 con 0x7f943c13d260 2023-08-11T15:53:26.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.707+0000 7f94327fc640 1 -- 172.21.15.27:0/2398058268 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7f94340180a0 con 0x7f943c13d260 2023-08-11T15:53:26.708 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:53:26.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.709+0000 7f94428e5640 1 -- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f942006e460 msgr2=0x7f9420070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.709+0000 7f94428e5640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f942006e460 0x7f9420070920 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9438036000 tx=0x7f943803c040 comp rx=0 tx=0).stop 2023-08-11T15:53:26.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.709+0000 7f94428e5640 1 -- 172.21.15.27:0/2398058268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13d260 msgr2=0x7f943c0b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.709+0000 7f94428e5640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13d260 0x7f943c0b2590 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f943400c7e0 tx=0x7f943400cca0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.710+0000 7f94428e5640 1 -- 172.21.15.27:0/2398058268 shutdown_connections 2023-08-11T15:53:26.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.710+0000 7f94428e5640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f942006e460 0x7f9420070920 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.710+0000 7f94428e5640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c13d260 0x7f943c0b2590 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.710+0000 7f94428e5640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f943c13ce80 0x7f943c0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.710+0000 7f94428e5640 1 --2- 172.21.15.27:0/2398058268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f943c0cc720 0x7f943c13b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.710+0000 7f94428e5640 1 -- 172.21.15.27:0/2398058268 >> 172.21.15.27:0/2398058268 conn(0x7f943c069800 msgr2=0x7f943c153ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.710+0000 7f94428e5640 1 -- 172.21.15.27:0/2398058268 shutdown_connections 2023-08-11T15:53:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.710+0000 7f94428e5640 1 -- 172.21.15.27:0/2398058268 wait complete. 2023-08-11T15:53:26.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:53:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.971+0000 7f7712e0d640 1 Processor -- start 2023-08-11T15:53:26.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.971+0000 7f7712e0d640 1 -- start start 2023-08-11T15:53:26.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.973+0000 7f7712e0d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f770c0b3090 0x7f770c0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.973+0000 7f7712e0d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770c0af250 con 0x7f770c0b38f0 2023-08-11T15:53:26.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.973+0000 7f7712e0d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770c0af510 con 0x7f770c161210 2023-08-11T15:53:26.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.973+0000 7f7712e0d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770c0b0190 con 0x7f770c0b3090 2023-08-11T15:53:26.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.973+0000 7f7710ba9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f770c0b38f0 0x7f770c160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35282/0 (socket says 172.21.15.27:35282) 2023-08-11T15:53:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.973+0000 7f7710ba9640 1 -- 172.21.15.27:0/705737835 learned_addr learned my addr 172.21.15.27:0/705737835 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.973+0000 7f770bfff640 1 --2- 172.21.15.27:0/705737835 >> v2:172.21.15.27:3300/0 conn(0x7f770c0b3090 0x7f770c0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.974+0000 7f770b7fe640 1 -- 172.21.15.27:0/705737835 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795892114 0 0) 0x7f770c0af250 con 0x7f770c0b38f0 2023-08-11T15:53:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.974+0000 7f770b7fe640 1 -- 172.21.15.27:0/705737835 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76f8003520 con 0x7f770c0b38f0 2023-08-11T15:53:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.974+0000 7f770bfff640 1 -- 172.21.15.27:0/705737835 >> v1:172.21.15.119:6789/0 conn(0x7f770c161210 legacy=0x7f770c0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.974+0000 7f770bfff640 1 -- 172.21.15.27:0/705737835 >> v1:172.21.15.27:6789/0 conn(0x7f770c0b38f0 legacy=0x7f770c160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.974+0000 7f770bfff640 1 -- 172.21.15.27:0/705737835 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c0af7f0 con 0x7f770c0b3090 2023-08-11T15:53:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.975+0000 7f770bfff640 1 --2- 172.21.15.27:0/705737835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 0x7f770c0d30e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f76f40097a0 tx=0x7f76f402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bb1053956fd9e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.976+0000 7f770b7fe640 1 -- 172.21.15.27:0/705737835 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4030610 con 0x7f770c0b3090 2023-08-11T15:53:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.976+0000 7f770b7fe640 1 -- 172.21.15.27:0/705737835 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f4030770 con 0x7f770c0b3090 2023-08-11T15:53:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.976+0000 7f770b7fe640 1 -- 172.21.15.27:0/705737835 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4030b00 con 0x7f770c0b3090 2023-08-11T15:53:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.976+0000 7f7712e0d640 1 -- 172.21.15.27:0/705737835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 msgr2=0x7f770c0d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.976+0000 7f7712e0d640 1 --2- 172.21.15.27:0/705737835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 0x7f770c0d30e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f76f40097a0 tx=0x7f76f402b960 comp rx=0 tx=0).stop 2023-08-11T15:53:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.976+0000 7f7712e0d640 1 -- 172.21.15.27:0/705737835 shutdown_connections 2023-08-11T15:53:26.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.976+0000 7f7712e0d640 1 --2- 172.21.15.27:0/705737835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 0x7f770c0d30e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.976+0000 7f7712e0d640 1 -- 172.21.15.27:0/705737835 >> 172.21.15.27:0/705737835 conn(0x7f770c09b960 msgr2=0x7f770c0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:26.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.976+0000 7f7712e0d640 1 -- 172.21.15.27:0/705737835 shutdown_connections 2023-08-11T15:53:26.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.976+0000 7f7712e0d640 1 -- 172.21.15.27:0/705737835 wait complete. 2023-08-11T15:53:26.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.978+0000 7f7712e0d640 1 Processor -- start 2023-08-11T15:53:26.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.978+0000 7f7712e0d640 1 -- start start 2023-08-11T15:53:26.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.979+0000 7f7712e0d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 0x7f770c15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.979+0000 7f7712e0d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f770c0b38f0 0x7f770c1f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.979+0000 7f7712e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f770c161210 0x7f770c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.979+0000 7f7712e0d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0b4c80 con 0x7f770c0b3090 2023-08-11T15:53:26.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.979+0000 7f7712e0d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0b4070 con 0x7f770c161210 2023-08-11T15:53:26.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.979+0000 7f7712e0d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0b4350 con 0x7f770c0b38f0 2023-08-11T15:53:26.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f7710ba9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 0x7f770c15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f7710ba9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 0x7f770c15cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54260/0 (socket says 172.21.15.27:54260) 2023-08-11T15:53:26.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f7710ba9640 1 -- 172.21.15.27:0/2204089440 learned_addr learned my addr 172.21.15.27:0/2204089440 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:26.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f770bfff640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f770c0b38f0 0x7f770c1f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f7710ba9640 1 -- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f770c0b38f0 msgr2=0x7f770c1f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f7710ba9640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f770c0b38f0 0x7f770c1f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f7710ba9640 1 -- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f770c161210 msgr2=0x7f770c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:26.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f77113aa640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f770c161210 0x7f770c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f7710ba9640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f770c161210 0x7f770c0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:26.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f7710ba9640 1 -- 172.21.15.27:0/2204089440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f4009400 con 0x7f770c0b3090 2023-08-11T15:53:26.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.980+0000 7f77113aa640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f770c161210 0x7f770c0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:26.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.981+0000 7f7710ba9640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 0x7f770c15cd50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0015e0 tx=0x7f76fc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:26.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.981+0000 7f77097fa640 1 -- 172.21.15.27:0/2204089440 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc00a640 con 0x7f770c0b3090 2023-08-11T15:53:26.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.981+0000 7f77097fa640 1 -- 172.21.15.27:0/2204089440 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76fc00a7a0 con 0x7f770c0b3090 2023-08-11T15:53:26.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.981+0000 7f77097fa640 1 -- 172.21.15.27:0/2204089440 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc00ab90 con 0x7f770c0b3090 2023-08-11T15:53:26.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.981+0000 7f7712e0d640 1 -- 172.21.15.27:0/2204089440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c0b4630 con 0x7f770c0b3090 2023-08-11T15:53:26.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.981+0000 7f7712e0d640 1 -- 172.21.15.27:0/2204089440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c15d290 con 0x7f770c0b3090 2023-08-11T15:53:26.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.983+0000 7f7712e0d640 1 -- 172.21.15.27:0/2204089440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c04e420 con 0x7f770c0b3090 2023-08-11T15:53:26.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.983+0000 7f77097fa640 1 -- 172.21.15.27:0/2204089440 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f76fc00acf0 con 0x7f770c0b3090 2023-08-11T15:53:26.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.984+0000 7f77097fa640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76ec06a050 0x7f76ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:26.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.984+0000 7f77097fa640 1 -- 172.21.15.27:0/2204089440 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f76fc0880b0 con 0x7f770c0b3090 2023-08-11T15:53:26.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.985+0000 7f770bfff640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76ec06a050 0x7f76ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:26.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.986+0000 7f770bfff640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76ec06a050 0x7f76ec06c510 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f76f4036000 tx=0x7f76f4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:26.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:26.988+0000 7f77097fa640 1 -- 172.21.15.27:0/2204089440 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f76fc053720 con 0x7f770c0b3090 2023-08-11T15:53:27.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.184+0000 7f7712e0d640 1 -- 172.21.15.27:0/2204089440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f770c0b86b0 con 0x7f770c0b3090 2023-08-11T15:53:27.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.185+0000 7f77097fa640 1 -- 172.21.15.27:0/2204089440 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76fc056d40 con 0x7f770c0b3090 2023-08-11T15:53:27.187 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:53:27.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.187+0000 7f7712e0d640 1 -- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76ec06a050 msgr2=0x7f76ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76ec06a050 0x7f76ec06c510 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f76f4036000 tx=0x7f76f4008030 comp rx=0 tx=0).stop 2023-08-11T15:53:27.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 -- 172.21.15.27:0/2204089440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 msgr2=0x7f770c15cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 0x7f770c15cd50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0015e0 tx=0x7f76fc00ccf0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 -- 172.21.15.27:0/2204089440 shutdown_connections 2023-08-11T15:53:27.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76ec06a050 0x7f76ec06c510 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f770c0b38f0 0x7f770c1f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f770c161210 0x7f770c0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 --2- 172.21.15.27:0/2204089440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770c0b3090 0x7f770c15cd50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 -- 172.21.15.27:0/2204089440 >> 172.21.15.27:0/2204089440 conn(0x7f770c09b960 msgr2=0x7f770c0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:27.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 -- 172.21.15.27:0/2204089440 shutdown_connections 2023-08-11T15:53:27.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.188+0000 7f7712e0d640 1 -- 172.21.15.27:0/2204089440 wait complete. 2023-08-11T15:53:27.209 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:53:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.455+0000 7f7cb88aa640 1 Processor -- start 2023-08-11T15:53:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.455+0000 7f7cb88aa640 1 -- start start 2023-08-11T15:53:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.456+0000 7f7cb88aa640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7cb415c5b0 0x7f7cb40d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.456+0000 7f7cb88aa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7cb40d0790 0x7f7cb40cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.456+0000 7f7cb88aa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40d0dc0 con 0x7f7cb40cf140 2023-08-11T15:53:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.456+0000 7f7cb88aa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40cf5b0 con 0x7f7cb40d0790 2023-08-11T15:53:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.456+0000 7f7cb88aa640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40d10a0 con 0x7f7cb415c5b0 2023-08-11T15:53:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.456+0000 7f7cb3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7cb40cf140 0x7f7cb40b3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35292/0 (socket says 172.21.15.27:35292) 2023-08-11T15:53:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.456+0000 7f7cb3fff640 1 -- 172.21.15.27:0/3079693929 learned_addr learned my addr 172.21.15.27:0/3079693929 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:27.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.457+0000 7f7caaffd640 1 --2- 172.21.15.27:0/3079693929 >> v2:172.21.15.27:3300/0 conn(0x7f7cb40d0790 0x7f7cb40cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.457+0000 7f7cb37fe640 1 --2- 172.21.15.27:0/3079693929 >> v2:172.21.15.119:3300/0 conn(0x7f7cb415c5b0 0x7f7cb40d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.457+0000 7f7cb2ffd640 1 -- 172.21.15.27:0/3079693929 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128127424 0 0) 0x7f7cb40d0dc0 con 0x7f7cb40cf140 2023-08-11T15:53:27.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.457+0000 7f7cb2ffd640 1 -- 172.21.15.27:0/3079693929 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ca0003520 con 0x7f7cb40cf140 2023-08-11T15:53:27.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.457+0000 7f7caaffd640 1 -- 172.21.15.27:0/3079693929 >> v2:172.21.15.119:3300/0 conn(0x7f7cb415c5b0 msgr2=0x7f7cb40d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:27.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.457+0000 7f7caaffd640 1 --2- 172.21.15.27:0/3079693929 >> v2:172.21.15.119:3300/0 conn(0x7f7cb415c5b0 0x7f7cb40d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.457+0000 7f7caaffd640 1 -- 172.21.15.27:0/3079693929 >> v1:172.21.15.27:6789/0 conn(0x7f7cb40cf140 legacy=0x7f7cb40b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.457+0000 7f7caaffd640 1 -- 172.21.15.27:0/3079693929 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40d1380 con 0x7f7cb40d0790 2023-08-11T15:53:27.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.457+0000 7f7cb37fe640 1 --2- 172.21.15.27:0/3079693929 >> v2:172.21.15.119:3300/0 conn(0x7f7cb415c5b0 0x7f7cb40d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:27.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.458+0000 7f7caaffd640 1 --2- 172.21.15.27:0/3079693929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb40d0790 0x7f7cb40cec00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4071f10 tx=0x7f7c9c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00a9d095171099b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb2ffd640 1 -- 172.21.15.27:0/3079693929 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c030610 con 0x7f7cb40d0790 2023-08-11T15:53:27.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb2ffd640 1 -- 172.21.15.27:0/3079693929 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c030770 con 0x7f7cb40d0790 2023-08-11T15:53:27.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb2ffd640 1 -- 172.21.15.27:0/3079693929 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c030ae0 con 0x7f7cb40d0790 2023-08-11T15:53:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb88aa640 1 -- 172.21.15.27:0/3079693929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb40d0790 msgr2=0x7f7cb40cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb88aa640 1 --2- 172.21.15.27:0/3079693929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb40d0790 0x7f7cb40cec00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4071f10 tx=0x7f7c9c02b960 comp rx=0 tx=0).stop 2023-08-11T15:53:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb88aa640 1 -- 172.21.15.27:0/3079693929 shutdown_connections 2023-08-11T15:53:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb88aa640 1 --2- 172.21.15.27:0/3079693929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb40d0790 0x7f7cb40cec00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb88aa640 1 --2- 172.21.15.27:0/3079693929 >> v2:172.21.15.119:3300/0 conn(0x7f7cb415c5b0 0x7f7cb40d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb88aa640 1 -- 172.21.15.27:0/3079693929 >> 172.21.15.27:0/3079693929 conn(0x7f7cb406a270 msgr2=0x7f7cb406a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:27.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb88aa640 1 -- 172.21.15.27:0/3079693929 shutdown_connections 2023-08-11T15:53:27.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.459+0000 7f7cb88aa640 1 -- 172.21.15.27:0/3079693929 wait complete. 2023-08-11T15:53:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.461+0000 7f7cb88aa640 1 Processor -- start 2023-08-11T15:53:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.461+0000 7f7cb88aa640 1 -- start start 2023-08-11T15:53:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.462+0000 7f7cb88aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb40cf140 0x7f7cb4154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.462+0000 7f7cb88aa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cb40d0790 0x7f7cb40c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.462+0000 7f7cb88aa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb415c5b0 0x7f7cb4156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.462+0000 7f7cb88aa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415d9e0 con 0x7f7cb415c5b0 2023-08-11T15:53:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.462+0000 7f7cb88aa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40c5ba0 con 0x7f7cb40cf140 2023-08-11T15:53:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.462+0000 7f7cb88aa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415dbf0 con 0x7f7cb40d0790 2023-08-11T15:53:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb415c5b0 0x7f7cb4156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb415c5b0 0x7f7cb4156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54284/0 (socket says 172.21.15.27:54284) 2023-08-11T15:53:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7cb3fff640 1 -- 172.21.15.27:0/418534962 learned_addr learned my addr 172.21.15.27:0/418534962 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7caaffd640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cb40d0790 0x7f7cb40c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7cb37fe640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb40cf140 0x7f7cb4154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7cb3fff640 1 -- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cb40d0790 msgr2=0x7f7cb40c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7cb3fff640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cb40d0790 0x7f7cb40c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7cb3fff640 1 -- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb40cf140 msgr2=0x7f7cb4154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7cb3fff640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb40cf140 0x7f7cb4154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7cb3fff640 1 -- 172.21.15.27:0/418534962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c009400 con 0x7f7cb415c5b0 2023-08-11T15:53:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7caaffd640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cb40d0790 0x7f7cb40c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.463+0000 7f7cb37fe640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb40cf140 0x7f7cb4154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.464+0000 7f7cb3fff640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb415c5b0 0x7f7cb4156920 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00cb40 tx=0x7f7cac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.464+0000 7f7cb0ff9640 1 -- 172.21.15.27:0/418534962 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00aca0 con 0x7f7cb415c5b0 2023-08-11T15:53:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.464+0000 7f7cb0ff9640 1 -- 172.21.15.27:0/418534962 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac00ae00 con 0x7f7cb415c5b0 2023-08-11T15:53:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.464+0000 7f7cb0ff9640 1 -- 172.21.15.27:0/418534962 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac012620 con 0x7f7cb415c5b0 2023-08-11T15:53:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.464+0000 7f7cb88aa640 1 -- 172.21.15.27:0/418534962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb415ded0 con 0x7f7cb415c5b0 2023-08-11T15:53:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.464+0000 7f7cb88aa640 1 -- 172.21.15.27:0/418534962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb415e490 con 0x7f7cb415c5b0 2023-08-11T15:53:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.465+0000 7f7cb88aa640 1 -- 172.21.15.27:0/418534962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb415e6b0 con 0x7f7cb415c5b0 2023-08-11T15:53:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.466+0000 7f7cb0ff9640 1 -- 172.21.15.27:0/418534962 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7cac007070 con 0x7f7cb415c5b0 2023-08-11T15:53:27.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.467+0000 7f7cb0ff9640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7c9406a050 0x7f7c9406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.467+0000 7f7cb0ff9640 1 -- 172.21.15.27:0/418534962 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7cac0886f0 con 0x7f7cb415c5b0 2023-08-11T15:53:27.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.468+0000 7f7cb37fe640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7c9406a050 0x7f7c9406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.469+0000 7f7cb37fe640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7c9406a050 0x7f7c9406c510 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4158390 tx=0x7f7ca4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.472+0000 7f7cb0ff9640 1 -- 172.21.15.27:0/418534962 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f7cac053ca0 con 0x7f7cb415c5b0 2023-08-11T15:53:27.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.661+0000 7f7cb88aa640 1 -- 172.21.15.27:0/418534962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7cb40cdbf0 con 0x7f7cb415c5b0 2023-08-11T15:53:27.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.662+0000 7f7cb0ff9640 1 -- 172.21.15.27:0/418534962 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7cac0572c0 con 0x7f7cb415c5b0 2023-08-11T15:53:27.664 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:53:27.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 -- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7c9406a050 msgr2=0x7f7c9406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7c9406a050 0x7f7c9406c510 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4158390 tx=0x7f7ca4008040 comp rx=0 tx=0).stop 2023-08-11T15:53:27.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 -- 172.21.15.27:0/418534962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb415c5b0 msgr2=0x7f7cb4156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb415c5b0 0x7f7cb4156920 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00cb40 tx=0x7f7cac00a060 comp rx=0 tx=0).stop 2023-08-11T15:53:27.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 -- 172.21.15.27:0/418534962 shutdown_connections 2023-08-11T15:53:27.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f7c9406a050 0x7f7c9406c510 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cb40d0790 0x7f7cb40c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb415c5b0 0x7f7cb4156920 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 --2- 172.21.15.27:0/418534962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb40cf140 0x7f7cb4154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 -- 172.21.15.27:0/418534962 >> 172.21.15.27:0/418534962 conn(0x7f7cb406a270 msgr2=0x7f7cb41599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 -- 172.21.15.27:0/418534962 shutdown_connections 2023-08-11T15:53:27.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.665+0000 7f7cb88aa640 1 -- 172.21.15.27:0/418534962 wait complete. 2023-08-11T15:53:27.685 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:53:27.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.914+0000 7f76c9a46640 1 Processor -- start 2023-08-11T15:53:27.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.914+0000 7f76c9a46640 1 -- start start 2023-08-11T15:53:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.915+0000 7f76c9a46640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415a3b0 0x7f76c4071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.915+0000 7f76c9a46640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f76c415df00 0x7f76c40b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.915+0000 7f76c9a46640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c40cb520 con 0x7f76c406f6e0 2023-08-11T15:53:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.915+0000 7f76c9a46640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c415eea0 con 0x7f76c415df00 2023-08-11T15:53:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.915+0000 7f76c9a46640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c415a790 con 0x7f76c415a3b0 2023-08-11T15:53:27.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.916+0000 7f76c2ffd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415a3b0 0x7f76c4071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.916+0000 7f76c2ffd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415a3b0 0x7f76c4071690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:44058/0 (socket says 172.21.15.27:44058) 2023-08-11T15:53:27.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.916+0000 7f76c37fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f76c406f6e0 0x7f76c40c8f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:37880/0 (socket says 172.21.15.27:37880) 2023-08-11T15:53:27.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.916+0000 7f76c2ffd640 1 -- 172.21.15.27:0/2759518812 learned_addr learned my addr 172.21.15.27:0/2759518812 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:27.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.916+0000 7f76bbfff640 1 --2- 172.21.15.27:0/2759518812 >> v2:172.21.15.119:3300/0 conn(0x7f76c415df00 0x7f76c40b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.917+0000 7f76bbfff640 1 -- 172.21.15.27:0/2759518812 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415a3b0 msgr2=0x7f76c4071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:27.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.917+0000 7f76bbfff640 1 --2- 172.21.15.27:0/2759518812 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415a3b0 0x7f76c4071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.917+0000 7f76bbfff640 1 -- 172.21.15.27:0/2759518812 >> v1:172.21.15.119:6789/0 conn(0x7f76c406f6e0 legacy=0x7f76c40c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.917+0000 7f76bbfff640 1 -- 172.21.15.27:0/2759518812 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c415e360 con 0x7f76c415df00 2023-08-11T15:53:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.919+0000 7f76bbfff640 1 --2- 172.21.15.27:0/2759518812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c415df00 0x7f76c40b0ee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0097a0 tx=0x7f76ac02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=557f7cd0f3ba65b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.919+0000 7f76c27fc640 1 -- 172.21.15.27:0/2759518812 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac02d040 con 0x7f76c415df00 2023-08-11T15:53:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.919+0000 7f76c27fc640 1 -- 172.21.15.27:0/2759518812 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac031810 con 0x7f76c415df00 2023-08-11T15:53:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.919+0000 7f76c27fc640 1 -- 172.21.15.27:0/2759518812 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac031ba0 con 0x7f76c415df00 2023-08-11T15:53:27.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.920+0000 7f76c9a46640 1 -- 172.21.15.27:0/2759518812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c415df00 msgr2=0x7f76c40b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.920+0000 7f76c9a46640 1 --2- 172.21.15.27:0/2759518812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c415df00 0x7f76c40b0ee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0097a0 tx=0x7f76ac02b8b0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.920+0000 7f76c9a46640 1 -- 172.21.15.27:0/2759518812 shutdown_connections 2023-08-11T15:53:27.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.920+0000 7f76c9a46640 1 --2- 172.21.15.27:0/2759518812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c415df00 0x7f76c40b0ee0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.920+0000 7f76c9a46640 1 --2- 172.21.15.27:0/2759518812 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415a3b0 0x7f76c4071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.920+0000 7f76c9a46640 1 -- 172.21.15.27:0/2759518812 >> 172.21.15.27:0/2759518812 conn(0x7f76c4069e90 msgr2=0x7f76c40afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:27.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.920+0000 7f76c9a46640 1 -- 172.21.15.27:0/2759518812 shutdown_connections 2023-08-11T15:53:27.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.920+0000 7f76c9a46640 1 -- 172.21.15.27:0/2759518812 wait complete. 2023-08-11T15:53:27.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.922+0000 7f76c9a46640 1 Processor -- start 2023-08-11T15:53:27.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.923+0000 7f76c9a46640 1 -- start start 2023-08-11T15:53:27.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.924+0000 7f76c9a46640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c406f6e0 0x7f76c40acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.924+0000 7f76c9a46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c415a3b0 0x7f76c40c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.924+0000 7f76c9a46640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415df00 0x7f76c41591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.924+0000 7f76c9a46640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40c23a0 con 0x7f76c406f6e0 2023-08-11T15:53:27.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.924+0000 7f76c9a46640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c415ace0 con 0x7f76c415a3b0 2023-08-11T15:53:27.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.924+0000 7f76c9a46640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c415afc0 con 0x7f76c415df00 2023-08-11T15:53:27.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.924+0000 7f76c2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c406f6e0 0x7f76c40acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.924+0000 7f76c2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c406f6e0 0x7f76c40acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54292/0 (socket says 172.21.15.27:54292) 2023-08-11T15:53:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.924+0000 7f76c2ffd640 1 -- 172.21.15.27:0/3721281600 learned_addr learned my addr 172.21.15.27:0/3721281600 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:27.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c37fe640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415df00 0x7f76c41591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76bbfff640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c415a3b0 0x7f76c40c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c2ffd640 1 -- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415df00 msgr2=0x7f76c41591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c2ffd640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415df00 0x7f76c41591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c2ffd640 1 -- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c415a3b0 msgr2=0x7f76c40c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c2ffd640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c415a3b0 0x7f76c40c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c2ffd640 1 -- 172.21.15.27:0/3721281600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac009400 con 0x7f76c406f6e0 2023-08-11T15:53:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c37fe640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415df00 0x7f76c41591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c2ffd640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c406f6e0 0x7f76c40acb80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009fd0 tx=0x7f76b400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c0ff9640 1 -- 172.21.15.27:0/3721281600 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b400c940 con 0x7f76c406f6e0 2023-08-11T15:53:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c0ff9640 1 -- 172.21.15.27:0/3721281600 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b400caa0 con 0x7f76c406f6e0 2023-08-11T15:53:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.925+0000 7f76c9a46640 1 -- 172.21.15.27:0/3721281600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c415b2a0 con 0x7f76c406f6e0 2023-08-11T15:53:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.926+0000 7f76c0ff9640 1 -- 172.21.15.27:0/3721281600 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b400ce30 con 0x7f76c406f6e0 2023-08-11T15:53:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.926+0000 7f76c9a46640 1 -- 172.21.15.27:0/3721281600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c415a960 con 0x7f76c406f6e0 2023-08-11T15:53:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.927+0000 7f76c9a46640 1 -- 172.21.15.27:0/3721281600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c4073990 con 0x7f76c406f6e0 2023-08-11T15:53:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.928+0000 7f76c0ff9640 1 -- 172.21.15.27:0/3721281600 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f76b4013470 con 0x7f76c406f6e0 2023-08-11T15:53:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.929+0000 7f76c0ff9640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76a406a050 0x7f76a406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.929+0000 7f76c0ff9640 1 -- 172.21.15.27:0/3721281600 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f76b40887f0 con 0x7f76c406f6e0 2023-08-11T15:53:27.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.929+0000 7f76bbfff640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76a406a050 0x7f76a406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.930+0000 7f76bbfff640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76a406a050 0x7f76a406c510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76ac036000 tx=0x7f76ac03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:27.935+0000 7f76c0ff9640 1 -- 172.21.15.27:0/3721281600 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f76b4053e30 con 0x7f76c406f6e0 2023-08-11T15:53:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.143+0000 7f76c9a46640 1 -- 172.21.15.27:0/3721281600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f76c40b07b0 con 0x7f76c406f6e0 2023-08-11T15:53:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.143+0000 7f76c0ff9640 1 -- 172.21.15.27:0/3721281600 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7f76b4057450 con 0x7f76c406f6e0 2023-08-11T15:53:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-08-11T15:53:28.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.146+0000 7f76c9a46640 1 -- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76a406a050 msgr2=0x7f76a406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:28.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.146+0000 7f76c9a46640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76a406a050 0x7f76a406c510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76ac036000 tx=0x7f76ac03c040 comp rx=0 tx=0).stop 2023-08-11T15:53:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.146+0000 7f76c9a46640 1 -- 172.21.15.27:0/3721281600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c406f6e0 msgr2=0x7f76c40acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.146+0000 7f76c9a46640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c406f6e0 0x7f76c40acb80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009fd0 tx=0x7f76b400ed40 comp rx=0 tx=0).stop 2023-08-11T15:53:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.146+0000 7f76c9a46640 1 -- 172.21.15.27:0/3721281600 shutdown_connections 2023-08-11T15:53:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.146+0000 7f76c9a46640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f76a406a050 0x7f76a406c510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.146+0000 7f76c9a46640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76c415df00 0x7f76c41591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.146+0000 7f76c9a46640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c415a3b0 0x7f76c40c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.146+0000 7f76c9a46640 1 --2- 172.21.15.27:0/3721281600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c406f6e0 0x7f76c40acb80 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.146+0000 7f76c9a46640 1 -- 172.21.15.27:0/3721281600 >> 172.21.15.27:0/3721281600 conn(0x7f76c4069e90 msgr2=0x7f76c40cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.147+0000 7f76c9a46640 1 -- 172.21.15.27:0/3721281600 shutdown_connections 2023-08-11T15:53:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.147+0000 7f76c9a46640 1 -- 172.21.15.27:0/3721281600 wait complete. 2023-08-11T15:53:28.169 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi119.front.sepia.ceph.com... 2023-08-11T15:53:28.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:28.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-11T15:53:28.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:ceph-fuse[96551]: fuse finished with error 0 and tester_r 0 2023-08-11T15:53:28.276 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:53:28.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:28.277 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-11T15:53:28.434 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:53:28.434 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:53:28.499 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053651,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-11T15:53:28.500 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:53:28.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:28.500 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-11T15:53:28.556 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-11T15:53:28.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:53:28.558 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:53:28.558 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:53:28.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-11T15:53:28.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:53:28.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:28.621 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-11T15:53:28.677 INFO:teuthology.orchestra.run.smithi119.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-11T15:53:28.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:53:28.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:53:28.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.917+0000 7f4f46542640 1 Processor -- start 2023-08-11T15:53:28.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.918+0000 7f4f46542640 1 -- start start 2023-08-11T15:53:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.919+0000 7f4f46542640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f4015c620 0x7f4f400b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.919+0000 7f4f46542640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f400cfb00 0x7f4f400d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.919+0000 7f4f46542640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f400cda30 con 0x7f4f400b82f0 2023-08-11T15:53:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.919+0000 7f4f46542640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f400ccc10 con 0x7f4f400cfb00 2023-08-11T15:53:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.919+0000 7f4f46542640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f40161110 con 0x7f4f4015c620 2023-08-11T15:53:28.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.920+0000 7f4f3ffff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f4015c620 0x7f4f400b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:28.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.920+0000 7f4f44adf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f400cfb00 0x7f4f400d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:28.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.920+0000 7f4f3ffff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f4015c620 0x7f4f400b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:44066/0 (socket says 172.21.15.27:44066) 2023-08-11T15:53:28.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.920+0000 7f4f3ffff640 1 -- 172.21.15.27:0/2546136655 learned_addr learned my addr 172.21.15.27:0/2546136655 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:28.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.921+0000 7f4f3ffff640 1 -- 172.21.15.27:0/2546136655 >> v2:172.21.15.119:3300/0 conn(0x7f4f400cfb00 msgr2=0x7f4f400d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:28.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.921+0000 7f4f3ffff640 1 --2- 172.21.15.27:0/2546136655 >> v2:172.21.15.119:3300/0 conn(0x7f4f400cfb00 0x7f4f400d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:28.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.921+0000 7f4f3ffff640 1 -- 172.21.15.27:0/2546136655 >> v1:172.21.15.119:6789/0 conn(0x7f4f400b82f0 legacy=0x7f4f400cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:28.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.921+0000 7f4f3ffff640 1 -- 172.21.15.27:0/2546136655 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f400cce20 con 0x7f4f4015c620 2023-08-11T15:53:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.921+0000 7f4f44adf640 1 --2- 172.21.15.27:0/2546136655 >> v2:172.21.15.119:3300/0 conn(0x7f4f400cfb00 0x7f4f400d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.922+0000 7f4f3ffff640 1 --2- 172.21.15.27:0/2546136655 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f4015c620 0x7f4f400b7db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009870 tx=0x7f4f3002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec6102bdf88ea97 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.923+0000 7f4f3effd640 1 -- 172.21.15.27:0/2546136655 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30030840 con 0x7f4f4015c620 2023-08-11T15:53:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.923+0000 7f4f3effd640 1 -- 172.21.15.27:0/2546136655 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f300309a0 con 0x7f4f4015c620 2023-08-11T15:53:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.923+0000 7f4f3effd640 1 -- 172.21.15.27:0/2546136655 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30030d30 con 0x7f4f4015c620 2023-08-11T15:53:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.924+0000 7f4f46542640 1 -- 172.21.15.27:0/2546136655 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f4015c620 msgr2=0x7f4f400b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:28.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.924+0000 7f4f46542640 1 --2- 172.21.15.27:0/2546136655 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f4015c620 0x7f4f400b7db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009870 tx=0x7f4f3002b980 comp rx=0 tx=0).stop 2023-08-11T15:53:28.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.924+0000 7f4f46542640 1 -- 172.21.15.27:0/2546136655 shutdown_connections 2023-08-11T15:53:28.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.924+0000 7f4f46542640 1 --2- 172.21.15.27:0/2546136655 >> v2:172.21.15.119:3300/0 conn(0x7f4f400cfb00 0x7f4f400d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:28.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.924+0000 7f4f46542640 1 --2- 172.21.15.27:0/2546136655 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f4015c620 0x7f4f400b7db0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:28.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.924+0000 7f4f46542640 1 -- 172.21.15.27:0/2546136655 >> 172.21.15.27:0/2546136655 conn(0x7f4f40071110 msgr2=0x7f4f400714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:28.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.924+0000 7f4f46542640 1 -- 172.21.15.27:0/2546136655 shutdown_connections 2023-08-11T15:53:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.924+0000 7f4f46542640 1 -- 172.21.15.27:0/2546136655 wait complete. 2023-08-11T15:53:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.926+0000 7f4f46542640 1 Processor -- start 2023-08-11T15:53:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.926+0000 7f4f46542640 1 -- start start 2023-08-11T15:53:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f46542640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f400b82f0 0x7f4f401f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f46542640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f400cfb00 0x7f4f40156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f46542640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f4015c620 0x7f4f401574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f46542640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4015d7b0 con 0x7f4f4015c620 2023-08-11T15:53:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f46542640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400c8f20 con 0x7f4f400b82f0 2023-08-11T15:53:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f46542640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4015cba0 con 0x7f4f400cfb00 2023-08-11T15:53:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f44adf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f4015c620 0x7f4f401574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f44adf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f4015c620 0x7f4f401574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54306/0 (socket says 172.21.15.27:54306) 2023-08-11T15:53:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f44adf640 1 -- 172.21.15.27:0/616420575 learned_addr learned my addr 172.21.15.27:0/616420575 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:28.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f3f7fe640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f400cfb00 0x7f4f40156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:28.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f44adf640 1 -- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f400cfb00 msgr2=0x7f4f40156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f44adf640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f400cfb00 0x7f4f40156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f44adf640 1 -- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f400b82f0 msgr2=0x7f4f401f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f44adf640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f400b82f0 0x7f4f401f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.927+0000 7f4f44adf640 1 -- 172.21.15.27:0/616420575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38009a70 con 0x7f4f4015c620 2023-08-11T15:53:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.928+0000 7f4f44adf640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f4015c620 0x7f4f401574c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800efb0 tx=0x7f4f3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.928+0000 7f4f3cff9640 1 -- 172.21.15.27:0/616420575 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38009d50 con 0x7f4f4015c620 2023-08-11T15:53:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.928+0000 7f4f3cff9640 1 -- 172.21.15.27:0/616420575 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3800f040 con 0x7f4f4015c620 2023-08-11T15:53:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.928+0000 7f4f46542640 1 -- 172.21.15.27:0/616420575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f300094d0 con 0x7f4f4015c620 2023-08-11T15:53:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.928+0000 7f4f3cff9640 1 -- 172.21.15.27:0/616420575 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f380146a0 con 0x7f4f4015c620 2023-08-11T15:53:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.928+0000 7f4f46542640 1 -- 172.21.15.27:0/616420575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4015d160 con 0x7f4f4015c620 2023-08-11T15:53:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.930+0000 7f4f46542640 1 -- 172.21.15.27:0/616420575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f400cddf0 con 0x7f4f4015c620 2023-08-11T15:53:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.931+0000 7f4f3cff9640 1 -- 172.21.15.27:0/616420575 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4f38014820 con 0x7f4f4015c620 2023-08-11T15:53:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.932+0000 7f4f3cff9640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f2c06a050 0x7f4f2c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.932+0000 7f4f3cff9640 1 -- 172.21.15.27:0/616420575 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4f38089720 con 0x7f4f4015c620 2023-08-11T15:53:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.933+0000 7f4f3ffff640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f2c06a050 0x7f4f2c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.934+0000 7f4f3ffff640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f2c06a050 0x7f4f2c06c510 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4f30036000 tx=0x7f4f3003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:28.937+0000 7f4f3cff9640 1 -- 172.21.15.27:0/616420575 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f4f38058760 con 0x7f4f4015c620 2023-08-11T15:53:29.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.170+0000 7f4f46542640 1 -- 172.21.15.27:0/616420575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f40073990 con 0x7f4f4015c620 2023-08-11T15:53:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.171+0000 7f4f3cff9640 1 -- 172.21.15.27:0/616420575 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f4f380582f0 con 0x7f4f4015c620 2023-08-11T15:53:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:29.173 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2294518038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2294518038},{"type":"v1","addr":"172.21.15.119:6836","nonce":2294518038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5188,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2104666724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2104666724},{"type":"v1","addr":"172.21.15.119:6837","nonce":2104666724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2379252997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2379252997},{"type":"v1","addr":"172.21.15.27:6833","nonce":2379252997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:53:09.220611+0000","modified":"2023-08-11T15:53:12.792531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5643},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5643":{"gid":5643,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/899405615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899405615},{"type":"v1","addr":"172.21.15.27:6835","nonce":899405615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-11T15:53:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 18 2023-08-11T15:53:29.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 -- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f2c06a050 msgr2=0x7f4f2c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f2c06a050 0x7f4f2c06c510 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4f30036000 tx=0x7f4f3003b040 comp rx=0 tx=0).stop 2023-08-11T15:53:29.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 -- 172.21.15.27:0/616420575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f4015c620 msgr2=0x7f4f401574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f4015c620 0x7f4f401574c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800efb0 tx=0x7f4f3800c490 comp rx=0 tx=0).stop 2023-08-11T15:53:29.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 -- 172.21.15.27:0/616420575 shutdown_connections 2023-08-11T15:53:29.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f2c06a050 0x7f4f2c06c510 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f400cfb00 0x7f4f40156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f4015c620 0x7f4f401574c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 --2- 172.21.15.27:0/616420575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f400b82f0 0x7f4f401f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 -- 172.21.15.27:0/616420575 >> 172.21.15.27:0/616420575 conn(0x7f4f40071110 msgr2=0x7f4f400b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 -- 172.21.15.27:0/616420575 shutdown_connections 2023-08-11T15:53:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.174+0000 7f4f46542640 1 -- 172.21.15.27:0/616420575 wait complete. 2023-08-11T15:53:29.196 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:53:29.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.418+0000 7f0a182a6640 1 Processor -- start 2023-08-11T15:53:29.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.418+0000 7f0a182a6640 1 -- start start 2023-08-11T15:53:29.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.420+0000 7f0a182a6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0a1015a140 0x7f0a1006b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.420+0000 7f0a182a6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0a100cb430 0x7f0a100b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.420+0000 7f0a182a6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a100c8a30 con 0x7f0a1006f5d0 2023-08-11T15:53:29.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.420+0000 7f0a182a6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1006c0c0 con 0x7f0a1015a140 2023-08-11T15:53:29.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.420+0000 7f0a182a6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1006c3a0 con 0x7f0a100cb430 2023-08-11T15:53:29.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.421+0000 7f0a16042640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0a1015a140 0x7f0a1006b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.421+0000 7f0a16042640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0a1015a140 0x7f0a1006b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52860/0 (socket says 172.21.15.27:52860) 2023-08-11T15:53:29.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.421+0000 7f0a16042640 1 -- 172.21.15.27:0/2213314430 learned_addr learned my addr 172.21.15.27:0/2213314430 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:29.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.421+0000 7f0a15841640 1 --2- 172.21.15.27:0/2213314430 >> v2:172.21.15.119:3300/0 conn(0x7f0a100cb430 0x7f0a100b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.421+0000 7f0a16042640 1 -- 172.21.15.27:0/2213314430 >> v2:172.21.15.119:3300/0 conn(0x7f0a100cb430 msgr2=0x7f0a100b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:29.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.421+0000 7f0a16042640 1 --2- 172.21.15.27:0/2213314430 >> v2:172.21.15.119:3300/0 conn(0x7f0a100cb430 0x7f0a100b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.421+0000 7f0a16042640 1 -- 172.21.15.27:0/2213314430 >> v1:172.21.15.119:6789/0 conn(0x7f0a1006f5d0 legacy=0x7f0a100c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.421+0000 7f0a16042640 1 -- 172.21.15.27:0/2213314430 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1006c680 con 0x7f0a1015a140 2023-08-11T15:53:29.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.423+0000 7f0a16042640 1 --2- 172.21.15.27:0/2213314430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1015a140 0x7f0a1006b870 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0a100b4130 tx=0x7f0a0c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa79f7a8b0ff8c43 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:29.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.423+0000 7f0a15040640 1 -- 172.21.15.27:0/2213314430 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c004230 con 0x7f0a1015a140 2023-08-11T15:53:29.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.423+0000 7f0a15040640 1 -- 172.21.15.27:0/2213314430 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c004390 con 0x7f0a1015a140 2023-08-11T15:53:29.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.423+0000 7f0a15040640 1 -- 172.21.15.27:0/2213314430 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c030840 con 0x7f0a1015a140 2023-08-11T15:53:29.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.424+0000 7f0a182a6640 1 -- 172.21.15.27:0/2213314430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1015a140 msgr2=0x7f0a1006b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.424+0000 7f0a182a6640 1 --2- 172.21.15.27:0/2213314430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1015a140 0x7f0a1006b870 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0a100b4130 tx=0x7f0a0c02b980 comp rx=0 tx=0).stop 2023-08-11T15:53:29.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.424+0000 7f0a182a6640 1 -- 172.21.15.27:0/2213314430 shutdown_connections 2023-08-11T15:53:29.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.424+0000 7f0a182a6640 1 --2- 172.21.15.27:0/2213314430 >> v2:172.21.15.119:3300/0 conn(0x7f0a100cb430 0x7f0a100b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.424+0000 7f0a182a6640 1 --2- 172.21.15.27:0/2213314430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1015a140 0x7f0a1006b870 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.424+0000 7f0a182a6640 1 -- 172.21.15.27:0/2213314430 >> 172.21.15.27:0/2213314430 conn(0x7f0a100afb70 msgr2=0x7f0a100aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:29.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.424+0000 7f0a182a6640 1 -- 172.21.15.27:0/2213314430 shutdown_connections 2023-08-11T15:53:29.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.424+0000 7f0a182a6640 1 -- 172.21.15.27:0/2213314430 wait complete. 2023-08-11T15:53:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.426+0000 7f0a182a6640 1 Processor -- start 2023-08-11T15:53:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.426+0000 7f0a182a6640 1 -- start start 2023-08-11T15:53:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.427+0000 7f0a182a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1006f5d0 0x7f0a100acc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.427+0000 7f0a182a6640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a100cb430 0x7f0a100a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.427+0000 7f0a182a6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a1015a140 0x7f0a100c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.427+0000 7f0a182a6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1014f870 con 0x7f0a1006f5d0 2023-08-11T15:53:29.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.427+0000 7f0a182a6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a100ad180 con 0x7f0a1015a140 2023-08-11T15:53:29.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.427+0000 7f0a182a6640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a100ad440 con 0x7f0a100cb430 2023-08-11T15:53:29.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.428+0000 7f0a16042640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1006f5d0 0x7f0a100acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.428+0000 7f0a16042640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1006f5d0 0x7f0a100acc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52868/0 (socket says 172.21.15.27:52868) 2023-08-11T15:53:29.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.428+0000 7f0a16042640 1 -- 172.21.15.27:0/1572495665 learned_addr learned my addr 172.21.15.27:0/1572495665 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:29.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.428+0000 7f0a15841640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a100cb430 0x7f0a100a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.428+0000 7f0a16843640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a1015a140 0x7f0a100c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.428+0000 7f0a16042640 1 -- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a100cb430 msgr2=0x7f0a100a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.428+0000 7f0a16042640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a100cb430 0x7f0a100a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.428+0000 7f0a16042640 1 -- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a1015a140 msgr2=0x7f0a100c3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.428+0000 7f0a16042640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a1015a140 0x7f0a100c3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.428+0000 7f0a16042640 1 -- 172.21.15.27:0/1572495665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c009460 con 0x7f0a1006f5d0 2023-08-11T15:53:29.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.429+0000 7f0a16042640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1006f5d0 0x7f0a100acc40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c000c00 tx=0x7f0a0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:29.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.429+0000 7f0a06ffd640 1 -- 172.21.15.27:0/1572495665 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c03d070 con 0x7f0a1006f5d0 2023-08-11T15:53:29.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.429+0000 7f0a06ffd640 1 -- 172.21.15.27:0/1572495665 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c004500 con 0x7f0a1006f5d0 2023-08-11T15:53:29.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.429+0000 7f0a06ffd640 1 -- 172.21.15.27:0/1572495665 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c030e40 con 0x7f0a1006f5d0 2023-08-11T15:53:29.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.429+0000 7f0a182a6640 1 -- 172.21.15.27:0/1572495665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a100ad720 con 0x7f0a1006f5d0 2023-08-11T15:53:29.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.429+0000 7f0a182a6640 1 -- 172.21.15.27:0/1572495665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a100adce0 con 0x7f0a1006f5d0 2023-08-11T15:53:29.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.430+0000 7f0a182a6640 1 -- 172.21.15.27:0/1572495665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1015a520 con 0x7f0a1006f5d0 2023-08-11T15:53:29.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.431+0000 7f0a06ffd640 1 -- 172.21.15.27:0/1572495665 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0a0c003610 con 0x7f0a1006f5d0 2023-08-11T15:53:29.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.432+0000 7f0a06ffd640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09f406a080 0x7f09f406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.432+0000 7f0a06ffd640 1 -- 172.21.15.27:0/1572495665 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0a0c0aa7b0 con 0x7f0a1006f5d0 2023-08-11T15:53:29.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.433+0000 7f0a15841640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09f406a080 0x7f09f406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.434+0000 7f0a15841640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09f406a080 0x7f09f406c540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a100c14d0 tx=0x7f0a00008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:29.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.437+0000 7f0a06ffd640 1 -- 172.21.15.27:0/1572495665 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0a0c075e70 con 0x7f0a1006f5d0 2023-08-11T15:53:29.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.642+0000 7f0a182a6640 1 -- 172.21.15.27:0/1572495665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a10073990 con 0x7f0a1006f5d0 2023-08-11T15:53:29.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.644+0000 7f0a06ffd640 1 -- 172.21.15.27:0/1572495665 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18500 (secure 0 0 0) 0x7f0a0c079490 con 0x7f0a1006f5d0 2023-08-11T15:53:29.647 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:29.647 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":43,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:53:10.738370+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:53:06.898951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-11T15:53:08.258996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:53:29.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 -- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09f406a080 msgr2=0x7f09f406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09f406a080 0x7f09f406c540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a100c14d0 tx=0x7f0a00008040 comp rx=0 tx=0).stop 2023-08-11T15:53:29.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 -- 172.21.15.27:0/1572495665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1006f5d0 msgr2=0x7f0a100acc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1006f5d0 0x7f0a100acc40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c000c00 tx=0x7f0a0c0046c0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 -- 172.21.15.27:0/1572495665 shutdown_connections 2023-08-11T15:53:29.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f09f406a080 0x7f09f406c540 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0a100cb430 0x7f0a100a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a1015a140 0x7f0a100c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 --2- 172.21.15.27:0/1572495665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a1006f5d0 0x7f0a100acc40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 -- 172.21.15.27:0/1572495665 >> 172.21.15.27:0/1572495665 conn(0x7f0a100afb70 msgr2=0x7f0a10072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:29.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 -- 172.21.15.27:0/1572495665 shutdown_connections 2023-08-11T15:53:29.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.647+0000 7f0a182a6640 1 -- 172.21.15.27:0/1572495665 wait complete. 2023-08-11T15:53:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:50:46.794326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-11T15:50:53.535004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:53:06.898951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:53:08.258996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T15:53:29.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:53:29.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.907+0000 7f363cba4640 1 Processor -- start 2023-08-11T15:53:29.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.907+0000 7f363cba4640 1 -- start start 2023-08-11T15:53:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.908+0000 7f363cba4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f363815c760 0x7f36380d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.909+0000 7f363cba4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f363806b670 0x7f36380d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.909+0000 7f363cba4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363806bc10 con 0x7f36380d2850 2023-08-11T15:53:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.909+0000 7f363cba4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363806c790 con 0x7f363806b670 2023-08-11T15:53:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.909+0000 7f363cba4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363806bef0 con 0x7f363815c760 2023-08-11T15:53:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.909+0000 7f3635d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f363806b670 0x7f36380d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.909+0000 7f3635d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f363806b670 0x7f36380d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52872/0 (socket says 172.21.15.27:52872) 2023-08-11T15:53:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.909+0000 7f3635d9b640 1 -- 172.21.15.27:0/536574256 learned_addr learned my addr 172.21.15.27:0/536574256 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.909+0000 7f363659c640 1 --2- 172.21.15.27:0/536574256 >> v2:172.21.15.119:3300/0 conn(0x7f363815c760 0x7f36380d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.910+0000 7f3635d9b640 1 -- 172.21.15.27:0/536574256 >> v2:172.21.15.119:3300/0 conn(0x7f363815c760 msgr2=0x7f36380d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.910+0000 7f3635d9b640 1 --2- 172.21.15.27:0/536574256 >> v2:172.21.15.119:3300/0 conn(0x7f363815c760 0x7f36380d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.910+0000 7f3635d9b640 1 -- 172.21.15.27:0/536574256 >> v1:172.21.15.119:6789/0 conn(0x7f36380d2850 legacy=0x7f36381602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.910+0000 7f3635d9b640 1 -- 172.21.15.27:0/536574256 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363806c1d0 con 0x7f363806b670 2023-08-11T15:53:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.911+0000 7f3635d9b640 1 --2- 172.21.15.27:0/536574256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363806b670 0x7f36380d2310 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f36380a0580 tx=0x7f362c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3f86bb6c51b2fc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.911+0000 7f363559a640 1 -- 172.21.15.27:0/536574256 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c031870 con 0x7f363806b670 2023-08-11T15:53:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.911+0000 7f363559a640 1 -- 172.21.15.27:0/536574256 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c0319d0 con 0x7f363806b670 2023-08-11T15:53:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.911+0000 7f363559a640 1 -- 172.21.15.27:0/536574256 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c031d60 con 0x7f363806b670 2023-08-11T15:53:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.912+0000 7f363cba4640 1 -- 172.21.15.27:0/536574256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363806b670 msgr2=0x7f36380d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.912+0000 7f363cba4640 1 --2- 172.21.15.27:0/536574256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363806b670 0x7f36380d2310 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f36380a0580 tx=0x7f362c02b910 comp rx=0 tx=0).stop 2023-08-11T15:53:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.912+0000 7f363cba4640 1 -- 172.21.15.27:0/536574256 shutdown_connections 2023-08-11T15:53:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.912+0000 7f363cba4640 1 --2- 172.21.15.27:0/536574256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363806b670 0x7f36380d2310 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.912+0000 7f363cba4640 1 --2- 172.21.15.27:0/536574256 >> v2:172.21.15.119:3300/0 conn(0x7f363815c760 0x7f36380d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.912+0000 7f363cba4640 1 -- 172.21.15.27:0/536574256 >> 172.21.15.27:0/536574256 conn(0x7f3638089ff0 msgr2=0x7f363809b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.912+0000 7f363cba4640 1 -- 172.21.15.27:0/536574256 shutdown_connections 2023-08-11T15:53:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.912+0000 7f363cba4640 1 -- 172.21.15.27:0/536574256 wait complete. 2023-08-11T15:53:29.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.914+0000 7f363cba4640 1 Processor -- start 2023-08-11T15:53:29.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.914+0000 7f363cba4640 1 -- start start 2023-08-11T15:53:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.915+0000 7f363cba4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f363806b670 0x7f36380c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.915+0000 7f363cba4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36380d2850 0x7f36380d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.915+0000 7f363cba4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363815c760 0x7f36380b6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.915+0000 7f363cba4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36380c8620 con 0x7f363815c760 2023-08-11T15:53:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.915+0000 7f363cba4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36380c90d0 con 0x7f36380d2850 2023-08-11T15:53:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.915+0000 7f363cba4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363815ded0 con 0x7f363806b670 2023-08-11T15:53:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.916+0000 7f3636d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363815c760 0x7f36380b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.916+0000 7f3636d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363815c760 0x7f36380b6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52880/0 (socket says 172.21.15.27:52880) 2023-08-11T15:53:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.916+0000 7f3636d9d640 1 -- 172.21.15.27:0/2861550782 learned_addr learned my addr 172.21.15.27:0/2861550782 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.916+0000 7f3635d9b640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36380d2850 0x7f36380d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.916+0000 7f363659c640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f363806b670 0x7f36380c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.916+0000 7f3636d9d640 1 -- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f363806b670 msgr2=0x7f36380c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.916+0000 7f3636d9d640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f363806b670 0x7f36380c7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.916+0000 7f3636d9d640 1 -- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36380d2850 msgr2=0x7f36380d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:29.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.916+0000 7f3636d9d640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36380d2850 0x7f36380d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:29.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.916+0000 7f3636d9d640 1 -- 172.21.15.27:0/2861550782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0094d0 con 0x7f363815c760 2023-08-11T15:53:29.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.917+0000 7f3636d9d640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363815c760 0x7f36380b6870 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f363000c440 tx=0x7f363000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:29.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.917+0000 7f3626ffd640 1 -- 172.21.15.27:0/2861550782 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3630011790 con 0x7f363815c760 2023-08-11T15:53:29.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.917+0000 7f3626ffd640 1 -- 172.21.15.27:0/2861550782 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36300118f0 con 0x7f363815c760 2023-08-11T15:53:29.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.917+0000 7f3626ffd640 1 -- 172.21.15.27:0/2861550782 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3630011ca0 con 0x7f363815c760 2023-08-11T15:53:29.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.917+0000 7f363cba4640 1 -- 172.21.15.27:0/2861550782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36380c8830 con 0x7f363815c760 2023-08-11T15:53:29.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.917+0000 7f363cba4640 1 -- 172.21.15.27:0/2861550782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36380c8df0 con 0x7f363815c760 2023-08-11T15:53:29.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.918+0000 7f363cba4640 1 -- 172.21.15.27:0/2861550782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363804e420 con 0x7f363815c760 2023-08-11T15:53:29.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.919+0000 7f3626ffd640 1 -- 172.21.15.27:0/2861550782 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f363001c460 con 0x7f363815c760 2023-08-11T15:53:29.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.920+0000 7f3626ffd640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f361806a080 0x7f361806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:29.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.920+0000 7f3626ffd640 1 -- 172.21.15.27:0/2861550782 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3630088580 con 0x7f363815c760 2023-08-11T15:53:29.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.921+0000 7f363659c640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f361806a080 0x7f361806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:29.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.922+0000 7f363659c640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f361806a080 0x7f361806c540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f36280099c0 tx=0x7f3628003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:29.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:29.925+0000 7f3626ffd640 1 -- 172.21.15.27:0/2861550782 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f3630053b20 con 0x7f363815c760 2023-08-11T15:53:30.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.118+0000 7f363cba4640 1 -- 172.21.15.27:0/2861550782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f363815e010 con 0x7f363815c760 2023-08-11T15:53:30.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.119+0000 7f3626ffd640 1 -- 172.21.15.27:0/2861550782 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3630057140 con 0x7f363815c760 2023-08-11T15:53:30.120 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-08-11T15:53:30.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.121+0000 7f363cba4640 1 -- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f361806a080 msgr2=0x7f361806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.121+0000 7f363cba4640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f361806a080 0x7f361806c540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f36280099c0 tx=0x7f3628003990 comp rx=0 tx=0).stop 2023-08-11T15:53:30.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.122+0000 7f363cba4640 1 -- 172.21.15.27:0/2861550782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363815c760 msgr2=0x7f36380b6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.122+0000 7f363cba4640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363815c760 0x7f36380b6870 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f363000c440 tx=0x7f363000c900 comp rx=0 tx=0).stop 2023-08-11T15:53:30.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.122+0000 7f363cba4640 1 -- 172.21.15.27:0/2861550782 shutdown_connections 2023-08-11T15:53:30.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.122+0000 7f363cba4640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f361806a080 0x7f361806c540 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.122+0000 7f363cba4640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363815c760 0x7f36380b6870 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.122+0000 7f363cba4640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36380d2850 0x7f36380d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.122+0000 7f363cba4640 1 --2- 172.21.15.27:0/2861550782 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f363806b670 0x7f36380c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.122+0000 7f363cba4640 1 -- 172.21.15.27:0/2861550782 >> 172.21.15.27:0/2861550782 conn(0x7f3638089ff0 msgr2=0x7f363815a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.122+0000 7f363cba4640 1 -- 172.21.15.27:0/2861550782 shutdown_connections 2023-08-11T15:53:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.122+0000 7f363cba4640 1 -- 172.21.15.27:0/2861550782 wait complete. 2023-08-11T15:53:30.141 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-11T15:53:30.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.363+0000 7f213dc63640 1 Processor -- start 2023-08-11T15:53:30.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.363+0000 7f213dc63640 1 -- start start 2023-08-11T15:53:30.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.365+0000 7f213dc63640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f213813ff20 0x7f21380cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.365+0000 7f213dc63640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f213813d260 0x7f21380ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.365+0000 7f213dc63640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21380cd1a0 con 0x7f213813ce80 2023-08-11T15:53:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.365+0000 7f213dc63640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2138140670 con 0x7f213813ff20 2023-08-11T15:53:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.365+0000 7f213dc63640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21380cd470 con 0x7f213813d260 2023-08-11T15:53:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.365+0000 7f213cc61640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f213813ce80 0x7f213813f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50048/0 (socket says 172.21.15.27:50048) 2023-08-11T15:53:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.365+0000 7f213cc61640 1 -- 172.21.15.27:0/3936654923 learned_addr learned my addr 172.21.15.27:0/3936654923 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.366+0000 7f2137fff640 1 --2- 172.21.15.27:0/3936654923 >> v2:172.21.15.27:3300/0 conn(0x7f213813ff20 0x7f21380cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.366+0000 7f213d462640 1 --2- 172.21.15.27:0/3936654923 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f213813d260 0x7f21380ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.366+0000 7f21377fe640 1 -- 172.21.15.27:0/3936654923 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513823896 0 0) 0x7f21380cd1a0 con 0x7f213813ce80 2023-08-11T15:53:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.366+0000 7f21377fe640 1 -- 172.21.15.27:0/3936654923 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2128003520 con 0x7f213813ce80 2023-08-11T15:53:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.366+0000 7f2137fff640 1 -- 172.21.15.27:0/3936654923 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f213813d260 msgr2=0x7f21380ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.366+0000 7f2137fff640 1 --2- 172.21.15.27:0/3936654923 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f213813d260 0x7f21380ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.366+0000 7f2137fff640 1 -- 172.21.15.27:0/3936654923 >> v1:172.21.15.27:6789/0 conn(0x7f213813ce80 legacy=0x7f213813f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.366+0000 7f2137fff640 1 -- 172.21.15.27:0/3936654923 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21380cd750 con 0x7f213813ff20 2023-08-11T15:53:30.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.368+0000 7f2137fff640 1 --2- 172.21.15.27:0/3936654923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ff20 0x7f21380cc720 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f21240097a0 tx=0x7f212402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abafa18c8092e6d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.368+0000 7f21377fe640 1 -- 172.21.15.27:0/3936654923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212402d040 con 0x7f213813ff20 2023-08-11T15:53:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.368+0000 7f21377fe640 1 -- 172.21.15.27:0/3936654923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2124031810 con 0x7f213813ff20 2023-08-11T15:53:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.368+0000 7f21377fe640 1 -- 172.21.15.27:0/3936654923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124031b80 con 0x7f213813ff20 2023-08-11T15:53:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.368+0000 7f213dc63640 1 -- 172.21.15.27:0/3936654923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ff20 msgr2=0x7f21380cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.368+0000 7f213dc63640 1 --2- 172.21.15.27:0/3936654923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ff20 0x7f21380cc720 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f21240097a0 tx=0x7f212402b8b0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.369+0000 7f213dc63640 1 -- 172.21.15.27:0/3936654923 shutdown_connections 2023-08-11T15:53:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.369+0000 7f213dc63640 1 --2- 172.21.15.27:0/3936654923 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f213813d260 0x7f21380ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.369+0000 7f213dc63640 1 --2- 172.21.15.27:0/3936654923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ff20 0x7f21380cc720 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.369+0000 7f213dc63640 1 -- 172.21.15.27:0/3936654923 >> 172.21.15.27:0/3936654923 conn(0x7f2138069800 msgr2=0x7f2138069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.369+0000 7f213dc63640 1 -- 172.21.15.27:0/3936654923 shutdown_connections 2023-08-11T15:53:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.369+0000 7f213dc63640 1 -- 172.21.15.27:0/3936654923 wait complete. 2023-08-11T15:53:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.371+0000 7f213dc63640 1 Processor -- start 2023-08-11T15:53:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.371+0000 7f213dc63640 1 -- start start 2023-08-11T15:53:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213dc63640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ce80 0x7f2138070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213dc63640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f213813d260 0x7f2138159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213dc63640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f213813ff20 0x7f21381598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213dc63640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380b0af0 con 0x7f213813ce80 2023-08-11T15:53:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213dc63640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213813ba10 con 0x7f213813d260 2023-08-11T15:53:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213dc63640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213813bcb0 con 0x7f213813ff20 2023-08-11T15:53:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213cc61640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ce80 0x7f2138070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213cc61640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ce80 0x7f2138070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52894/0 (socket says 172.21.15.27:52894) 2023-08-11T15:53:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213cc61640 1 -- 172.21.15.27:0/4252484021 learned_addr learned my addr 172.21.15.27:0/4252484021 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213cc61640 1 -- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f213813ff20 msgr2=0x7f21381598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213cc61640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f213813ff20 0x7f21381598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213cc61640 1 -- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f213813d260 msgr2=0x7f2138159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213cc61640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f213813d260 0x7f2138159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.372+0000 7f213cc61640 1 -- 172.21.15.27:0/4252484021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2124009400 con 0x7f213813ce80 2023-08-11T15:53:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.373+0000 7f213cc61640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ce80 0x7f2138070410 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f212c00c430 tx=0x7f212c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.373+0000 7f21357fa640 1 -- 172.21.15.27:0/4252484021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c005ce0 con 0x7f213813ce80 2023-08-11T15:53:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.373+0000 7f21357fa640 1 -- 172.21.15.27:0/4252484021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c005e40 con 0x7f213813ce80 2023-08-11T15:53:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.373+0000 7f21357fa640 1 -- 172.21.15.27:0/4252484021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c0072f0 con 0x7f213813ce80 2023-08-11T15:53:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.373+0000 7f213dc63640 1 -- 172.21.15.27:0/4252484021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213813bf90 con 0x7f213813ce80 2023-08-11T15:53:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.373+0000 7f213dc63640 1 -- 172.21.15.27:0/4252484021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213813c550 con 0x7f213813ce80 2023-08-11T15:53:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.374+0000 7f213dc63640 1 -- 172.21.15.27:0/4252484021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213813c6e0 con 0x7f213813ce80 2023-08-11T15:53:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.375+0000 7f21357fa640 1 -- 172.21.15.27:0/4252484021 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f212c007450 con 0x7f213813ce80 2023-08-11T15:53:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.376+0000 7f21357fa640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f211c06a050 0x7f211c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.376+0000 7f21357fa640 1 -- 172.21.15.27:0/4252484021 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f212c087db0 con 0x7f213813ce80 2023-08-11T15:53:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.377+0000 7f2137fff640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f211c06a050 0x7f211c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.378+0000 7f2137fff640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f211c06a050 0x7f211c06c510 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2124036000 tx=0x7f2124030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:30.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.383+0000 7f21357fa640 1 -- 172.21.15.27:0/4252484021 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f212c08c070 con 0x7f213813ce80 2023-08-11T15:53:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.586+0000 7f213dc63640 1 -- 172.21.15.27:0/4252484021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2138066e80 con 0x7f213813ce80 2023-08-11T15:53:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.587+0000 7f21357fa640 1 -- 172.21.15.27:0/4252484021 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7f212c0170a0 con 0x7f213813ce80 2023-08-11T15:53:30.589 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-08-11T15:53:30.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 -- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f211c06a050 msgr2=0x7f211c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f211c06a050 0x7f211c06c510 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2124036000 tx=0x7f2124030000 comp rx=0 tx=0).stop 2023-08-11T15:53:30.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 -- 172.21.15.27:0/4252484021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ce80 msgr2=0x7f2138070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ce80 0x7f2138070410 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f212c00c430 tx=0x7f212c00c8f0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 -- 172.21.15.27:0/4252484021 shutdown_connections 2023-08-11T15:53:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f211c06a050 0x7f211c06c510 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f213813ff20 0x7f21381598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f213813d260 0x7f2138159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 --2- 172.21.15.27:0/4252484021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f213813ce80 0x7f2138070410 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 -- 172.21.15.27:0/4252484021 >> 172.21.15.27:0/4252484021 conn(0x7f2138069800 msgr2=0x7f21380cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 -- 172.21.15.27:0/4252484021 shutdown_connections 2023-08-11T15:53:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.590+0000 7f213dc63640 1 -- 172.21.15.27:0/4252484021 wait complete. 2023-08-11T15:53:30.611 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:53:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.851+0000 7ff781542640 1 Processor -- start 2023-08-11T15:53:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.851+0000 7ff781542640 1 -- start start 2023-08-11T15:53:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.852+0000 7ff781542640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff77c0b1d10 0x7ff77c0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.852+0000 7ff781542640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff77c0cf1f0 0x7ff77c0cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.852+0000 7ff781542640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c0b2630 con 0x7ff77c0cada0 2023-08-11T15:53:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff781542640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c15a8b0 con 0x7ff77c0b1d10 2023-08-11T15:53:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff781542640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c0cfb20 con 0x7ff77c0cf1f0 2023-08-11T15:53:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff77affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff77c0b1d10 0x7ff77c0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff77affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff77c0b1d10 0x7ff77c0b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52900/0 (socket says 172.21.15.27:52900) 2023-08-11T15:53:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff77affd640 1 -- 172.21.15.27:0/2366047141 learned_addr learned my addr 172.21.15.27:0/2366047141 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff77b7fe640 1 --2- 172.21.15.27:0/2366047141 >> v2:172.21.15.119:3300/0 conn(0x7ff77c0cf1f0 0x7ff77c0cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff77affd640 1 -- 172.21.15.27:0/2366047141 >> v2:172.21.15.119:3300/0 conn(0x7ff77c0cf1f0 msgr2=0x7ff77c0cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff77affd640 1 --2- 172.21.15.27:0/2366047141 >> v2:172.21.15.119:3300/0 conn(0x7ff77c0cf1f0 0x7ff77c0cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff77affd640 1 -- 172.21.15.27:0/2366047141 >> v1:172.21.15.119:6789/0 conn(0x7ff77c0cada0 legacy=0x7ff77c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff77affd640 1 -- 172.21.15.27:0/2366047141 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0cfe00 con 0x7ff77c0b1d10 2023-08-11T15:53:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.853+0000 7ff77b7fe640 1 --2- 172.21.15.27:0/2366047141 >> v2:172.21.15.119:3300/0 conn(0x7ff77c0cf1f0 0x7ff77c0cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.855+0000 7ff77affd640 1 --2- 172.21.15.27:0/2366047141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 0x7ff77c0b20f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff77c071750 tx=0x7ff76c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab60d0dc14dfb093 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.855+0000 7ff779ffb640 1 -- 172.21.15.27:0/2366047141 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c02c9f0 con 0x7ff77c0b1d10 2023-08-11T15:53:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.855+0000 7ff779ffb640 1 -- 172.21.15.27:0/2366047141 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c02cb50 con 0x7ff77c0b1d10 2023-08-11T15:53:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.855+0000 7ff779ffb640 1 -- 172.21.15.27:0/2366047141 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0343f0 con 0x7ff77c0b1d10 2023-08-11T15:53:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.856+0000 7ff781542640 1 -- 172.21.15.27:0/2366047141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 msgr2=0x7ff77c0b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.856+0000 7ff781542640 1 --2- 172.21.15.27:0/2366047141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 0x7ff77c0b20f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff77c071750 tx=0x7ff76c02be50 comp rx=0 tx=0).stop 2023-08-11T15:53:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.856+0000 7ff781542640 1 -- 172.21.15.27:0/2366047141 shutdown_connections 2023-08-11T15:53:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.856+0000 7ff781542640 1 --2- 172.21.15.27:0/2366047141 >> v2:172.21.15.119:3300/0 conn(0x7ff77c0cf1f0 0x7ff77c0cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.856+0000 7ff781542640 1 --2- 172.21.15.27:0/2366047141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 0x7ff77c0b20f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.856+0000 7ff781542640 1 -- 172.21.15.27:0/2366047141 >> 172.21.15.27:0/2366047141 conn(0x7ff77c0ccd10 msgr2=0x7ff77c0cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.856+0000 7ff781542640 1 -- 172.21.15.27:0/2366047141 shutdown_connections 2023-08-11T15:53:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.856+0000 7ff781542640 1 -- 172.21.15.27:0/2366047141 wait complete. 2023-08-11T15:53:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.858+0000 7ff781542640 1 Processor -- start 2023-08-11T15:53:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.858+0000 7ff781542640 1 -- start start 2023-08-11T15:53:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.859+0000 7ff781542640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 0x7ff77c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.859+0000 7ff781542640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77c0cada0 0x7ff77c0c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.859+0000 7ff781542640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff77c0cf1f0 0x7ff77c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.859+0000 7ff781542640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c0c8b20 con 0x7ff77c0b1d10 2023-08-11T15:53:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.859+0000 7ff781542640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c0ace50 con 0x7ff77c0cada0 2023-08-11T15:53:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.859+0000 7ff781542640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c0ad130 con 0x7ff77c0cf1f0 2023-08-11T15:53:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.859+0000 7ff77affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 0x7ff77c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.859+0000 7ff77affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 0x7ff77c1578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52914/0 (socket says 172.21.15.27:52914) 2023-08-11T15:53:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.859+0000 7ff77affd640 1 -- 172.21.15.27:0/2227211658 learned_addr learned my addr 172.21.15.27:0/2227211658 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.860+0000 7ff77a7fc640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77c0cada0 0x7ff77c0c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.860+0000 7ff77b7fe640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff77c0cf1f0 0x7ff77c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.860+0000 7ff77affd640 1 -- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff77c0cf1f0 msgr2=0x7ff77c156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.860+0000 7ff77affd640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff77c0cf1f0 0x7ff77c156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.860+0000 7ff77affd640 1 -- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77c0cada0 msgr2=0x7ff77c0c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.860+0000 7ff77affd640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77c0cada0 0x7ff77c0c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.860+0000 7ff77affd640 1 -- 172.21.15.27:0/2227211658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c009b30 con 0x7ff77c0b1d10 2023-08-11T15:53:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.860+0000 7ff77b7fe640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff77c0cf1f0 0x7ff77c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.860+0000 7ff77a7fc640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77c0cada0 0x7ff77c0c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.860+0000 7ff77affd640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 0x7ff77c1578d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff76c000f80 tx=0x7ff76c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.861+0000 7ff7737fe640 1 -- 172.21.15.27:0/2227211658 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c03e450 con 0x7ff77c0b1d10 2023-08-11T15:53:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.861+0000 7ff7737fe640 1 -- 172.21.15.27:0/2227211658 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c03e5f0 con 0x7ff77c0b1d10 2023-08-11T15:53:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.861+0000 7ff7737fe640 1 -- 172.21.15.27:0/2227211658 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c03e9a0 con 0x7ff77c0b1d10 2023-08-11T15:53:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.861+0000 7ff781542640 1 -- 172.21.15.27:0/2227211658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c0ad410 con 0x7ff77c0b1d10 2023-08-11T15:53:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.861+0000 7ff781542640 1 -- 172.21.15.27:0/2227211658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c0ad9d0 con 0x7ff77c0b1d10 2023-08-11T15:53:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.862+0000 7ff781542640 1 -- 172.21.15.27:0/2227211658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c04e420 con 0x7ff77c0b1d10 2023-08-11T15:53:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.863+0000 7ff7737fe640 1 -- 172.21.15.27:0/2227211658 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff76c03eb00 con 0x7ff77c0b1d10 2023-08-11T15:53:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.864+0000 7ff7737fe640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff75c06a050 0x7ff75c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.865+0000 7ff7737fe640 1 -- 172.21.15.27:0/2227211658 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff76c0ab650 con 0x7ff77c0b1d10 2023-08-11T15:53:30.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.865+0000 7ff77a7fc640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff75c06a050 0x7ff75c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:30.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.866+0000 7ff77a7fc640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff75c06a050 0x7ff75c06c510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff77c157e10 tx=0x7ff764008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:30.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:30.869+0000 7ff7737fe640 1 -- 172.21.15.27:0/2227211658 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7ff76c076d30 con 0x7ff77c0b1d10 2023-08-11T15:53:31.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.060+0000 7ff781542640 1 -- 172.21.15.27:0/2227211658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff77c0af330 con 0x7ff77c0b1d10 2023-08-11T15:53:31.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.061+0000 7ff7737fe640 1 -- 172.21.15.27:0/2227211658 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff76c03edb0 con 0x7ff77c0b1d10 2023-08-11T15:53:31.062 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-08-11T15:53:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.063+0000 7ff781542640 1 -- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff75c06a050 msgr2=0x7ff75c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.063+0000 7ff781542640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff75c06a050 0x7ff75c06c510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff77c157e10 tx=0x7ff764008040 comp rx=0 tx=0).stop 2023-08-11T15:53:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.063+0000 7ff781542640 1 -- 172.21.15.27:0/2227211658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 msgr2=0x7ff77c1578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.063+0000 7ff781542640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 0x7ff77c1578d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff76c000f80 tx=0x7ff76c004a40 comp rx=0 tx=0).stop 2023-08-11T15:53:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.064+0000 7ff781542640 1 -- 172.21.15.27:0/2227211658 shutdown_connections 2023-08-11T15:53:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.064+0000 7ff781542640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7ff75c06a050 0x7ff75c06c510 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.064+0000 7ff781542640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff77c0cf1f0 0x7ff77c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.064+0000 7ff781542640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77c0cada0 0x7ff77c0c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.064+0000 7ff781542640 1 --2- 172.21.15.27:0/2227211658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77c0b1d10 0x7ff77c1578d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.064+0000 7ff781542640 1 -- 172.21.15.27:0/2227211658 >> 172.21.15.27:0/2227211658 conn(0x7ff77c0ccd10 msgr2=0x7ff77c0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.064+0000 7ff781542640 1 -- 172.21.15.27:0/2227211658 shutdown_connections 2023-08-11T15:53:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.064+0000 7ff781542640 1 -- 172.21.15.27:0/2227211658 wait complete. 2023-08-11T15:53:31.086 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:53:31.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.316+0000 7f32e1e83640 1 Processor -- start 2023-08-11T15:53:31.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.316+0000 7f32e1e83640 1 -- start start 2023-08-11T15:53:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.318+0000 7f32e1e83640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f32dc151220 0x7f32dc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.318+0000 7f32e1e83640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32dc06c850 con 0x7f32dc06bdf0 2023-08-11T15:53:31.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.318+0000 7f32e1e83640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32dc06c290 con 0x7f32dc15e680 2023-08-11T15:53:31.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.318+0000 7f32e1e83640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32dc06c570 con 0x7f32dc151220 2023-08-11T15:53:31.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.318+0000 7f32daffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f32dc06bdf0 0x7f32dc15df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50050/0 (socket says 172.21.15.27:50050) 2023-08-11T15:53:31.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.319+0000 7f32daffd640 1 -- 172.21.15.27:0/1477106455 learned_addr learned my addr 172.21.15.27:0/1477106455 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:31.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.319+0000 7f32db7fe640 1 --2- 172.21.15.27:0/1477106455 >> v2:172.21.15.27:3300/0 conn(0x7f32dc151220 0x7f32dc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.319+0000 7f32db7fe640 1 -- 172.21.15.27:0/1477106455 >> v1:172.21.15.119:6789/0 conn(0x7f32dc15e680 legacy=0x7f32dc0ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:31.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.319+0000 7f32db7fe640 1 -- 172.21.15.27:0/1477106455 >> v1:172.21.15.27:6789/0 conn(0x7f32dc06bdf0 legacy=0x7f32dc15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.319+0000 7f32db7fe640 1 -- 172.21.15.27:0/1477106455 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc0ca050 con 0x7f32dc151220 2023-08-11T15:53:31.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.320+0000 7f32db7fe640 1 --2- 172.21.15.27:0/1477106455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc151220 0x7f32dc06b8b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0097a0 tx=0x7f32cc02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac174c07aac074e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:31.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.321+0000 7f32da7fc640 1 -- 172.21.15.27:0/1477106455 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc02f050 con 0x7f32dc151220 2023-08-11T15:53:31.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.321+0000 7f32da7fc640 1 -- 172.21.15.27:0/1477106455 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc031700 con 0x7f32dc151220 2023-08-11T15:53:31.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.321+0000 7f32da7fc640 1 -- 172.21.15.27:0/1477106455 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc031a90 con 0x7f32dc151220 2023-08-11T15:53:31.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.321+0000 7f32e1e83640 1 -- 172.21.15.27:0/1477106455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc151220 msgr2=0x7f32dc06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.321+0000 7f32e1e83640 1 --2- 172.21.15.27:0/1477106455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc151220 0x7f32dc06b8b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0097a0 tx=0x7f32cc02b7a0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.322+0000 7f32e1e83640 1 -- 172.21.15.27:0/1477106455 shutdown_connections 2023-08-11T15:53:31.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.322+0000 7f32e1e83640 1 --2- 172.21.15.27:0/1477106455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc151220 0x7f32dc06b8b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.322+0000 7f32e1e83640 1 -- 172.21.15.27:0/1477106455 >> 172.21.15.27:0/1477106455 conn(0x7f32dc0afbe0 msgr2=0x7f32dc0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:31.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.322+0000 7f32e1e83640 1 -- 172.21.15.27:0/1477106455 shutdown_connections 2023-08-11T15:53:31.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.322+0000 7f32e1e83640 1 -- 172.21.15.27:0/1477106455 wait complete. 2023-08-11T15:53:31.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.323+0000 7f32e1e83640 1 Processor -- start 2023-08-11T15:53:31.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.324+0000 7f32e1e83640 1 -- start start 2023-08-11T15:53:31.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.324+0000 7f32e1e83640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32dc06bdf0 0x7f32dc1f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.324+0000 7f32e1e83640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc151220 0x7f32dc150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32e1e83640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc15e680 0x7f32dc159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32e1e83640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc1f6710 con 0x7f32dc15e680 2023-08-11T15:53:31.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32e1e83640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc1f31d0 con 0x7f32dc151220 2023-08-11T15:53:31.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32e1e83640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0c6ec0 con 0x7f32dc06bdf0 2023-08-11T15:53:31.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32dbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc15e680 0x7f32dc159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32dbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc15e680 0x7f32dc159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52924/0 (socket says 172.21.15.27:52924) 2023-08-11T15:53:31.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32dbfff640 1 -- 172.21.15.27:0/3392642747 learned_addr learned my addr 172.21.15.27:0/3392642747 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:31.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32db7fe640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32dc06bdf0 0x7f32dc1f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32daffd640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc151220 0x7f32dc150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32dbfff640 1 -- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32dc06bdf0 msgr2=0x7f32dc1f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32dbfff640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32dc06bdf0 0x7f32dc1f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32dbfff640 1 -- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc151220 msgr2=0x7f32dc150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32dbfff640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc151220 0x7f32dc150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32dbfff640 1 -- 172.21.15.27:0/3392642747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc009400 con 0x7f32dc15e680 2023-08-11T15:53:31.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32daffd640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc151220 0x7f32dc150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:31.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.325+0000 7f32db7fe640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32dc06bdf0 0x7f32dc1f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:31.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.326+0000 7f32dbfff640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc15e680 0x7f32dc159450 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f32d400c780 tx=0x7f32d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:31.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.326+0000 7f32c3fff640 1 -- 172.21.15.27:0/3392642747 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4004b60 con 0x7f32dc15e680 2023-08-11T15:53:31.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.326+0000 7f32c3fff640 1 -- 172.21.15.27:0/3392642747 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d400ce90 con 0x7f32dc15e680 2023-08-11T15:53:31.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.326+0000 7f32c3fff640 1 -- 172.21.15.27:0/3392642747 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d400a850 con 0x7f32dc15e680 2023-08-11T15:53:31.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.326+0000 7f32e1e83640 1 -- 172.21.15.27:0/3392642747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32dc1f2650 con 0x7f32dc15e680 2023-08-11T15:53:31.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.326+0000 7f32e1e83640 1 -- 172.21.15.27:0/3392642747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32dc1f2c10 con 0x7f32dc15e680 2023-08-11T15:53:31.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.327+0000 7f32e1e83640 1 -- 172.21.15.27:0/3392642747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32dc04e420 con 0x7f32dc15e680 2023-08-11T15:53:31.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.328+0000 7f32c3fff640 1 -- 172.21.15.27:0/3392642747 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f32d400a9b0 con 0x7f32dc15e680 2023-08-11T15:53:31.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.329+0000 7f32c3fff640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32c806a050 0x7f32c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.329+0000 7f32c3fff640 1 -- 172.21.15.27:0/3392642747 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f32d4088990 con 0x7f32dc15e680 2023-08-11T15:53:31.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.330+0000 7f32db7fe640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32c806a050 0x7f32c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.331+0000 7f32db7fe640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32c806a050 0x7f32c806c510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f32cc035000 tx=0x7f32cc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:31.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.334+0000 7f32c3fff640 1 -- 172.21.15.27:0/3392642747 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f32d4053ea0 con 0x7f32dc15e680 2023-08-11T15:53:31.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.530+0000 7f32e1e83640 1 -- 172.21.15.27:0/3392642747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f32dc0affc0 con 0x7f32dc15e680 2023-08-11T15:53:31.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.531+0000 7f32c3fff640 1 -- 172.21.15.27:0/3392642747 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f32d40574c0 con 0x7f32dc15e680 2023-08-11T15:53:31.533 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-08-11T15:53:31.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 -- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32c806a050 msgr2=0x7f32c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32c806a050 0x7f32c806c510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f32cc035000 tx=0x7f32cc008030 comp rx=0 tx=0).stop 2023-08-11T15:53:31.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 -- 172.21.15.27:0/3392642747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc15e680 msgr2=0x7f32dc159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc15e680 0x7f32dc159450 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f32d400c780 tx=0x7f32d400cc40 comp rx=0 tx=0).stop 2023-08-11T15:53:31.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 -- 172.21.15.27:0/3392642747 shutdown_connections 2023-08-11T15:53:31.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f32c806a050 0x7f32c806c510 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32dc15e680 0x7f32dc159450 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc151220 0x7f32dc150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 --2- 172.21.15.27:0/3392642747 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32dc06bdf0 0x7f32dc1f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 -- 172.21.15.27:0/3392642747 >> 172.21.15.27:0/3392642747 conn(0x7f32dc0afbe0 msgr2=0x7f32dc15c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:31.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 -- 172.21.15.27:0/3392642747 shutdown_connections 2023-08-11T15:53:31.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.535+0000 7f32e1e83640 1 -- 172.21.15.27:0/3392642747 wait complete. 2023-08-11T15:53:31.554 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:53:31.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.787+0000 7f39a0c45640 1 Processor -- start 2023-08-11T15:53:31.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.787+0000 7f39a0c45640 1 -- start start 2023-08-11T15:53:31.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.788+0000 7f39a0c45640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c0cb430 0x7f399c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.788+0000 7f39a0c45640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f399c06f5d0 0x7f399c06fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.788+0000 7f39a0c45640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399c0c8a30 con 0x7f399c15a140 2023-08-11T15:53:31.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.789+0000 7f39a0c45640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399c06ffa0 con 0x7f399c06f5d0 2023-08-11T15:53:31.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.789+0000 7f39a0c45640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c06c0c0 con 0x7f399c0cb430 2023-08-11T15:53:31.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.789+0000 7f399ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f399c06f5d0 0x7f399c06fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.789+0000 7f399ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f399c06f5d0 0x7f399c06fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52926/0 (socket says 172.21.15.27:52926) 2023-08-11T15:53:31.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.789+0000 7f399ad9d640 1 -- 172.21.15.27:0/2894622897 learned_addr learned my addr 172.21.15.27:0/2894622897 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:31.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.789+0000 7f3999d9b640 1 --2- 172.21.15.27:0/2894622897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c0cb430 0x7f399c0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.789+0000 7f399ad9d640 1 -- 172.21.15.27:0/2894622897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c0cb430 msgr2=0x7f399c0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:31.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.789+0000 7f399ad9d640 1 --2- 172.21.15.27:0/2894622897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c0cb430 0x7f399c0b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.789+0000 7f399ad9d640 1 -- 172.21.15.27:0/2894622897 >> v1:172.21.15.119:6789/0 conn(0x7f399c15a140 legacy=0x7f399c06b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:31.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.789+0000 7f399ad9d640 1 -- 172.21.15.27:0/2894622897 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c06c3a0 con 0x7f399c06f5d0 2023-08-11T15:53:31.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.791+0000 7f399ad9d640 1 --2- 172.21.15.27:0/2894622897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 0x7f399c06fa60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3994009e50 tx=0x7f399402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365b1ceb562399a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:31.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.791+0000 7f399959a640 1 -- 172.21.15.27:0/2894622897 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399402ca70 con 0x7f399c06f5d0 2023-08-11T15:53:31.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.791+0000 7f399959a640 1 -- 172.21.15.27:0/2894622897 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399402cbd0 con 0x7f399c06f5d0 2023-08-11T15:53:31.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.791+0000 7f399959a640 1 -- 172.21.15.27:0/2894622897 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39940343f0 con 0x7f399c06f5d0 2023-08-11T15:53:31.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.791+0000 7f39a0c45640 1 -- 172.21.15.27:0/2894622897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 msgr2=0x7f399c06fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.791+0000 7f39a0c45640 1 --2- 172.21.15.27:0/2894622897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 0x7f399c06fa60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3994009e50 tx=0x7f399402bed0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.792+0000 7f39a0c45640 1 -- 172.21.15.27:0/2894622897 shutdown_connections 2023-08-11T15:53:31.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.792+0000 7f39a0c45640 1 --2- 172.21.15.27:0/2894622897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 0x7f399c06fa60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.792+0000 7f39a0c45640 1 --2- 172.21.15.27:0/2894622897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c0cb430 0x7f399c0b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.792+0000 7f39a0c45640 1 -- 172.21.15.27:0/2894622897 >> 172.21.15.27:0/2894622897 conn(0x7f399c0afb70 msgr2=0x7f399c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:31.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.792+0000 7f39a0c45640 1 -- 172.21.15.27:0/2894622897 shutdown_connections 2023-08-11T15:53:31.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.792+0000 7f39a0c45640 1 -- 172.21.15.27:0/2894622897 wait complete. 2023-08-11T15:53:31.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.793+0000 7f39a0c45640 1 Processor -- start 2023-08-11T15:53:31.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.794+0000 7f39a0c45640 1 -- start start 2023-08-11T15:53:31.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.794+0000 7f39a0c45640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 0x7f399c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.794+0000 7f39a0c45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c0cb430 0x7f399c0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.794+0000 7f39a0c45640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c15a140 0x7f399c0acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.794+0000 7f39a0c45640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0c6d00 con 0x7f399c06f5d0 2023-08-11T15:53:31.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.794+0000 7f39a0c45640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0c6460 con 0x7f399c0cb430 2023-08-11T15:53:31.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.794+0000 7f39a0c45640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0c6740 con 0x7f399c15a140 2023-08-11T15:53:31.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f399a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 0x7f399c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f399a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 0x7f399c158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52938/0 (socket says 172.21.15.27:52938) 2023-08-11T15:53:31.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f399a59c640 1 -- 172.21.15.27:0/1451619287 learned_addr learned my addr 172.21.15.27:0/1451619287 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:31.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f3999d9b640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c0cb430 0x7f399c0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f399ad9d640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c15a140 0x7f399c0acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f399a59c640 1 -- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c15a140 msgr2=0x7f399c0acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f399a59c640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c15a140 0x7f399c0acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f399a59c640 1 -- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c0cb430 msgr2=0x7f399c0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:31.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f399a59c640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c0cb430 0x7f399c0c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:31.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f399a59c640 1 -- 172.21.15.27:0/1451619287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3994009b30 con 0x7f399c06f5d0 2023-08-11T15:53:31.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.795+0000 7f399ad9d640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c15a140 0x7f399c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:31.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.796+0000 7f399a59c640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 0x7f399c158b30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f398c00b3e0 tx=0x7f398c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:31.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.796+0000 7f398affd640 1 -- 172.21.15.27:0/1451619287 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c011830 con 0x7f399c06f5d0 2023-08-11T15:53:31.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.796+0000 7f398affd640 1 -- 172.21.15.27:0/1451619287 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c0119b0 con 0x7f399c06f5d0 2023-08-11T15:53:31.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.796+0000 7f398affd640 1 -- 172.21.15.27:0/1451619287 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c011da0 con 0x7f399c06f5d0 2023-08-11T15:53:31.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.796+0000 7f39a0c45640 1 -- 172.21.15.27:0/1451619287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c0c6a20 con 0x7f399c06f5d0 2023-08-11T15:53:31.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.796+0000 7f39a0c45640 1 -- 172.21.15.27:0/1451619287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c159070 con 0x7f399c06f5d0 2023-08-11T15:53:31.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.798+0000 7f39a0c45640 1 -- 172.21.15.27:0/1451619287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c0c2cd0 con 0x7f399c06f5d0 2023-08-11T15:53:31.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.799+0000 7f398affd640 1 -- 172.21.15.27:0/1451619287 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f398c01a410 con 0x7f399c06f5d0 2023-08-11T15:53:31.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.800+0000 7f398affd640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f397c06a050 0x7f397c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:31.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.800+0000 7f398affd640 1 -- 172.21.15.27:0/1451619287 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f398c088680 con 0x7f399c06f5d0 2023-08-11T15:53:31.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.800+0000 7f3999d9b640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f397c06a050 0x7f397c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:31.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.802+0000 7f3999d9b640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f397c06a050 0x7f397c06c510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f399c0c7150 tx=0x7f3990008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:31.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:31.804+0000 7f398affd640 1 -- 172.21.15.27:0/1451619287 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f399c0c2cd0 con 0x7f399c06f5d0 2023-08-11T15:53:32.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.037+0000 7f39a0c45640 1 -- 172.21.15.27:0/1451619287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f399c0c30b0 con 0x7f399c06f5d0 2023-08-11T15:53:32.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.039+0000 7f398affd640 1 -- 172.21.15.27:0/1451619287 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f399c0c30b0 con 0x7f399c06f5d0 2023-08-11T15:53:32.040 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:32.041 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2294518038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2294518038},{"type":"v1","addr":"172.21.15.119:6836","nonce":2294518038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5188,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2104666724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2104666724},{"type":"v1","addr":"172.21.15.119:6837","nonce":2104666724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2379252997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2379252997},{"type":"v1","addr":"172.21.15.27:6833","nonce":2379252997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:53:09.220611+0000","modified":"2023-08-11T15:53:12.792531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5643},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5643":{"gid":5643,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/899405615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899405615},{"type":"v1","addr":"172.21.15.27:6835","nonce":899405615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-11T15:53:32.041 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 18 2023-08-11T15:53:32.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.041+0000 7f39a0c45640 1 -- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f397c06a050 msgr2=0x7f397c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.041+0000 7f39a0c45640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f397c06a050 0x7f397c06c510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f399c0c7150 tx=0x7f3990008040 comp rx=0 tx=0).stop 2023-08-11T15:53:32.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.041+0000 7f39a0c45640 1 -- 172.21.15.27:0/1451619287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 msgr2=0x7f399c158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.041+0000 7f39a0c45640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 0x7f399c158b30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f398c00b3e0 tx=0x7f398c00b8a0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.042+0000 7f39a0c45640 1 -- 172.21.15.27:0/1451619287 shutdown_connections 2023-08-11T15:53:32.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.042+0000 7f39a0c45640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f397c06a050 0x7f397c06c510 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.042+0000 7f39a0c45640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f399c15a140 0x7f399c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.042+0000 7f39a0c45640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f399c0cb430 0x7f399c0c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.042+0000 7f39a0c45640 1 --2- 172.21.15.27:0/1451619287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f399c06f5d0 0x7f399c158b30 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.042+0000 7f39a0c45640 1 -- 172.21.15.27:0/1451619287 >> 172.21.15.27:0/1451619287 conn(0x7f399c0afb70 msgr2=0x7f399c0c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:32.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.042+0000 7f39a0c45640 1 -- 172.21.15.27:0/1451619287 shutdown_connections 2023-08-11T15:53:32.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.042+0000 7f39a0c45640 1 -- 172.21.15.27:0/1451619287 wait complete. 2023-08-11T15:53:32.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:53:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.295+0000 7fb394d0c640 1 Processor -- start 2023-08-11T15:53:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.296+0000 7fb394d0c640 1 -- start start 2023-08-11T15:53:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.297+0000 7fb394d0c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb39015c620 0x7fb3900b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.297+0000 7fb394d0c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb3900b82f0 0x7fb3900cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.297+0000 7fb394d0c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb3900b55c0 0x7fb3900cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.297+0000 7fb394d0c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900ccc10 con 0x7fb3900b82f0 2023-08-11T15:53:32.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.297+0000 7fb394d0c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900cda30 con 0x7fb3900b55c0 2023-08-11T15:53:32.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.297+0000 7fb394d0c640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb390161110 con 0x7fb39015c620 2023-08-11T15:53:32.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.298+0000 7fb387fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb3900b82f0 0x7fb3900cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.298+0000 7fb387fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb3900b82f0 0x7fb3900cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52948/0 (socket says 172.21.15.27:52948) 2023-08-11T15:53:32.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.298+0000 7fb387fff640 1 -- 172.21.15.27:0/690127136 learned_addr learned my addr 172.21.15.27:0/690127136 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.298+0000 7fb38e59c640 1 --2- 172.21.15.27:0/690127136 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb39015c620 0x7fb3900b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.298+0000 7fb38ed9d640 1 --2- 172.21.15.27:0/690127136 >> v2:172.21.15.119:3300/0 conn(0x7fb3900b55c0 0x7fb3900cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.298+0000 7fb387fff640 1 -- 172.21.15.27:0/690127136 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb39015c620 msgr2=0x7fb3900b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.298+0000 7fb387fff640 1 --2- 172.21.15.27:0/690127136 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb39015c620 0x7fb3900b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.298+0000 7fb387fff640 1 -- 172.21.15.27:0/690127136 >> v2:172.21.15.119:3300/0 conn(0x7fb3900b55c0 msgr2=0x7fb3900cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.298+0000 7fb387fff640 1 --2- 172.21.15.27:0/690127136 >> v2:172.21.15.119:3300/0 conn(0x7fb3900b55c0 0x7fb3900cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.299+0000 7fb387fff640 1 -- 172.21.15.27:0/690127136 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900cce20 con 0x7fb3900b82f0 2023-08-11T15:53:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.300+0000 7fb387fff640 1 --2- 172.21.15.27:0/690127136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 0x7fb3900cf3f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb3900d1e10 tx=0x7fb37802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca82abe367df0be server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.300+0000 7fb38dd9b640 1 -- 172.21.15.27:0/690127136 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378031870 con 0x7fb3900b82f0 2023-08-11T15:53:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.300+0000 7fb38dd9b640 1 -- 172.21.15.27:0/690127136 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3780319d0 con 0x7fb3900b82f0 2023-08-11T15:53:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.300+0000 7fb38dd9b640 1 -- 172.21.15.27:0/690127136 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378031d60 con 0x7fb3900b82f0 2023-08-11T15:53:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.301+0000 7fb394d0c640 1 -- 172.21.15.27:0/690127136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 msgr2=0x7fb3900cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.301+0000 7fb394d0c640 1 --2- 172.21.15.27:0/690127136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 0x7fb3900cf3f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb3900d1e10 tx=0x7fb37802b910 comp rx=0 tx=0).stop 2023-08-11T15:53:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.301+0000 7fb394d0c640 1 -- 172.21.15.27:0/690127136 shutdown_connections 2023-08-11T15:53:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.301+0000 7fb394d0c640 1 --2- 172.21.15.27:0/690127136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 0x7fb3900cf3f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.301+0000 7fb394d0c640 1 --2- 172.21.15.27:0/690127136 >> v2:172.21.15.119:3300/0 conn(0x7fb3900b55c0 0x7fb3900cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.301+0000 7fb394d0c640 1 --2- 172.21.15.27:0/690127136 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb39015c620 0x7fb3900b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.301+0000 7fb394d0c640 1 -- 172.21.15.27:0/690127136 >> 172.21.15.27:0/690127136 conn(0x7fb390071110 msgr2=0x7fb3900714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.301+0000 7fb394d0c640 1 -- 172.21.15.27:0/690127136 shutdown_connections 2023-08-11T15:53:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.301+0000 7fb394d0c640 1 -- 172.21.15.27:0/690127136 wait complete. 2023-08-11T15:53:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.303+0000 7fb394d0c640 1 Processor -- start 2023-08-11T15:53:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.303+0000 7fb394d0c640 1 -- start start 2023-08-11T15:53:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.304+0000 7fb394d0c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3900b55c0 0x7fb3900cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.304+0000 7fb394d0c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 0x7fb39015f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.304+0000 7fb394d0c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39015c620 0x7fb3900ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.304+0000 7fb394d0c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900cbd60 con 0x7fb3900b82f0 2023-08-11T15:53:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.304+0000 7fb394d0c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900cae60 con 0x7fb39015c620 2023-08-11T15:53:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.304+0000 7fb394d0c640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900cb0e0 con 0x7fb3900b55c0 2023-08-11T15:53:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb387fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 0x7fb39015f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb387fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 0x7fb39015f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52954/0 (socket says 172.21.15.27:52954) 2023-08-11T15:53:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb387fff640 1 -- 172.21.15.27:0/1102279686 learned_addr learned my addr 172.21.15.27:0/1102279686 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb38ed9d640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39015c620 0x7fb3900ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb38e59c640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3900b55c0 0x7fb3900cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb387fff640 1 -- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3900b55c0 msgr2=0x7fb3900cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb387fff640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3900b55c0 0x7fb3900cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb387fff640 1 -- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39015c620 msgr2=0x7fb3900ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb387fff640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39015c620 0x7fb3900ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb387fff640 1 -- 172.21.15.27:0/1102279686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3780094d0 con 0x7fb3900b82f0 2023-08-11T15:53:32.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.305+0000 7fb38e59c640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3900b55c0 0x7fb3900cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.306+0000 7fb387fff640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 0x7fb39015f770 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb37802b950 tx=0x7fb3780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.306+0000 7fb386ffd640 1 -- 172.21.15.27:0/1102279686 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37803ad30 con 0x7fb3900b82f0 2023-08-11T15:53:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.306+0000 7fb386ffd640 1 -- 172.21.15.27:0/1102279686 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3780443f0 con 0x7fb3900b82f0 2023-08-11T15:53:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.306+0000 7fb386ffd640 1 -- 172.21.15.27:0/1102279686 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3780447a0 con 0x7fb3900b82f0 2023-08-11T15:53:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.306+0000 7fb394d0c640 1 -- 172.21.15.27:0/1102279686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3900cb3c0 con 0x7fb3900b82f0 2023-08-11T15:53:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.306+0000 7fb394d0c640 1 -- 172.21.15.27:0/1102279686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3900cb980 con 0x7fb3900b82f0 2023-08-11T15:53:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.307+0000 7fb394d0c640 1 -- 172.21.15.27:0/1102279686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3900cd3c0 con 0x7fb3900b82f0 2023-08-11T15:53:32.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.308+0000 7fb386ffd640 1 -- 172.21.15.27:0/1102279686 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb378044900 con 0x7fb3900b82f0 2023-08-11T15:53:32.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.309+0000 7fb386ffd640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb37006a050 0x7fb37006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.309+0000 7fb386ffd640 1 -- 172.21.15.27:0/1102279686 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb37802d040 con 0x7fb3900b82f0 2023-08-11T15:53:32.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.310+0000 7fb38e59c640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb37006a050 0x7fb37006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.311+0000 7fb38e59c640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb37006a050 0x7fb37006c510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb380006fd0 tx=0x7fb380008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:32.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.314+0000 7fb386ffd640 1 -- 172.21.15.27:0/1102279686 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fb3900cd3c0 con 0x7fb3900b82f0 2023-08-11T15:53:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.509+0000 7fb394d0c640 1 -- 172.21.15.27:0/1102279686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3900cd390 con 0x7fb3900b82f0 2023-08-11T15:53:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.513+0000 7fb386ffd640 1 -- 172.21.15.27:0/1102279686 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18500 (secure 0 0 0) 0x7fb3780762f0 con 0x7fb3900b82f0 2023-08-11T15:53:32.515 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:32.515 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":43,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:53:10.738370+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:53:06.898951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-11T15:53:08.258996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:53:32.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 -- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb37006a050 msgr2=0x7fb37006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb37006a050 0x7fb37006c510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb380006fd0 tx=0x7fb380008040 comp rx=0 tx=0).stop 2023-08-11T15:53:32.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 -- 172.21.15.27:0/1102279686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 msgr2=0x7fb39015f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 0x7fb39015f770 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb37802b950 tx=0x7fb3780046c0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 -- 172.21.15.27:0/1102279686 shutdown_connections 2023-08-11T15:53:32.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb37006a050 0x7fb37006c510 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39015c620 0x7fb3900ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3900b82f0 0x7fb39015f770 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 --2- 172.21.15.27:0/1102279686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3900b55c0 0x7fb3900cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 -- 172.21.15.27:0/1102279686 >> 172.21.15.27:0/1102279686 conn(0x7fb390071110 msgr2=0x7fb3900cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:32.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 -- 172.21.15.27:0/1102279686 shutdown_connections 2023-08-11T15:53:32.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.516+0000 7fb394d0c640 1 -- 172.21.15.27:0/1102279686 wait complete. 2023-08-11T15:53:32.536 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-11T15:53:32.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:53:32.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.775+0000 7f2e1ee4d640 1 Processor -- start 2023-08-11T15:53:32.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.775+0000 7f2e1ee4d640 1 -- start start 2023-08-11T15:53:32.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.777+0000 7f2e1ee4d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e180b55c0 0x7f2e180cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.777+0000 7f2e1ee4d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e180cf930 0x7f2e180d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.777+0000 7f2e1ee4d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e180cfd80 con 0x7f2e1815c620 2023-08-11T15:53:32.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.777+0000 7f2e1ee4d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e180ccc10 con 0x7f2e180b55c0 2023-08-11T15:53:32.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.777+0000 7f2e1ee4d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180cda30 con 0x7f2e180cf930 2023-08-11T15:53:32.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.777+0000 7f2e1cbe9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2e1815c620 0x7f2e180b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50056/0 (socket says 172.21.15.27:50056) 2023-08-11T15:53:32.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.777+0000 7f2e1cbe9640 1 -- 172.21.15.27:0/2393465807 learned_addr learned my addr 172.21.15.27:0/2393465807 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:32.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e17fff640 1 --2- 172.21.15.27:0/2393465807 >> v2:172.21.15.119:3300/0 conn(0x7f2e180b55c0 0x7f2e180cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e1d3ea640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e180cf930 0x7f2e180d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776457660 0 0) 0x7f2e0c001110 con 0x7f2e1815c620 2023-08-11T15:53:32.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e08003520 con 0x7f2e1815c620 2023-08-11T15:53:32.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2617908012 0 0) 0x7f2e08003520 con 0x7f2e1815c620 2023-08-11T15:53:32.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e180cf930 msgr2=0x7f2e180d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:32.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e180cf930 0x7f2e180d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 >> v2:172.21.15.119:3300/0 conn(0x7f2e180b55c0 msgr2=0x7f2e180cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:32.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 --2- 172.21.15.27:0/2393465807 >> v2:172.21.15.119:3300/0 conn(0x7f2e180b55c0 0x7f2e180cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18161110 con 0x7f2e1815c620 2023-08-11T15:53:32.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f2e0c001800 con 0x7f2e1815c620 2023-08-11T15:53:32.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 >> v1:172.21.15.27:6789/0 conn(0x7f2e1815c620 legacy=0x7f2e180b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e08003970 0x7f2e08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 0x7f2e08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.778+0000 7f2e177fe640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e180cf930 0x7f2e08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.779+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180ccc10 con 0x7f2e080047c0 2023-08-11T15:53:32.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.779+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180cda30 con 0x7f2e08003970 2023-08-11T15:53:32.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.779+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0800b980 con 0x7f2e180cf930 2023-08-11T15:53:32.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.779+0000 7f2e1d3ea640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e180cf930 0x7f2e08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.779+0000 7f2e1cbe9640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e08003970 0x7f2e08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.779+0000 7f2e17fff640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 0x7f2e08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.780+0000 7f2e17fff640 1 -- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e180cf930 msgr2=0x7f2e08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:32.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.780+0000 7f2e17fff640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e180cf930 0x7f2e08005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.780+0000 7f2e17fff640 1 -- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e08003970 msgr2=0x7f2e08003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:32.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.780+0000 7f2e17fff640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e08003970 0x7f2e08003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.780+0000 7f2e17fff640 1 -- 172.21.15.27:0/2393465807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e08003520 con 0x7f2e080047c0 2023-08-11T15:53:32.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.781+0000 7f2e17fff640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 0x7f2e08004c40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009ac0 tx=0x7f2e080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3efd4305d242d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:32.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.781+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0402c710 con 0x7f2e080047c0 2023-08-11T15:53:32.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.781+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0402c870 con 0x7f2e080047c0 2023-08-11T15:53:32.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.781+0000 7f2e177fe640 1 -- 172.21.15.27:0/2393465807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0402cc00 con 0x7f2e080047c0 2023-08-11T15:53:32.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.782+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/2393465807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 msgr2=0x7f2e08004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.782+0000 7f2e1ee4d640 1 --2- 172.21.15.27:0/2393465807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 0x7f2e08004c40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009ac0 tx=0x7f2e080057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.782+0000 7f2e1d3ea640 1 -- 172.21.15.27:0/2393465807 reap_dead start 2023-08-11T15:53:32.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.782+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/2393465807 shutdown_connections 2023-08-11T15:53:32.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.782+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/2393465807 >> 172.21.15.27:0/2393465807 conn(0x7f2e18071110 msgr2=0x7f2e180714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:32.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.782+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/2393465807 shutdown_connections 2023-08-11T15:53:32.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.782+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/2393465807 wait complete. 2023-08-11T15:53:32.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.784+0000 7f2e1ee4d640 1 Processor -- start 2023-08-11T15:53:32.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.784+0000 7f2e1ee4d640 1 -- start start 2023-08-11T15:53:32.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.785+0000 7f2e1ee4d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 0x7f2e180cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.785+0000 7f2e1ee4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180b3610 0x7f2e180b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.785+0000 7f2e1ee4d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e181568f0 0x7f2e18156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.785+0000 7f2e1ee4d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180b3fd0 con 0x7f2e080047c0 2023-08-11T15:53:32.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.785+0000 7f2e1ee4d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e18147570 con 0x7f2e180b3610 2023-08-11T15:53:32.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.785+0000 7f2e1ee4d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e18147850 con 0x7f2e181568f0 2023-08-11T15:53:32.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e1cbe9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 0x7f2e180cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e1cbe9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 0x7f2e180cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52962/0 (socket says 172.21.15.27:52962) 2023-08-11T15:53:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e1cbe9640 1 -- 172.21.15.27:0/1465777222 learned_addr learned my addr 172.21.15.27:0/1465777222 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e17fff640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180b3610 0x7f2e180b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e1d3ea640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e181568f0 0x7f2e18156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e1cbe9640 1 -- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e181568f0 msgr2=0x7f2e18156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e1cbe9640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e181568f0 0x7f2e18156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e1cbe9640 1 -- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180b3610 msgr2=0x7f2e180b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e1cbe9640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180b3610 0x7f2e180b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e1cbe9640 1 -- 172.21.15.27:0/1465777222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18147b30 con 0x7f2e080047c0 2023-08-11T15:53:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.786+0000 7f2e1cbe9640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 0x7f2e180cc100 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c007340 tx=0x7f2e0c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.787+0000 7f2e157fa640 1 -- 172.21.15.27:0/1465777222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c00f8c0 con 0x7f2e080047c0 2023-08-11T15:53:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.787+0000 7f2e157fa640 1 -- 172.21.15.27:0/1465777222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c00fa20 con 0x7f2e080047c0 2023-08-11T15:53:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.787+0000 7f2e157fa640 1 -- 172.21.15.27:0/1465777222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c00fdd0 con 0x7f2e080047c0 2023-08-11T15:53:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.787+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/1465777222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e18147e10 con 0x7f2e080047c0 2023-08-11T15:53:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.787+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/1465777222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e181572b0 con 0x7f2e080047c0 2023-08-11T15:53:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.788+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/1465777222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e181476b0 con 0x7f2e080047c0 2023-08-11T15:53:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.789+0000 7f2e157fa640 1 -- 172.21.15.27:0/1465777222 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e0c01d460 con 0x7f2e080047c0 2023-08-11T15:53:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.790+0000 7f2e157fa640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2dfc06a050 0x7f2dfc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.790+0000 7f2e157fa640 1 -- 172.21.15.27:0/1465777222 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2e0c0887e0 con 0x7f2e080047c0 2023-08-11T15:53:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.791+0000 7f2e17fff640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2dfc06a050 0x7f2dfc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.792+0000 7f2e17fff640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2dfc06a050 0x7f2dfc06c510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e04036000 tx=0x7f2e04030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:32.795+0000 7f2e157fa640 1 -- 172.21.15.27:0/1465777222 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f2e181476b0 con 0x7f2e080047c0 2023-08-11T15:53:33.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.028+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/1465777222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e18147990 con 0x7f2e080047c0 2023-08-11T15:53:33.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.029+0000 7f2e157fa640 1 -- 172.21.15.27:0/1465777222 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f2e18147990 con 0x7f2e080047c0 2023-08-11T15:53:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:33.031 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2294518038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2294518038},{"type":"v1","addr":"172.21.15.119:6836","nonce":2294518038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5188,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2104666724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2104666724},{"type":"v1","addr":"172.21.15.119:6837","nonce":2104666724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2379252997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2379252997},{"type":"v1","addr":"172.21.15.27:6833","nonce":2379252997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:53:09.220611+0000","modified":"2023-08-11T15:53:12.792531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5643},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5643":{"gid":5643,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/899405615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899405615},{"type":"v1","addr":"172.21.15.27:6835","nonce":899405615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-11T15:53:33.031 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 18 2023-08-11T15:53:33.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.031+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2dfc06a050 msgr2=0x7f2dfc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2dfc06a050 0x7f2dfc06c510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e04036000 tx=0x7f2e04030000 comp rx=0 tx=0).stop 2023-08-11T15:53:33.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/1465777222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 msgr2=0x7f2e180cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 0x7f2e180cc100 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c007340 tx=0x7f2e0c007800 comp rx=0 tx=0).stop 2023-08-11T15:53:33.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/1465777222 shutdown_connections 2023-08-11T15:53:33.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f2dfc06a050 0x7f2dfc06c510 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e181568f0 0x7f2e18156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180b3610 0x7f2e180b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 --2- 172.21.15.27:0/1465777222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e080047c0 0x7f2e180cc100 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/1465777222 >> 172.21.15.27:0/1465777222 conn(0x7f2e18071110 msgr2=0x7f2e1815bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:33.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/1465777222 shutdown_connections 2023-08-11T15:53:33.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.032+0000 7f2e1ee4d640 1 -- 172.21.15.27:0/1465777222 wait complete. 2023-08-11T15:53:33.054 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:53:33.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.293+0000 7fad38c71640 1 Processor -- start 2023-08-11T15:53:33.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.293+0000 7fad38c71640 1 -- start start 2023-08-11T15:53:33.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.294+0000 7fad38c71640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3415c590 0x7fad340b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.294+0000 7fad38c71640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad340d1480 0x7fad340d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.294+0000 7fad38c71640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad340d1e20 con 0x7fad340cd0e0 2023-08-11T15:53:33.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.294+0000 7fad38c71640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad340d0ed0 con 0x7fad340d1480 2023-08-11T15:53:33.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.294+0000 7fad38c71640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3415c970 con 0x7fad3415c590 2023-08-11T15:53:33.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.294+0000 7fad33fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fad340cd0e0 0x7fad340cd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50068/0 (socket says 172.21.15.27:50068) 2023-08-11T15:53:33.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad33fff640 1 -- 172.21.15.27:0/2661866670 learned_addr learned my addr 172.21.15.27:0/2661866670 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:33.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad32ffd640 1 --2- 172.21.15.27:0/2661866670 >> v2:172.21.15.119:3300/0 conn(0x7fad340d1480 0x7fad340d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad337fe640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3415c590 0x7fad340b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716662885 0 0) 0x7fad340d1e20 con 0x7fad340cd0e0 2023-08-11T15:53:33.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad20003520 con 0x7fad340cd0e0 2023-08-11T15:53:33.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3175021576 0 0) 0x7fad20003520 con 0x7fad340cd0e0 2023-08-11T15:53:33.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3415c590 msgr2=0x7fad340b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:33.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad327fc640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3415c590 0x7fad340b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 >> v2:172.21.15.119:3300/0 conn(0x7fad340d1480 msgr2=0x7fad340d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:33.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad327fc640 1 --2- 172.21.15.27:0/2661866670 >> v2:172.21.15.119:3300/0 conn(0x7fad340d1480 0x7fad340d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340d0250 con 0x7fad340cd0e0 2023-08-11T15:53:33.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fad2c001670 con 0x7fad340cd0e0 2023-08-11T15:53:33.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.295+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 >> v1:172.21.15.27:6789/0 conn(0x7fad340cd0e0 legacy=0x7fad340cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad327fc640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3415c590 0x7fad20003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad327fc640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad20003ef0 0x7fad200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad327fc640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad20004d00 0x7fad20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad340d0ed0 con 0x7fad20003ef0 2023-08-11T15:53:33.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3415c970 con 0x7fad20004d00 2023-08-11T15:53:33.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2000ba60 con 0x7fad3415c590 2023-08-11T15:53:33.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad32ffd640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad20003ef0 0x7fad200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad33fff640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad20004d00 0x7fad20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad32ffd640 1 -- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3415c590 msgr2=0x7fad20003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:33.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad337fe640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3415c590 0x7fad20003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad32ffd640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3415c590 0x7fad20003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad32ffd640 1 -- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad20004d00 msgr2=0x7fad20005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:33.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad32ffd640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad20004d00 0x7fad20005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad32ffd640 1 -- 172.21.15.27:0/2661866670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20003520 con 0x7fad20003ef0 2023-08-11T15:53:33.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad337fe640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3415c590 0x7fad20003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:33.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.296+0000 7fad33fff640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad20004d00 0x7fad20005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:33.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.298+0000 7fad32ffd640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad20003ef0 0x7fad200047c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009fc0 tx=0x7fad200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788c51f0a67fc690 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:33.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.298+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c02cc30 con 0x7fad20003ef0 2023-08-11T15:53:33.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.298+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c00f040 con 0x7fad20003ef0 2023-08-11T15:53:33.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.298+0000 7fad327fc640 1 -- 172.21.15.27:0/2661866670 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c034440 con 0x7fad20003ef0 2023-08-11T15:53:33.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.298+0000 7fad38c71640 1 -- 172.21.15.27:0/2661866670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad20003ef0 msgr2=0x7fad200047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.298+0000 7fad38c71640 1 --2- 172.21.15.27:0/2661866670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad20003ef0 0x7fad200047c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009fc0 tx=0x7fad200057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.298+0000 7fad33fff640 1 -- 172.21.15.27:0/2661866670 reap_dead start 2023-08-11T15:53:33.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.298+0000 7fad38c71640 1 -- 172.21.15.27:0/2661866670 shutdown_connections 2023-08-11T15:53:33.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.298+0000 7fad38c71640 1 -- 172.21.15.27:0/2661866670 >> 172.21.15.27:0/2661866670 conn(0x7fad3406a250 msgr2=0x7fad3406a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:33.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.299+0000 7fad38c71640 1 -- 172.21.15.27:0/2661866670 shutdown_connections 2023-08-11T15:53:33.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.299+0000 7fad38c71640 1 -- 172.21.15.27:0/2661866670 wait complete. 2023-08-11T15:53:33.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.300+0000 7fad38c71640 1 Processor -- start 2023-08-11T15:53:33.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.301+0000 7fad38c71640 1 -- start start 2023-08-11T15:53:33.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.301+0000 7fad38c71640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad20003ef0 0x7fad340c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad38c71640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad340b4900 0x7fad340b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad38c71640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3414f250 0x7fad340b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad38c71640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad340c7840 con 0x7fad340b4900 2023-08-11T15:53:33.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad38c71640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad340c7b20 con 0x7fad20003ef0 2023-08-11T15:53:33.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad38c71640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad340b4100 con 0x7fad3414f250 2023-08-11T15:53:33.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad32ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad340b4900 0x7fad340b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad32ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad340b4900 0x7fad340b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52982/0 (socket says 172.21.15.27:52982) 2023-08-11T15:53:33.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad32ffd640 1 -- 172.21.15.27:0/3299249858 learned_addr learned my addr 172.21.15.27:0/3299249858 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:33.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad337fe640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad20003ef0 0x7fad340c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad33fff640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3414f250 0x7fad340b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad32ffd640 1 -- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3414f250 msgr2=0x7fad340b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad32ffd640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3414f250 0x7fad340b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad32ffd640 1 -- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad20003ef0 msgr2=0x7fad340c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad32ffd640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad20003ef0 0x7fad340c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.302+0000 7fad32ffd640 1 -- 172.21.15.27:0/3299249858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340b43e0 con 0x7fad340b4900 2023-08-11T15:53:33.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.303+0000 7fad32ffd640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad340b4900 0x7fad340b1dc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00e230 tx=0x7fad1c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:33.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.303+0000 7fad2bfff640 1 -- 172.21.15.27:0/3299249858 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c034c50 con 0x7fad340b4900 2023-08-11T15:53:33.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.303+0000 7fad2bfff640 1 -- 172.21.15.27:0/3299249858 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c034db0 con 0x7fad340b4900 2023-08-11T15:53:33.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.303+0000 7fad2bfff640 1 -- 172.21.15.27:0/3299249858 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c003b90 con 0x7fad340b4900 2023-08-11T15:53:33.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.303+0000 7fad38c71640 1 -- 172.21.15.27:0/3299249858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad340b3db0 con 0x7fad340b4900 2023-08-11T15:53:33.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.303+0000 7fad38c71640 1 -- 172.21.15.27:0/3299249858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3414f7a0 con 0x7fad340b4900 2023-08-11T15:53:33.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.304+0000 7fad38c71640 1 -- 172.21.15.27:0/3299249858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad340c7ce0 con 0x7fad340b4900 2023-08-11T15:53:33.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.305+0000 7fad2bfff640 1 -- 172.21.15.27:0/3299249858 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fad1c040450 con 0x7fad340b4900 2023-08-11T15:53:33.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.306+0000 7fad2bfff640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fad1406a080 0x7fad1406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.306+0000 7fad2bfff640 1 -- 172.21.15.27:0/3299249858 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fad1c0aaf90 con 0x7fad340b4900 2023-08-11T15:53:33.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.307+0000 7fad337fe640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fad1406a080 0x7fad1406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.308+0000 7fad337fe640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fad1406a080 0x7fad1406c540 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fad240099c0 tx=0x7fad24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:33.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.311+0000 7fad2bfff640 1 -- 172.21.15.27:0/3299249858 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fad1c0dd7c0 con 0x7fad340b4900 2023-08-11T15:53:33.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.544+0000 7fad38c71640 1 -- 172.21.15.27:0/3299249858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad3404e420 con 0x7fad340b4900 2023-08-11T15:53:33.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.545+0000 7fad2bfff640 1 -- 172.21.15.27:0/3299249858 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fad1c0765d0 con 0x7fad340b4900 2023-08-11T15:53:33.547 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:33.547 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2294518038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2294518038},{"type":"v1","addr":"172.21.15.119:6836","nonce":2294518038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5188,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2104666724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2104666724},{"type":"v1","addr":"172.21.15.119:6837","nonce":2104666724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2379252997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2379252997},{"type":"v1","addr":"172.21.15.27:6833","nonce":2379252997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:53:09.220611+0000","modified":"2023-08-11T15:53:12.792531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5643},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5643":{"gid":5643,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/899405615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899405615},{"type":"v1","addr":"172.21.15.27:6835","nonce":899405615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-11T15:53:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 18 2023-08-11T15:53:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 -- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fad1406a080 msgr2=0x7fad1406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fad1406a080 0x7fad1406c540 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fad240099c0 tx=0x7fad24008040 comp rx=0 tx=0).stop 2023-08-11T15:53:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 -- 172.21.15.27:0/3299249858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad340b4900 msgr2=0x7fad340b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad340b4900 0x7fad340b1dc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00e230 tx=0x7fad1c009750 comp rx=0 tx=0).stop 2023-08-11T15:53:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 -- 172.21.15.27:0/3299249858 shutdown_connections 2023-08-11T15:53:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fad1406a080 0x7fad1406c540 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad3414f250 0x7fad340b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad340b4900 0x7fad340b1dc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 --2- 172.21.15.27:0/3299249858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad20003ef0 0x7fad340c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 -- 172.21.15.27:0/3299249858 >> 172.21.15.27:0/3299249858 conn(0x7fad3406a250 msgr2=0x7fad34159790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 -- 172.21.15.27:0/3299249858 shutdown_connections 2023-08-11T15:53:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.548+0000 7fad38c71640 1 -- 172.21.15.27:0/3299249858 wait complete. 2023-08-11T15:53:33.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:53:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.812+0000 7efe4bc87640 1 Processor -- start 2023-08-11T15:53:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.813+0000 7efe4bc87640 1 -- start start 2023-08-11T15:53:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.814+0000 7efe4bc87640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4415a140 0x7efe4406b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.814+0000 7efe4bc87640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe4406f5d0 0x7efe4406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.814+0000 7efe4bc87640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe440c8a30 con 0x7efe440cb430 2023-08-11T15:53:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.814+0000 7efe4bc87640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4406ffa0 con 0x7efe4406f5d0 2023-08-11T15:53:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.814+0000 7efe4bc87640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe440b1580 con 0x7efe4415a140 2023-08-11T15:53:33.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.815+0000 7efe4a224640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe4406f5d0 0x7efe4406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.815+0000 7efe4a224640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe4406f5d0 0x7efe4406fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52992/0 (socket says 172.21.15.27:52992) 2023-08-11T15:53:33.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.815+0000 7efe4a224640 1 -- 172.21.15.27:0/2317674551 learned_addr learned my addr 172.21.15.27:0/2317674551 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:33.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.815+0000 7efe49a23640 1 --2- 172.21.15.27:0/2317674551 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4415a140 0x7efe4406b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.815+0000 7efe4a224640 1 -- 172.21.15.27:0/2317674551 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4415a140 msgr2=0x7efe4406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:33.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.815+0000 7efe4a224640 1 --2- 172.21.15.27:0/2317674551 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4415a140 0x7efe4406b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.815+0000 7efe4a224640 1 -- 172.21.15.27:0/2317674551 >> v1:172.21.15.119:6789/0 conn(0x7efe440cb430 legacy=0x7efe440b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:33.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.815+0000 7efe4a224640 1 -- 172.21.15.27:0/2317674551 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4406c0c0 con 0x7efe4406f5d0 2023-08-11T15:53:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.817+0000 7efe4a224640 1 --2- 172.21.15.27:0/2317674551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4406f5d0 0x7efe4406fa60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009e80 tx=0x7efe3c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79130291b9d138a4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.817+0000 7efe48a21640 1 -- 172.21.15.27:0/2317674551 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c02ca50 con 0x7efe4406f5d0 2023-08-11T15:53:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.817+0000 7efe48a21640 1 -- 172.21.15.27:0/2317674551 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c02cbb0 con 0x7efe4406f5d0 2023-08-11T15:53:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.817+0000 7efe48a21640 1 -- 172.21.15.27:0/2317674551 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0343f0 con 0x7efe4406f5d0 2023-08-11T15:53:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.818+0000 7efe4bc87640 1 -- 172.21.15.27:0/2317674551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4406f5d0 msgr2=0x7efe4406fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.818+0000 7efe4bc87640 1 --2- 172.21.15.27:0/2317674551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4406f5d0 0x7efe4406fa60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009e80 tx=0x7efe3c02beb0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.818+0000 7efe4bc87640 1 -- 172.21.15.27:0/2317674551 shutdown_connections 2023-08-11T15:53:33.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.818+0000 7efe4bc87640 1 --2- 172.21.15.27:0/2317674551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4406f5d0 0x7efe4406fa60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.818+0000 7efe4bc87640 1 --2- 172.21.15.27:0/2317674551 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4415a140 0x7efe4406b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.818+0000 7efe4bc87640 1 -- 172.21.15.27:0/2317674551 >> 172.21.15.27:0/2317674551 conn(0x7efe440afb70 msgr2=0x7efe440aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:33.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.818+0000 7efe4bc87640 1 -- 172.21.15.27:0/2317674551 shutdown_connections 2023-08-11T15:53:33.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.818+0000 7efe4bc87640 1 -- 172.21.15.27:0/2317674551 wait complete. 2023-08-11T15:53:33.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.820+0000 7efe4bc87640 1 Processor -- start 2023-08-11T15:53:33.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.820+0000 7efe4bc87640 1 -- start start 2023-08-11T15:53:33.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.821+0000 7efe4bc87640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4406f5d0 0x7efe44157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.821+0000 7efe4bc87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe440cb430 0x7efe440cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.821+0000 7efe4bc87640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4415a140 0x7efe440c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.821+0000 7efe4bc87640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4415c6f0 con 0x7efe4415a140 2023-08-11T15:53:33.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.821+0000 7efe4bc87640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe44158260 con 0x7efe440cb430 2023-08-11T15:53:33.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.821+0000 7efe4bc87640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe440cc0b0 con 0x7efe4406f5d0 2023-08-11T15:53:33.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.821+0000 7efe4a224640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4415a140 0x7efe440c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.821+0000 7efe4a224640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4415a140 0x7efe440c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53004/0 (socket says 172.21.15.27:53004) 2023-08-11T15:53:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.821+0000 7efe4a224640 1 -- 172.21.15.27:0/3068941678 learned_addr learned my addr 172.21.15.27:0/3068941678 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.822+0000 7efe49a23640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4406f5d0 0x7efe44157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.822+0000 7efe49222640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe440cb430 0x7efe440cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.822+0000 7efe4a224640 1 -- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4406f5d0 msgr2=0x7efe44157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.822+0000 7efe4a224640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4406f5d0 0x7efe44157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.822+0000 7efe4a224640 1 -- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe440cb430 msgr2=0x7efe440cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.822+0000 7efe4a224640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe440cb430 0x7efe440cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.822+0000 7efe4a224640 1 -- 172.21.15.27:0/3068941678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c009b30 con 0x7efe4415a140 2023-08-11T15:53:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.822+0000 7efe4a224640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4415a140 0x7efe440c7560 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009fa0 tx=0x7efe3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.823+0000 7efe3a7fc640 1 -- 172.21.15.27:0/3068941678 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c009510 con 0x7efe4415a140 2023-08-11T15:53:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.823+0000 7efe3a7fc640 1 -- 172.21.15.27:0/3068941678 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c03e450 con 0x7efe4415a140 2023-08-11T15:53:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.823+0000 7efe3a7fc640 1 -- 172.21.15.27:0/3068941678 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c03e7e0 con 0x7efe4415a140 2023-08-11T15:53:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.823+0000 7efe4bc87640 1 -- 172.21.15.27:0/3068941678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe440cc380 con 0x7efe4415a140 2023-08-11T15:53:33.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.823+0000 7efe4bc87640 1 -- 172.21.15.27:0/3068941678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe440cc940 con 0x7efe4415a140 2023-08-11T15:53:33.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.824+0000 7efe4bc87640 1 -- 172.21.15.27:0/3068941678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe440ccad0 con 0x7efe4415a140 2023-08-11T15:53:33.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.825+0000 7efe3a7fc640 1 -- 172.21.15.27:0/3068941678 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efe3c03e940 con 0x7efe4415a140 2023-08-11T15:53:33.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.826+0000 7efe3a7fc640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe2806a050 0x7efe2806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:33.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.826+0000 7efe3a7fc640 1 -- 172.21.15.27:0/3068941678 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7efe3c0ab530 con 0x7efe4415a140 2023-08-11T15:53:33.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.827+0000 7efe49a23640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe2806a050 0x7efe2806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:33.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.828+0000 7efe49a23640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe2806a050 0x7efe2806c510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efe4000b9f0 tx=0x7efe40008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:33.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:33.831+0000 7efe3a7fc640 1 -- 172.21.15.27:0/3068941678 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7efe3c076a40 con 0x7efe4415a140 2023-08-11T15:53:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.026+0000 7efe4bc87640 1 -- 172.21.15.27:0/3068941678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe44073990 con 0x7efe4415a140 2023-08-11T15:53:34.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.028+0000 7efe3a7fc640 1 -- 172.21.15.27:0/3068941678 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18500 (secure 0 0 0) 0x7efe3c07a060 con 0x7efe4415a140 2023-08-11T15:53:34.030 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:34.033 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":43,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:53:10.738370+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:53:06.898951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-11T15:53:08.258996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:53:34.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 -- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe2806a050 msgr2=0x7efe2806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe2806a050 0x7efe2806c510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efe4000b9f0 tx=0x7efe40008040 comp rx=0 tx=0).stop 2023-08-11T15:53:34.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 -- 172.21.15.27:0/3068941678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4415a140 msgr2=0x7efe440c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4415a140 0x7efe440c7560 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009fa0 tx=0x7efe3c004a40 comp rx=0 tx=0).stop 2023-08-11T15:53:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 -- 172.21.15.27:0/3068941678 shutdown_connections 2023-08-11T15:53:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efe2806a050 0x7efe2806c510 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4415a140 0x7efe440c7560 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe440cb430 0x7efe440cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 --2- 172.21.15.27:0/3068941678 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4406f5d0 0x7efe44157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 -- 172.21.15.27:0/3068941678 >> 172.21.15.27:0/3068941678 conn(0x7efe440afb70 msgr2=0x7efe440c5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 -- 172.21.15.27:0/3068941678 shutdown_connections 2023-08-11T15:53:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.031+0000 7efe4bc87640 1 -- 172.21.15.27:0/3068941678 wait complete. 2023-08-11T15:53:34.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-11T15:53:34.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.291+0000 7f8dd2912640 1 Processor -- start 2023-08-11T15:53:34.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.291+0000 7f8dd2912640 1 -- start start 2023-08-11T15:53:34.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.293+0000 7f8dd2912640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8dcc15a3b0 0x7f8dcc071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.293+0000 7f8dd2912640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dcc06f6e0 0x7f8dcc06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.293+0000 7f8dd2912640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dcc0cb520 con 0x7f8dcc15df00 2023-08-11T15:53:34.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.293+0000 7f8dd2912640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dcc15eea0 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.293+0000 7f8dd2912640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0700b0 con 0x7f8dcc06f6e0 2023-08-11T15:53:34.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.294+0000 7f8dcbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8dcc15a3b0 0x7f8dcc071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.294+0000 7f8dd0eaf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dcc06f6e0 0x7f8dcc06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.294+0000 7f8dcbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8dcc15a3b0 0x7f8dcc071690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48578/0 (socket says 172.21.15.27:48578) 2023-08-11T15:53:34.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.294+0000 7f8dcbfff640 1 -- 172.21.15.27:0/449759052 learned_addr learned my addr 172.21.15.27:0/449759052 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:34.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.294+0000 7f8dcb7fe640 1 --1- 172.21.15.27:0/449759052 >> v1:172.21.15.119:6789/0 conn(0x7f8dcc15df00 0x7f8dcc0b0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:58570/0 (socket says 172.21.15.27:58570) 2023-08-11T15:53:34.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.294+0000 7f8dcbfff640 1 -- 172.21.15.27:0/449759052 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dcc06f6e0 msgr2=0x7f8dcc06fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:34.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.294+0000 7f8dcbfff640 1 --2- 172.21.15.27:0/449759052 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dcc06f6e0 0x7f8dcc06fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.294+0000 7f8dcbfff640 1 -- 172.21.15.27:0/449759052 >> v1:172.21.15.119:6789/0 conn(0x7f8dcc15df00 legacy=0x7f8dcc0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.294+0000 7f8dcbfff640 1 -- 172.21.15.27:0/449759052 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc15a790 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.296+0000 7f8dcbfff640 1 --2- 172.21.15.27:0/449759052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dcc15a3b0 0x7f8dcc071690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009870 tx=0x7f8dbc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7851d0f225944143 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:34.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.296+0000 7f8dcaffd640 1 -- 172.21.15.27:0/449759052 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc030840 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.296+0000 7f8dcaffd640 1 -- 172.21.15.27:0/449759052 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc0309a0 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.296+0000 7f8dcaffd640 1 -- 172.21.15.27:0/449759052 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc030d30 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.297+0000 7f8dd2912640 1 -- 172.21.15.27:0/449759052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dcc15a3b0 msgr2=0x7f8dcc071690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.297+0000 7f8dd2912640 1 --2- 172.21.15.27:0/449759052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dcc15a3b0 0x7f8dcc071690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009870 tx=0x7f8dbc02b980 comp rx=0 tx=0).stop 2023-08-11T15:53:34.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.297+0000 7f8dd2912640 1 -- 172.21.15.27:0/449759052 shutdown_connections 2023-08-11T15:53:34.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.297+0000 7f8dd2912640 1 --2- 172.21.15.27:0/449759052 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dcc06f6e0 0x7f8dcc06fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.297+0000 7f8dd2912640 1 --2- 172.21.15.27:0/449759052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dcc15a3b0 0x7f8dcc071690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.297+0000 7f8dd2912640 1 -- 172.21.15.27:0/449759052 >> 172.21.15.27:0/449759052 conn(0x7f8dcc069e90 msgr2=0x7f8dcc0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:34.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.297+0000 7f8dd2912640 1 -- 172.21.15.27:0/449759052 shutdown_connections 2023-08-11T15:53:34.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.297+0000 7f8dd2912640 1 -- 172.21.15.27:0/449759052 wait complete. 2023-08-11T15:53:34.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.300+0000 7f8dd2912640 1 Processor -- start 2023-08-11T15:53:34.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.300+0000 7f8dd2912640 1 -- start start 2023-08-11T15:53:34.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.301+0000 7f8dd2912640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dcc06f6e0 0x7f8dcc1f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.301+0000 7f8dd2912640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dcc15a3b0 0x7f8dcc15d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.301+0000 7f8dd2912640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dcc15df00 0x7f8dcc0c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.301+0000 7f8dd2912640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0c4770 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.301+0000 7f8dd2912640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0c4b90 con 0x7f8dcc15df00 2023-08-11T15:53:34.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.301+0000 7f8dd2912640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0c3af0 con 0x7f8dcc06f6e0 2023-08-11T15:53:34.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.301+0000 7f8dcb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dcc15a3b0 0x7f8dcc15d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.302+0000 7f8dcb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dcc15a3b0 0x7f8dcc15d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53006/0 (socket says 172.21.15.27:53006) 2023-08-11T15:53:34.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.302+0000 7f8dcb7fe640 1 -- 172.21.15.27:0/1232061875 learned_addr learned my addr 172.21.15.27:0/1232061875 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:34.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.302+0000 7f8dcbfff640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dcc06f6e0 0x7f8dcc1f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.302+0000 7f8dd0eaf640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dcc15df00 0x7f8dcc0c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.302+0000 7f8dcb7fe640 1 -- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dcc06f6e0 msgr2=0x7f8dcc1f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.302+0000 7f8dcb7fe640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dcc06f6e0 0x7f8dcc1f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.302+0000 7f8dcb7fe640 1 -- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dcc15df00 msgr2=0x7f8dcc0c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.302+0000 7f8dcb7fe640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dcc15df00 0x7f8dcc0c31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.302+0000 7f8dcb7fe640 1 -- 172.21.15.27:0/1232061875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0094d0 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.302+0000 7f8dcb7fe640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dcc15a3b0 0x7f8dcc15d1a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000c430 tx=0x7f8dc000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:34.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.303+0000 7f8dc8ff9640 1 -- 172.21.15.27:0/1232061875 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00074e0 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.303+0000 7f8dc8ff9640 1 -- 172.21.15.27:0/1232061875 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc0007640 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.303+0000 7f8dc8ff9640 1 -- 172.21.15.27:0/1232061875 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0004a00 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.303+0000 7f8dd2912640 1 -- 172.21.15.27:0/1232061875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dcc0c3dd0 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.303+0000 7f8dd2912640 1 -- 172.21.15.27:0/1232061875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dcc0c4390 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.304+0000 7f8dd2912640 1 -- 172.21.15.27:0/1232061875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dcc0c4520 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.305+0000 7f8dc8ff9640 1 -- 172.21.15.27:0/1232061875 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8dc0004b60 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.306+0000 7f8dc8ff9640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8db806a050 0x7f8db806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.306+0000 7f8dc8ff9640 1 -- 172.21.15.27:0/1232061875 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8dc00881b0 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.307+0000 7f8dcbfff640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8db806a050 0x7f8db806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.308+0000 7f8dcbfff640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8db806a050 0x7f8db806c510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc036000 tx=0x7f8dbc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:34.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.310+0000 7f8dc8ff9640 1 -- 172.21.15.27:0/1232061875 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f8dc00536c0 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.545+0000 7f8dd2912640 1 -- 172.21.15.27:0/1232061875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8dcc0c48b0 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.548 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-08-11T15:53:34.547+0000 7f6b67b3c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-11T15:53:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.557+0000 7f8dc8ff9640 1 -- 172.21.15.27:0/1232061875 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8dcc0c48b0 con 0x7f8dcc15a3b0 2023-08-11T15:53:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-11T15:53:34.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.560+0000 7f8dd2912640 1 -- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8db806a050 msgr2=0x7f8db806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.560+0000 7f8dd2912640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8db806a050 0x7f8db806c510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc036000 tx=0x7f8dbc008030 comp rx=0 tx=0).stop 2023-08-11T15:53:34.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.560+0000 7f8dd2912640 1 -- 172.21.15.27:0/1232061875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dcc15a3b0 msgr2=0x7f8dcc15d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.560+0000 7f8dd2912640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dcc15a3b0 0x7f8dcc15d1a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000c430 tx=0x7f8dc000c8f0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.560+0000 7f8dd2912640 1 -- 172.21.15.27:0/1232061875 shutdown_connections 2023-08-11T15:53:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.560+0000 7f8dd2912640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8db806a050 0x7f8db806c510 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.560+0000 7f8dd2912640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dcc15df00 0x7f8dcc0c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.560+0000 7f8dd2912640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dcc15a3b0 0x7f8dcc15d1a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.560+0000 7f8dd2912640 1 --2- 172.21.15.27:0/1232061875 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8dcc06f6e0 0x7f8dcc1f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.561+0000 7f8dd2912640 1 -- 172.21.15.27:0/1232061875 >> 172.21.15.27:0/1232061875 conn(0x7f8dcc069e90 msgr2=0x7f8dcc0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.561+0000 7f8dd2912640 1 -- 172.21.15.27:0/1232061875 shutdown_connections 2023-08-11T15:53:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.561+0000 7f8dd2912640 1 -- 172.21.15.27:0/1232061875 wait complete. 2023-08-11T15:53:34.577 INFO:tasks.ceph.mds.a.smithi027.stderr: -1612> 2023-08-11T15:53:14.662+0000 7f0d7beb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-11T15:53:34.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-11T15:53:34.659 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-08-11T15:53:34.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.795+0000 7eff057b2640 1 Processor -- start 2023-08-11T15:53:34.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.795+0000 7eff057b2640 1 -- start start 2023-08-11T15:53:34.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.796+0000 7eff057b2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 0x7eff000c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.796+0000 7eff057b2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7eff000cd190 0x7eff0006f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.796+0000 7eff057b2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff000c97e0 con 0x7eff0015a200 2023-08-11T15:53:34.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.796+0000 7eff057b2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff00142eb0 con 0x7eff000cd190 2023-08-11T15:53:34.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.796+0000 7eff057b2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0006d180 con 0x7eff00142ad0 2023-08-11T15:53:34.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.797+0000 7eff04fb1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7eff000cd190 0x7eff0006f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.797+0000 7efeff7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7eff0015a200 0x7eff000cca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50084/0 (socket says 172.21.15.27:50084) 2023-08-11T15:53:34.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.797+0000 7efeff7fe640 1 -- 172.21.15.27:0/3537915641 learned_addr learned my addr 172.21.15.27:0/3537915641 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:34.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.797+0000 7efefffff640 1 --2- 172.21.15.27:0/3537915641 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 0x7eff000c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.797+0000 7eff04fb1640 1 -- 172.21.15.27:0/3537915641 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 msgr2=0x7eff000c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:34.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.797+0000 7eff04fb1640 1 --2- 172.21.15.27:0/3537915641 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 0x7eff000c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.797+0000 7eff04fb1640 1 -- 172.21.15.27:0/3537915641 >> v1:172.21.15.27:6789/0 conn(0x7eff0015a200 legacy=0x7eff000cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.797+0000 7eff04fb1640 1 -- 172.21.15.27:0/3537915641 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff000c9ac0 con 0x7eff000cd190 2023-08-11T15:53:34.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.797+0000 7efefffff640 1 --2- 172.21.15.27:0/3537915641 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 0x7eff000c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:34.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.797+0000 7efefeffd640 1 -- 172.21.15.27:0/3537915641 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777085207 0 0) 0x7eff000c97e0 con 0x7eff0015a200 2023-08-11T15:53:34.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.798+0000 7eff04fb1640 1 --2- 172.21.15.27:0/3537915641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff000cd190 0x7eff0006f2e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efef8009d90 tx=0x7efef802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ef91cc8cdc8850 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:34.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7efefeffd640 1 -- 172.21.15.27:0/3537915641 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef802cc50 con 0x7eff000cd190 2023-08-11T15:53:34.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7efefeffd640 1 -- 172.21.15.27:0/3537915641 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef802cdb0 con 0x7eff000cd190 2023-08-11T15:53:34.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7efefeffd640 1 -- 172.21.15.27:0/3537915641 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef8034660 con 0x7eff000cd190 2023-08-11T15:53:34.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7eff057b2640 1 -- 172.21.15.27:0/3537915641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff000cd190 msgr2=0x7eff0006f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7eff057b2640 1 --2- 172.21.15.27:0/3537915641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff000cd190 0x7eff0006f2e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efef8009d90 tx=0x7efef802bea0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7eff057b2640 1 -- 172.21.15.27:0/3537915641 shutdown_connections 2023-08-11T15:53:34.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7eff057b2640 1 --2- 172.21.15.27:0/3537915641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff000cd190 0x7eff0006f2e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7eff057b2640 1 --2- 172.21.15.27:0/3537915641 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 0x7eff000c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7eff057b2640 1 -- 172.21.15.27:0/3537915641 >> 172.21.15.27:0/3537915641 conn(0x7eff000afa00 msgr2=0x7eff000afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:34.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7eff057b2640 1 -- 172.21.15.27:0/3537915641 shutdown_connections 2023-08-11T15:53:34.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.799+0000 7eff057b2640 1 -- 172.21.15.27:0/3537915641 wait complete. 2023-08-11T15:53:34.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.801+0000 7eff057b2640 1 Processor -- start 2023-08-11T15:53:34.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.801+0000 7eff057b2640 1 -- start start 2023-08-11T15:53:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.802+0000 7eff057b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff000cd190 0x7eff000c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.802+0000 7eff057b2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 0x7eff00166830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.802+0000 7eff057b2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0015a200 0x7eff00158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.802+0000 7eff057b2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff00166d70 con 0x7eff0015a200 2023-08-11T15:53:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.802+0000 7eff057b2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff000c7b30 con 0x7eff000cd190 2023-08-11T15:53:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.802+0000 7eff057b2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff000c7850 con 0x7eff00142ad0 2023-08-11T15:53:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7eff04fb1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0015a200 0x7eff00158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7eff04fb1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0015a200 0x7eff00158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53034/0 (socket says 172.21.15.27:53034) 2023-08-11T15:53:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7eff04fb1640 1 -- 172.21.15.27:0/1609299854 learned_addr learned my addr 172.21.15.27:0/1609299854 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7efeff7fe640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 0x7eff00166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7efefffff640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff000cd190 0x7eff000c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7eff04fb1640 1 -- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 msgr2=0x7eff00166830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7eff04fb1640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 0x7eff00166830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7eff04fb1640 1 -- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff000cd190 msgr2=0x7eff000c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7eff04fb1640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff000cd190 0x7eff000c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7eff04fb1640 1 -- 172.21.15.27:0/1609299854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef8009a70 con 0x7eff0015a200 2023-08-11T15:53:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7efefffff640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff000cd190 0x7eff000c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.803+0000 7eff04fb1640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0015a200 0x7eff00158e80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efef8009d90 tx=0x7efef8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.804+0000 7efefcff9640 1 -- 172.21.15.27:0/1609299854 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef8003b90 con 0x7eff0015a200 2023-08-11T15:53:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.804+0000 7efefcff9640 1 -- 172.21.15.27:0/1609299854 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef8003cf0 con 0x7eff0015a200 2023-08-11T15:53:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.804+0000 7efefcff9640 1 -- 172.21.15.27:0/1609299854 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef8004270 con 0x7eff0015a200 2023-08-11T15:53:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.804+0000 7eff057b2640 1 -- 172.21.15.27:0/1609299854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff000c2d10 con 0x7eff0015a200 2023-08-11T15:53:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.804+0000 7eff057b2640 1 -- 172.21.15.27:0/1609299854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff000c32d0 con 0x7eff0015a200 2023-08-11T15:53:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.805+0000 7eff057b2640 1 -- 172.21.15.27:0/1609299854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0004e420 con 0x7eff0015a200 2023-08-11T15:53:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.806+0000 7efefcff9640 1 -- 172.21.15.27:0/1609299854 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efef803b050 con 0x7eff0015a200 2023-08-11T15:53:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.807+0000 7efefcff9640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efeec06a050 0x7efeec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.807+0000 7efefcff9640 1 -- 172.21.15.27:0/1609299854 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6468+0+0 (secure 0 0 0) 0x7efef80aace0 con 0x7eff0015a200 2023-08-11T15:53:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.808+0000 7efefffff640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efeec06a050 0x7efeec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:34.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.809+0000 7efefffff640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efeec06a050 0x7efeec06c510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efef0006fd0 tx=0x7efef0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:34.812+0000 7efefcff9640 1 -- 172.21.15.27:0/1609299854 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7efef80762a0 con 0x7eff0015a200 2023-08-11T15:53:35.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.041+0000 7eff057b2640 1 -- 172.21.15.27:0/1609299854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7eff00141f60 con 0x7eff0015a200 2023-08-11T15:53:35.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.558+0000 7efefcff9640 1 -- 172.21.15.27:0/1609299854 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7efef80798c0 con 0x7eff0015a200 2023-08-11T15:53:35.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.561+0000 7eff057b2640 1 -- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efeec06a050 msgr2=0x7efeec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:35.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.561+0000 7eff057b2640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efeec06a050 0x7efeec06c510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efef0006fd0 tx=0x7efef0008040 comp rx=0 tx=0).stop 2023-08-11T15:53:35.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.562+0000 7eff057b2640 1 -- 172.21.15.27:0/1609299854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0015a200 msgr2=0x7eff00158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:35.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.562+0000 7eff057b2640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0015a200 0x7eff00158e80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efef8009d90 tx=0x7efef8004a40 comp rx=0 tx=0).stop 2023-08-11T15:53:35.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.562+0000 7eff057b2640 1 -- 172.21.15.27:0/1609299854 shutdown_connections 2023-08-11T15:53:35.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.562+0000 7eff057b2640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7efeec06a050 0x7efeec06c510 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.562+0000 7eff057b2640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff00142ad0 0x7eff00166830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.562+0000 7eff057b2640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0015a200 0x7eff00158e80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.562+0000 7eff057b2640 1 --2- 172.21.15.27:0/1609299854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff000cd190 0x7eff000c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.562+0000 7eff057b2640 1 -- 172.21.15.27:0/1609299854 >> 172.21.15.27:0/1609299854 conn(0x7eff000afa00 msgr2=0x7eff0006c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:35.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.562+0000 7eff057b2640 1 -- 172.21.15.27:0/1609299854 shutdown_connections 2023-08-11T15:53:35.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.562+0000 7eff057b2640 1 -- 172.21.15.27:0/1609299854 wait complete. 2023-08-11T15:53:35.586 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-11T15:53:35.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.829+0000 7f4fb23c9640 1 Processor -- start 2023-08-11T15:53:35.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.829+0000 7f4fb23c9640 1 -- start start 2023-08-11T15:53:35.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.831+0000 7f4fb23c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4fac15c620 0x7f4fac0b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:35.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.831+0000 7f4fb23c9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac0cda30 con 0x7f4fac0cfb00 2023-08-11T15:53:35.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.831+0000 7f4fb23c9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac161110 con 0x7f4fac0b82f0 2023-08-11T15:53:35.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.831+0000 7f4fb23c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac0ccc10 con 0x7f4fac15c620 2023-08-11T15:53:35.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.831+0000 7f4fb0966640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4fac0cfb00 0x7f4fac0d0b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50086/0 (socket says 172.21.15.27:50086) 2023-08-11T15:53:35.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.831+0000 7f4fb0966640 1 -- 172.21.15.27:0/2883323831 learned_addr learned my addr 172.21.15.27:0/2883323831 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:35.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.831+0000 7f4fabfff640 1 --2- 172.21.15.27:0/2883323831 >> v2:172.21.15.119:3300/0 conn(0x7f4fac15c620 0x7f4fac0b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:35.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964890158 0 0) 0x7f4fac0cda30 con 0x7f4fac0cfb00 2023-08-11T15:53:35.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f94003520 con 0x7f4fac0cfb00 2023-08-11T15:53:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1848393965 0 0) 0x7f4f94003520 con 0x7f4fac0cfb00 2023-08-11T15:53:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 >> v2:172.21.15.119:3300/0 conn(0x7f4fac15c620 msgr2=0x7f4fac0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 --2- 172.21.15.27:0/2883323831 >> v2:172.21.15.119:3300/0 conn(0x7f4fac15c620 0x7f4fac0b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 >> v1:172.21.15.119:6789/0 conn(0x7f4fac0b82f0 legacy=0x7f4fac0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac0cce20 con 0x7f4fac0cfb00 2023-08-11T15:53:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f4fa4001670 con 0x7f4fac0cfb00 2023-08-11T15:53:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4fabfff640 1 --2- 172.21.15.27:0/2883323831 >> v2:172.21.15.119:3300/0 conn(0x7f4fac15c620 0x7f4fac0b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 >> v1:172.21.15.27:6789/0 conn(0x7f4fac0cfb00 legacy=0x7f4fac0d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f94003970 0x7f4f94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f940047c0 0x7f4f94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f94005180 0x7f4f9400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0ccc10 con 0x7f4f940047c0 2023-08-11T15:53:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9400c020 con 0x7f4f94005180 2023-08-11T15:53:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.832+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9400c1e0 con 0x7f4f94003970 2023-08-11T15:53:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.833+0000 7f4fab7fe640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f940047c0 0x7f4f94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.833+0000 7f4fabfff640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f94003970 0x7f4f94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.833+0000 7f4fab7fe640 1 -- 172.21.15.27:0/2883323831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f94003970 msgr2=0x7f4f94003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.833+0000 7f4fb0966640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f94005180 0x7f4f9400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.833+0000 7f4fab7fe640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f94003970 0x7f4f94003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.833+0000 7f4fab7fe640 1 -- 172.21.15.27:0/2883323831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f94005180 msgr2=0x7f4f9400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.833+0000 7f4fab7fe640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f94005180 0x7f4f9400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.833+0000 7f4fb0966640 1 -- 172.21.15.27:0/2883323831 reap_dead start 2023-08-11T15:53:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.833+0000 7f4fab7fe640 1 -- 172.21.15.27:0/2883323831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94003520 con 0x7f4f940047c0 2023-08-11T15:53:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.834+0000 7f4fab7fe640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f940047c0 0x7f4f94004c40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0001720 tx=0x7f4f940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb2b80b15da9815 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.834+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa00052d0 con 0x7f4f940047c0 2023-08-11T15:53:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.835+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa002e050 con 0x7f4f940047c0 2023-08-11T15:53:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.835+0000 7f4faaffd640 1 -- 172.21.15.27:0/2883323831 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0003830 con 0x7f4f940047c0 2023-08-11T15:53:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.835+0000 7f4fb23c9640 1 -- 172.21.15.27:0/2883323831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f940047c0 msgr2=0x7f4f94004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:35.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.835+0000 7f4fb23c9640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f940047c0 0x7f4f94004c40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0001720 tx=0x7f4f940057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.835+0000 7f4fb23c9640 1 -- 172.21.15.27:0/2883323831 shutdown_connections 2023-08-11T15:53:35.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.835+0000 7f4fb23c9640 1 --2- 172.21.15.27:0/2883323831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f940047c0 0x7f4f94004c40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.835+0000 7f4fb23c9640 1 -- 172.21.15.27:0/2883323831 >> 172.21.15.27:0/2883323831 conn(0x7f4fac071110 msgr2=0x7f4fac0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:35.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.835+0000 7f4fb23c9640 1 -- 172.21.15.27:0/2883323831 shutdown_connections 2023-08-11T15:53:35.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.835+0000 7f4fb23c9640 1 -- 172.21.15.27:0/2883323831 wait complete. 2023-08-11T15:53:35.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.837+0000 7f4fb23c9640 1 Processor -- start 2023-08-11T15:53:35.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.837+0000 7f4fb23c9640 1 -- start start 2023-08-11T15:53:35.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.838+0000 7f4fb23c9640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f940047c0 0x7f4fac0c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:35.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.838+0000 7f4fb23c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0cbb20 0x7f4fac1f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:35.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.838+0000 7f4fb23c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fac1f56c0 0x7f4fac1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:35.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.838+0000 7f4fb23c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0c9010 con 0x7f4fac0cbb20 2023-08-11T15:53:35.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.838+0000 7f4fb23c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1f6080 con 0x7f4fac1f56c0 2023-08-11T15:53:35.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.838+0000 7f4fb23c9640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0caa80 con 0x7f4f940047c0 2023-08-11T15:53:35.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fab7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0cbb20 0x7f4fac1f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:35.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fab7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0cbb20 0x7f4fac1f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53050/0 (socket says 172.21.15.27:53050) 2023-08-11T15:53:35.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fab7fe640 1 -- 172.21.15.27:0/1582476469 learned_addr learned my addr 172.21.15.27:0/1582476469 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:35.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fb0966640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fac1f56c0 0x7f4fac1f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:35.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fab7fe640 1 -- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f940047c0 msgr2=0x7f4fac0c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:35.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fabfff640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f940047c0 0x7f4fac0c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:35.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fab7fe640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f940047c0 0x7f4fac0c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fab7fe640 1 -- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fac1f56c0 msgr2=0x7f4fac1f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fab7fe640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fac1f56c0 0x7f4fac1f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fab7fe640 1 -- 172.21.15.27:0/1582476469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac0cad60 con 0x7f4fac0cbb20 2023-08-11T15:53:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.839+0000 7f4fab7fe640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0cbb20 0x7f4fac1f2a70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0005040 tx=0x7f4fa0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.840+0000 7f4fa8ff9640 1 -- 172.21.15.27:0/1582476469 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0007a30 con 0x7f4fac0cbb20 2023-08-11T15:53:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.840+0000 7f4fa8ff9640 1 -- 172.21.15.27:0/1582476469 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa00069e0 con 0x7f4fac0cbb20 2023-08-11T15:53:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.840+0000 7f4fa8ff9640 1 -- 172.21.15.27:0/1582476469 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0006d70 con 0x7f4fac0cbb20 2023-08-11T15:53:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.840+0000 7f4fb23c9640 1 -- 172.21.15.27:0/1582476469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac0cb040 con 0x7f4fac0cbb20 2023-08-11T15:53:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.840+0000 7f4fb23c9640 1 -- 172.21.15.27:0/1582476469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac0cb600 con 0x7f4fac0cbb20 2023-08-11T15:53:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.841+0000 7f4fb23c9640 1 -- 172.21.15.27:0/1582476469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac0cb790 con 0x7f4fac0cbb20 2023-08-11T15:53:35.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.842+0000 7f4fa8ff9640 1 -- 172.21.15.27:0/1582476469 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4fa00035f0 con 0x7f4fac0cbb20 2023-08-11T15:53:35.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.843+0000 7f4fa8ff9640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f9806a080 0x7f4f9806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:35.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.843+0000 7f4fa8ff9640 1 -- 172.21.15.27:0/1582476469 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f4fa00aa8e0 con 0x7f4fac0cbb20 2023-08-11T15:53:35.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.844+0000 7f4fabfff640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f9806a080 0x7f4f9806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:35.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.845+0000 7f4fabfff640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f9806a080 0x7f4f9806c540 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4fac068660 tx=0x7f4f9c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:35.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:35.848+0000 7f4fa8ff9640 1 -- 172.21.15.27:0/1582476469 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f4fa0075d90 con 0x7f4fac0cbb20 2023-08-11T15:53:36.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.041+0000 7f4fb23c9640 1 -- 172.21.15.27:0/1582476469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4fac04e420 con 0x7f4fac0cbb20 2023-08-11T15:53:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.793+0000 7f4fa8ff9640 1 -- 172.21.15.27:0/1582476469 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4fa00793b0 con 0x7f4fac0cbb20 2023-08-11T15:53:36.797 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-08-11T15:53:36.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.801+0000 7f4fb23c9640 1 -- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f9806a080 msgr2=0x7f4f9806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:36.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.801+0000 7f4fb23c9640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f9806a080 0x7f4f9806c540 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4fac068660 tx=0x7f4f9c00b040 comp rx=0 tx=0).stop 2023-08-11T15:53:36.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.801+0000 7f4fb23c9640 1 -- 172.21.15.27:0/1582476469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0cbb20 msgr2=0x7f4fac1f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:36.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.801+0000 7f4fb23c9640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0cbb20 0x7f4fac1f2a70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0005040 tx=0x7f4fa0001120 comp rx=0 tx=0).stop 2023-08-11T15:53:36.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.801+0000 7f4fb23c9640 1 -- 172.21.15.27:0/1582476469 shutdown_connections 2023-08-11T15:53:36.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.801+0000 7f4fb23c9640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f4f9806a080 0x7f4f9806c540 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4fac068660 tx=0x7f4f9c00b040 comp rx=0 tx=0).stop 2023-08-11T15:53:36.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.801+0000 7f4fb23c9640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fac1f56c0 0x7f4fac1f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:36.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.801+0000 7f4fb23c9640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0cbb20 0x7f4fac1f2a70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:36.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.801+0000 7f4fb23c9640 1 --2- 172.21.15.27:0/1582476469 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f940047c0 0x7f4fac0c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:36.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.801+0000 7f4fb23c9640 1 -- 172.21.15.27:0/1582476469 >> 172.21.15.27:0/1582476469 conn(0x7f4fac071110 msgr2=0x7f4fac0b79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:36.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.802+0000 7f4fb23c9640 1 -- 172.21.15.27:0/1582476469 shutdown_connections 2023-08-11T15:53:36.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:36.802+0000 7f4fb23c9640 1 -- 172.21.15.27:0/1582476469 wait complete. 2023-08-11T15:53:36.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-11T15:53:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.064+0000 7f8a8f08a640 1 Processor -- start 2023-08-11T15:53:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.064+0000 7f8a8f08a640 1 -- start start 2023-08-11T15:53:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.066+0000 7f8a8f08a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a8815c5c0 0x7f8a88160a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.066+0000 7f8a8f08a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a880cf770 0x7f8a880ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.066+0000 7f8a8f08a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a880cdee0 con 0x7f8a8806be50 2023-08-11T15:53:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.066+0000 7f8a8f08a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8815ff40 con 0x7f8a8815c5c0 2023-08-11T15:53:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.066+0000 7f8a8f08a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a88160220 con 0x7f8a880cf770 2023-08-11T15:53:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.067+0000 7f8a8d887640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a8815c5c0 0x7f8a88160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.067+0000 7f8a8d887640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a8815c5c0 0x7f8a88160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53066/0 (socket says 172.21.15.27:53066) 2023-08-11T15:53:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.067+0000 7f8a8d887640 1 -- 172.21.15.27:0/4071702519 learned_addr learned my addr 172.21.15.27:0/4071702519 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.067+0000 7f8a8e889640 1 --2- 172.21.15.27:0/4071702519 >> v2:172.21.15.119:3300/0 conn(0x7f8a880cf770 0x7f8a880ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.067+0000 7f8a8d887640 1 -- 172.21.15.27:0/4071702519 >> v2:172.21.15.119:3300/0 conn(0x7f8a880cf770 msgr2=0x7f8a880ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.067+0000 7f8a8d887640 1 --2- 172.21.15.27:0/4071702519 >> v2:172.21.15.119:3300/0 conn(0x7f8a880cf770 0x7f8a880ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.067+0000 7f8a8d887640 1 -- 172.21.15.27:0/4071702519 >> v1:172.21.15.119:6789/0 conn(0x7f8a8806be50 legacy=0x7f8a8806c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.067+0000 7f8a8d887640 1 -- 172.21.15.27:0/4071702519 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a88160500 con 0x7f8a8815c5c0 2023-08-11T15:53:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.068+0000 7f8a8d887640 1 --2- 172.21.15.27:0/4071702519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 0x7f8a88160a20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8a88071520 tx=0x7f8a7802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9f9e4df4260b8f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.069+0000 7f8a8d086640 1 -- 172.21.15.27:0/4071702519 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78031870 con 0x7f8a8815c5c0 2023-08-11T15:53:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.069+0000 7f8a8d086640 1 -- 172.21.15.27:0/4071702519 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a780319d0 con 0x7f8a8815c5c0 2023-08-11T15:53:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.069+0000 7f8a8d086640 1 -- 172.21.15.27:0/4071702519 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78031d60 con 0x7f8a8815c5c0 2023-08-11T15:53:37.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.069+0000 7f8a8f08a640 1 -- 172.21.15.27:0/4071702519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 msgr2=0x7f8a88160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:37.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.069+0000 7f8a8f08a640 1 --2- 172.21.15.27:0/4071702519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 0x7f8a88160a20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8a88071520 tx=0x7f8a7802b910 comp rx=0 tx=0).stop 2023-08-11T15:53:37.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.070+0000 7f8a8f08a640 1 -- 172.21.15.27:0/4071702519 shutdown_connections 2023-08-11T15:53:37.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.070+0000 7f8a8f08a640 1 --2- 172.21.15.27:0/4071702519 >> v2:172.21.15.119:3300/0 conn(0x7f8a880cf770 0x7f8a880ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:37.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.070+0000 7f8a8f08a640 1 --2- 172.21.15.27:0/4071702519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 0x7f8a88160a20 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:37.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.070+0000 7f8a8f08a640 1 -- 172.21.15.27:0/4071702519 >> 172.21.15.27:0/4071702519 conn(0x7f8a880cffa0 msgr2=0x7f8a880d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.070+0000 7f8a8f08a640 1 -- 172.21.15.27:0/4071702519 shutdown_connections 2023-08-11T15:53:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.070+0000 7f8a8f08a640 1 -- 172.21.15.27:0/4071702519 wait complete. 2023-08-11T15:53:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.071+0000 7f8a8f08a640 1 Processor -- start 2023-08-11T15:53:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.072+0000 7f8a8f08a640 1 -- start start 2023-08-11T15:53:37.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8f08a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a8806be50 0x7f8a881506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:37.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8f08a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a880cf770 0x7f8a88152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8f08a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 0x7f8a88148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8f08a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880c9590 con 0x7f8a8815c5c0 2023-08-11T15:53:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8f08a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880c8980 con 0x7f8a880cf770 2023-08-11T15:53:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8f08a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880c8c60 con 0x7f8a8806be50 2023-08-11T15:53:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8e889640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 0x7f8a88148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8e889640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 0x7f8a88148550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53078/0 (socket says 172.21.15.27:53078) 2023-08-11T15:53:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8e889640 1 -- 172.21.15.27:0/3712789024 learned_addr learned my addr 172.21.15.27:0/3712789024 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8d887640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a880cf770 0x7f8a88152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8e088640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a8806be50 0x7f8a881506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8e889640 1 -- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a8806be50 msgr2=0x7f8a881506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.073+0000 7f8a8e889640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a8806be50 0x7f8a881506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:37.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a8e889640 1 -- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a880cf770 msgr2=0x7f8a88152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:37.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a8e889640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a880cf770 0x7f8a88152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:37.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a8e889640 1 -- 172.21.15.27:0/3712789024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a780094d0 con 0x7f8a8815c5c0 2023-08-11T15:53:37.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a8e088640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a8806be50 0x7f8a881506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:37.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a8d887640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a880cf770 0x7f8a88152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:37.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a8e889640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 0x7f8a88148550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000efc0 tx=0x7f8a8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:37.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a7effd640 1 -- 172.21.15.27:0/3712789024 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80014480 con 0x7f8a8815c5c0 2023-08-11T15:53:37.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a7effd640 1 -- 172.21.15.27:0/3712789024 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8000f040 con 0x7f8a8815c5c0 2023-08-11T15:53:37.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a7effd640 1 -- 172.21.15.27:0/3712789024 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80014820 con 0x7f8a8815c5c0 2023-08-11T15:53:37.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a8f08a640 1 -- 172.21.15.27:0/3712789024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a880c8f40 con 0x7f8a8815c5c0 2023-08-11T15:53:37.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.074+0000 7f8a8f08a640 1 -- 172.21.15.27:0/3712789024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a88152970 con 0x7f8a8815c5c0 2023-08-11T15:53:37.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.076+0000 7f8a8f08a640 1 -- 172.21.15.27:0/3712789024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a880c9700 con 0x7f8a8815c5c0 2023-08-11T15:53:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.077+0000 7f8a7effd640 1 -- 172.21.15.27:0/3712789024 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8a80014980 con 0x7f8a8815c5c0 2023-08-11T15:53:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.078+0000 7f8a7effd640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8a6c06a050 0x7f8a6c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:37.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.078+0000 7f8a7effd640 1 -- 172.21.15.27:0/3712789024 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6030+0+0 (secure 0 0 0) 0x7f8a800892a0 con 0x7f8a8815c5c0 2023-08-11T15:53:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.079+0000 7f8a8e088640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8a6c06a050 0x7f8a6c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.080+0000 7f8a8e088640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8a6c06a050 0x7f8a6c06c510 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a84003eb0 tx=0x7f8a84009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:37.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.083+0000 7f8a7effd640 1 -- 172.21.15.27:0/3712789024 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f8a80054890 con 0x7f8a8815c5c0 2023-08-11T15:53:37.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.276+0000 7f8a8f08a640 1 -- 172.21.15.27:0/3712789024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8a8804e420 con 0x7f8a8815c5c0 2023-08-11T15:53:37.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.796+0000 7f8a7effd640 1 -- 172.21.15.27:0/3712789024 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8a80057eb0 con 0x7f8a8815c5c0 2023-08-11T15:53:37.798 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-08-11T15:53:37.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.801+0000 7f8a8f08a640 1 -- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8a6c06a050 msgr2=0x7f8a6c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:37.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.801+0000 7f8a8f08a640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8a6c06a050 0x7f8a6c06c510 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a84003eb0 tx=0x7f8a84009040 comp rx=0 tx=0).stop 2023-08-11T15:53:37.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.801+0000 7f8a8f08a640 1 -- 172.21.15.27:0/3712789024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 msgr2=0x7f8a88148550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:37.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.801+0000 7f8a8f08a640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 0x7f8a88148550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000efc0 tx=0x7f8a8000c490 comp rx=0 tx=0).stop 2023-08-11T15:53:37.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.801+0000 7f8a8f08a640 1 -- 172.21.15.27:0/3712789024 shutdown_connections 2023-08-11T15:53:37.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.801+0000 7f8a8f08a640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8a6c06a050 0x7f8a6c06c510 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:37.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.801+0000 7f8a8f08a640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a8815c5c0 0x7f8a88148550 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:37.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.801+0000 7f8a8f08a640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a880cf770 0x7f8a88152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:37.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.801+0000 7f8a8f08a640 1 --2- 172.21.15.27:0/3712789024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a8806be50 0x7f8a881506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:37.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.801+0000 7f8a8f08a640 1 -- 172.21.15.27:0/3712789024 >> 172.21.15.27:0/3712789024 conn(0x7f8a880cffa0 msgr2=0x7f8a8806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:37.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.802+0000 7f8a8f08a640 1 -- 172.21.15.27:0/3712789024 shutdown_connections 2023-08-11T15:53:37.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:37.802+0000 7f8a8f08a640 1 -- 172.21.15.27:0/3712789024 wait complete. 2023-08-11T15:53:37.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-11T15:53:38.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.058+0000 7fbc3f59e640 1 Processor -- start 2023-08-11T15:53:38.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.059+0000 7fbc3f59e640 1 -- start start 2023-08-11T15:53:38.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.060+0000 7fbc3f59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc400cd660 0x7fbc400d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.060+0000 7fbc3f59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbc400cebb0 0x7fbc400cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.060+0000 7fbc3f59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc400cf560 con 0x7fbc4015c450 2023-08-11T15:53:38.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.060+0000 7fbc3f59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4015c830 con 0x7fbc400cebb0 2023-08-11T15:53:38.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.060+0000 7fbc3f59e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc400d1340 con 0x7fbc400cd660 2023-08-11T15:53:38.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.061+0000 7fbc3e59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbc4015c450 0x7fbc400d1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50092/0 (socket says 172.21.15.27:50092) 2023-08-11T15:53:38.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.061+0000 7fbc3e59c640 1 -- 172.21.15.27:0/1684650116 learned_addr learned my addr 172.21.15.27:0/1684650116 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:38.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.061+0000 7fbc3ed9d640 1 --2- 172.21.15.27:0/1684650116 >> v2:172.21.15.27:3300/0 conn(0x7fbc400cebb0 0x7fbc400cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.061+0000 7fbc3dd9b640 1 --2- 172.21.15.27:0/1684650116 >> v2:172.21.15.119:3300/0 conn(0x7fbc400cd660 0x7fbc400d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.061+0000 7fbc3d59a640 1 -- 172.21.15.27:0/1684650116 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205841462 0 0) 0x7fbc400cf560 con 0x7fbc4015c450 2023-08-11T15:53:38.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.061+0000 7fbc3d59a640 1 -- 172.21.15.27:0/1684650116 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc28003520 con 0x7fbc4015c450 2023-08-11T15:53:38.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.061+0000 7fbc3ed9d640 1 -- 172.21.15.27:0/1684650116 >> v2:172.21.15.119:3300/0 conn(0x7fbc400cd660 msgr2=0x7fbc400d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:38.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.061+0000 7fbc3ed9d640 1 --2- 172.21.15.27:0/1684650116 >> v2:172.21.15.119:3300/0 conn(0x7fbc400cd660 0x7fbc400d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.061+0000 7fbc3ed9d640 1 -- 172.21.15.27:0/1684650116 >> v1:172.21.15.27:6789/0 conn(0x7fbc4015c450 legacy=0x7fbc400d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:38.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.061+0000 7fbc3ed9d640 1 -- 172.21.15.27:0/1684650116 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc400cfda0 con 0x7fbc400cebb0 2023-08-11T15:53:38.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.063+0000 7fbc3ed9d640 1 --2- 172.21.15.27:0/1684650116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc400cebb0 0x7fbc400cf020 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbc400ce810 tx=0x7fbc3802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c242381209d8c2f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:38.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.063+0000 7fbc3d59a640 1 -- 172.21.15.27:0/1684650116 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3802e070 con 0x7fbc400cebb0 2023-08-11T15:53:38.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.063+0000 7fbc3d59a640 1 -- 172.21.15.27:0/1684650116 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc380315a0 con 0x7fbc400cebb0 2023-08-11T15:53:38.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.063+0000 7fbc3d59a640 1 -- 172.21.15.27:0/1684650116 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38031910 con 0x7fbc400cebb0 2023-08-11T15:53:38.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.064+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1684650116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc400cebb0 msgr2=0x7fbc400cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:38.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.064+0000 7fbc3f59e640 1 --2- 172.21.15.27:0/1684650116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc400cebb0 0x7fbc400cf020 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbc400ce810 tx=0x7fbc3802b840 comp rx=0 tx=0).stop 2023-08-11T15:53:38.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.064+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1684650116 shutdown_connections 2023-08-11T15:53:38.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.064+0000 7fbc3f59e640 1 --2- 172.21.15.27:0/1684650116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc400cebb0 0x7fbc400cf020 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.065+0000 7fbc3f59e640 1 --2- 172.21.15.27:0/1684650116 >> v2:172.21.15.119:3300/0 conn(0x7fbc400cd660 0x7fbc400d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.065+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1684650116 >> 172.21.15.27:0/1684650116 conn(0x7fbc4006a270 msgr2=0x7fbc4006a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:38.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.065+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1684650116 shutdown_connections 2023-08-11T15:53:38.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.065+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1684650116 wait complete. 2023-08-11T15:53:38.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.068+0000 7fbc3f59e640 1 Processor -- start 2023-08-11T15:53:38.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.069+0000 7fbc3f59e640 1 -- start start 2023-08-11T15:53:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.071+0000 7fbc3f59e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc400cebb0 0x7fbc400b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.071+0000 7fbc3f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc4015c450 0x7fbc4015c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.071+0000 7fbc3f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4015ceb0 0x7fbc401e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.071+0000 7fbc3f59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400b2740 con 0x7fbc4015c450 2023-08-11T15:53:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.071+0000 7fbc3f59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400b0680 con 0x7fbc4015ceb0 2023-08-11T15:53:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.071+0000 7fbc3f59e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4015d430 con 0x7fbc400cebb0 2023-08-11T15:53:38.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.072+0000 7fbc3dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc4015c450 0x7fbc4015c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.072+0000 7fbc3dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc4015c450 0x7fbc4015c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53104/0 (socket says 172.21.15.27:53104) 2023-08-11T15:53:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.072+0000 7fbc3dd9b640 1 -- 172.21.15.27:0/1998179385 learned_addr learned my addr 172.21.15.27:0/1998179385 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.072+0000 7fbc3ed9d640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4015ceb0 0x7fbc401e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.072+0000 7fbc3e59c640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc400cebb0 0x7fbc400b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.073+0000 7fbc3dd9b640 1 -- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc400cebb0 msgr2=0x7fbc400b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.073+0000 7fbc3dd9b640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc400cebb0 0x7fbc400b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.073+0000 7fbc3dd9b640 1 -- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4015ceb0 msgr2=0x7fbc401e4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.073+0000 7fbc3dd9b640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4015ceb0 0x7fbc401e4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.073+0000 7fbc3dd9b640 1 -- 172.21.15.27:0/1998179385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38009400 con 0x7fbc4015c450 2023-08-11T15:53:38.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.073+0000 7fbc3dd9b640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc4015c450 0x7fbc4015c970 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400d830 tx=0x7fbc3400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.074+0000 7fbc2effd640 1 -- 172.21.15.27:0/1998179385 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400b6c0 con 0x7fbc4015c450 2023-08-11T15:53:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.074+0000 7fbc2effd640 1 -- 172.21.15.27:0/1998179385 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3400b820 con 0x7fbc4015c450 2023-08-11T15:53:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.074+0000 7fbc2effd640 1 -- 172.21.15.27:0/1998179385 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400bbb0 con 0x7fbc4015c450 2023-08-11T15:53:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.074+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1998179385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4015d710 con 0x7fbc4015c450 2023-08-11T15:53:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.074+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1998179385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4015dcd0 con 0x7fbc4015c450 2023-08-11T15:53:38.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.076+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1998179385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4004e420 con 0x7fbc4015c450 2023-08-11T15:53:38.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.077+0000 7fbc2effd640 1 -- 172.21.15.27:0/1998179385 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbc3401e460 con 0x7fbc4015c450 2023-08-11T15:53:38.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.078+0000 7fbc2effd640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc2006a080 0x7fbc2006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.078+0000 7fbc2effd640 1 -- 172.21.15.27:0/1998179385 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fbc34088b00 con 0x7fbc4015c450 2023-08-11T15:53:38.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.078+0000 7fbc3e59c640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc2006a080 0x7fbc2006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.080+0000 7fbc3e59c640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc2006a080 0x7fbc2006c540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbc30001560 tx=0x7fbc30009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:38.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.082+0000 7fbc2effd640 1 -- 172.21.15.27:0/1998179385 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fbc3408ae60 con 0x7fbc4015c450 2023-08-11T15:53:38.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.275+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1998179385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbc400d0040 con 0x7fbc4015c450 2023-08-11T15:53:38.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.280+0000 7fbc2effd640 1 -- 172.21.15.27:0/1998179385 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbc34054330 con 0x7fbc4015c450 2023-08-11T15:53:38.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.283+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc2006a080 msgr2=0x7fbc2006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:38.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.283+0000 7fbc3f59e640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc2006a080 0x7fbc2006c540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbc30001560 tx=0x7fbc30009040 comp rx=0 tx=0).stop 2023-08-11T15:53:38.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.283+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1998179385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc4015c450 msgr2=0x7fbc4015c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:38.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.283+0000 7fbc3f59e640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc4015c450 0x7fbc4015c970 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400d830 tx=0x7fbc3400dcf0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.283+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1998179385 shutdown_connections 2023-08-11T15:53:38.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.283+0000 7fbc3f59e640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fbc2006a080 0x7fbc2006c540 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.283+0000 7fbc3f59e640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4015ceb0 0x7fbc401e4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.283+0000 7fbc3f59e640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc4015c450 0x7fbc4015c970 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.283+0000 7fbc3f59e640 1 --2- 172.21.15.27:0/1998179385 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc400cebb0 0x7fbc400b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.283+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1998179385 >> 172.21.15.27:0/1998179385 conn(0x7fbc4006a270 msgr2=0x7fbc400c92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:38.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.284+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1998179385 shutdown_connections 2023-08-11T15:53:38.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.284+0000 7fbc3f59e640 1 -- 172.21.15.27:0/1998179385 wait complete. 2023-08-11T15:53:38.302 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-08-11T15:53:38.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.531+0000 7fb55359e640 1 Processor -- start 2023-08-11T15:53:38.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.531+0000 7fb55359e640 1 -- start start 2023-08-11T15:53:38.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.533+0000 7fb55359e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb55415a020 0x7fb55415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.533+0000 7fb55359e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 0x7fb554071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.533+0000 7fb55359e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5540c85f0 con 0x7fb55406f5d0 2023-08-11T15:53:38.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.533+0000 7fb55359e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55415eae0 con 0x7fb55415a020 2023-08-11T15:53:38.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.533+0000 7fb55359e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb554071980 con 0x7fb5540c9600 2023-08-11T15:53:38.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.533+0000 7fb55259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb55415a020 0x7fb55415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.534+0000 7fb549d9b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 0x7fb554071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.534+0000 7fb55259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb55415a020 0x7fb55415e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48710/0 (socket says 172.21.15.27:48710) 2023-08-11T15:53:38.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.534+0000 7fb55259c640 1 -- 172.21.15.27:0/4238711632 learned_addr learned my addr 172.21.15.27:0/4238711632 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:38.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.534+0000 7fb55259c640 1 -- 172.21.15.27:0/4238711632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 msgr2=0x7fb554071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:38.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.534+0000 7fb55259c640 1 --2- 172.21.15.27:0/4238711632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 0x7fb554071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.534+0000 7fb55259c640 1 -- 172.21.15.27:0/4238711632 >> v1:172.21.15.119:6789/0 conn(0x7fb55406f5d0 legacy=0x7fb55406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:38.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.534+0000 7fb55259c640 1 -- 172.21.15.27:0/4238711632 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540c8840 con 0x7fb55415a020 2023-08-11T15:53:38.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.536+0000 7fb55259c640 1 --2- 172.21.15.27:0/4238711632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55415a020 0x7fb55415e5a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb544009e50 tx=0x7fb54402bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba38d40ce93a0a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:38.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.536+0000 7fb551d9b640 1 -- 172.21.15.27:0/4238711632 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54402ca80 con 0x7fb55415a020 2023-08-11T15:53:38.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.536+0000 7fb551d9b640 1 -- 172.21.15.27:0/4238711632 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54402cbe0 con 0x7fb55415a020 2023-08-11T15:53:38.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.536+0000 7fb551d9b640 1 -- 172.21.15.27:0/4238711632 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5440343f0 con 0x7fb55415a020 2023-08-11T15:53:38.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.537+0000 7fb55359e640 1 -- 172.21.15.27:0/4238711632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55415a020 msgr2=0x7fb55415e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:38.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.537+0000 7fb55359e640 1 --2- 172.21.15.27:0/4238711632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55415a020 0x7fb55415e5a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb544009e50 tx=0x7fb54402bee0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.537+0000 7fb55359e640 1 -- 172.21.15.27:0/4238711632 shutdown_connections 2023-08-11T15:53:38.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.537+0000 7fb55359e640 1 --2- 172.21.15.27:0/4238711632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 0x7fb554071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.537+0000 7fb55359e640 1 --2- 172.21.15.27:0/4238711632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55415a020 0x7fb55415e5a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.537+0000 7fb55359e640 1 -- 172.21.15.27:0/4238711632 >> 172.21.15.27:0/4238711632 conn(0x7fb5540afb70 msgr2=0x7fb5540aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:38.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.537+0000 7fb55359e640 1 -- 172.21.15.27:0/4238711632 shutdown_connections 2023-08-11T15:53:38.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.537+0000 7fb55359e640 1 -- 172.21.15.27:0/4238711632 wait complete. 2023-08-11T15:53:38.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.540+0000 7fb55359e640 1 Processor -- start 2023-08-11T15:53:38.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.540+0000 7fb55359e640 1 -- start start 2023-08-11T15:53:38.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.541+0000 7fb55359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406f5d0 0x7fb554159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.541+0000 7fb55359e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 0x7fb5540c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.541+0000 7fb55359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb55415a020 0x7fb5540cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.541+0000 7fb55359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540c1c00 con 0x7fb55415a020 2023-08-11T15:53:38.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.541+0000 7fb55359e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540cb580 con 0x7fb55406f5d0 2023-08-11T15:53:38.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.541+0000 7fb55359e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540cace0 con 0x7fb5540c9600 2023-08-11T15:53:38.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb552d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb55415a020 0x7fb5540cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb552d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb55415a020 0x7fb5540cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53114/0 (socket says 172.21.15.27:53114) 2023-08-11T15:53:38.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb552d9d640 1 -- 172.21.15.27:0/787640795 learned_addr learned my addr 172.21.15.27:0/787640795 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:38.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb55259c640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406f5d0 0x7fb554159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb549d9b640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 0x7fb5540c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb552d9d640 1 -- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 msgr2=0x7fb5540c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb552d9d640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 0x7fb5540c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb552d9d640 1 -- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406f5d0 msgr2=0x7fb554159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb552d9d640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406f5d0 0x7fb554159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb552d9d640 1 -- 172.21.15.27:0/787640795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544009b30 con 0x7fb55415a020 2023-08-11T15:53:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.542+0000 7fb549d9b640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 0x7fb5540c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.543+0000 7fb552d9d640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb55415a020 0x7fb5540cb910 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00c390 tx=0x7fb54c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:38.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.543+0000 7fb54b7fe640 1 -- 172.21.15.27:0/787640795 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0052d0 con 0x7fb55415a020 2023-08-11T15:53:38.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.543+0000 7fb54b7fe640 1 -- 172.21.15.27:0/787640795 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c005430 con 0x7fb55415a020 2023-08-11T15:53:38.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.543+0000 7fb54b7fe640 1 -- 172.21.15.27:0/787640795 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c003810 con 0x7fb55415a020 2023-08-11T15:53:38.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.543+0000 7fb55359e640 1 -- 172.21.15.27:0/787640795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5540cafc0 con 0x7fb55415a020 2023-08-11T15:53:38.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.543+0000 7fb55359e640 1 -- 172.21.15.27:0/787640795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541599a0 con 0x7fb55415a020 2023-08-11T15:53:38.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.545+0000 7fb55359e640 1 -- 172.21.15.27:0/787640795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5540c1d70 con 0x7fb55415a020 2023-08-11T15:53:38.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.546+0000 7fb54b7fe640 1 -- 172.21.15.27:0/787640795 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb54c003970 con 0x7fb55415a020 2023-08-11T15:53:38.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.547+0000 7fb54b7fe640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb53406a050 0x7fb53406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:38.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.547+0000 7fb54b7fe640 1 -- 172.21.15.27:0/787640795 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb54c0878d0 con 0x7fb55415a020 2023-08-11T15:53:38.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.547+0000 7fb55259c640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb53406a050 0x7fb53406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:38.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.548+0000 7fb55259c640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb53406a050 0x7fb53406c510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb544035000 tx=0x7fb544030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:38.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.551+0000 7fb54b7fe640 1 -- 172.21.15.27:0/787640795 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fb54c0531b0 con 0x7fb55415a020 2023-08-11T15:53:38.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:38.817+0000 7fb55359e640 1 -- 172.21.15.27:0/787640795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7fb55404e420 con 0x7fb55415a020 2023-08-11T15:53:39.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.818+0000 7fb54b7fe640 1 -- 172.21.15.27:0/787640795 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v172) v1 ==== 160+0+0 (secure 0 0 0) 0x7fb54c0567d0 con 0x7fb55415a020 2023-08-11T15:53:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 -- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb53406a050 msgr2=0x7fb53406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb53406a050 0x7fb53406c510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb544035000 tx=0x7fb544030000 comp rx=0 tx=0).stop 2023-08-11T15:53:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 -- 172.21.15.27:0/787640795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb55415a020 msgr2=0x7fb5540cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb55415a020 0x7fb5540cb910 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00c390 tx=0x7fb54c00c850 comp rx=0 tx=0).stop 2023-08-11T15:53:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 -- 172.21.15.27:0/787640795 shutdown_connections 2023-08-11T15:53:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fb53406a050 0x7fb53406c510 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5540c9600 0x7fb5540c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb55415a020 0x7fb5540cb910 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 --2- 172.21.15.27:0/787640795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406f5d0 0x7fb554159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 -- 172.21.15.27:0/787640795 >> 172.21.15.27:0/787640795 conn(0x7fb5540afb70 msgr2=0x7fb5540cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 -- 172.21.15.27:0/787640795 shutdown_connections 2023-08-11T15:53:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:39.822+0000 7fb55359e640 1 -- 172.21.15.27:0/787640795 wait complete. 2023-08-11T15:53:39.843 INFO:tasks.cephfs_test_runner:test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) ... ERROR 2023-08-11T15:53:39.844 INFO:tasks.cephfs_test_runner: 2023-08-11T15:53:39.844 INFO:tasks.cephfs_test_runner:====================================================================== 2023-08-11T15:53:39.845 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-08-11T15:53:39.845 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-08-11T15:53:39.845 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-08-11T15:53:39.845 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-08-11T15:53:39.845 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-08-11T15:53:39.846 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/cephfs/mount.py", line 1566, in setfattr 2023-08-11T15:53:39.846 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-08-11T15:53:39.846 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-08-11T15:53:39.846 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-08-11T15:53:39.847 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/remote.py", line 522, in run 2023-08-11T15:53:39.847 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-08-11T15:53:39.847 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 455, in run 2023-08-11T15:53:39.847 INFO:tasks.cephfs_test_runner: r.wait() 2023-08-11T15:53:39.847 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 161, in wait 2023-08-11T15:53:39.848 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-08-11T15:53:39.848 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-08-11T15:53:39.848 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-08-11T15:53:39.848 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi027 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-08-11T15:53:39.848 INFO:tasks.cephfs_test_runner: 2023-08-11T15:53:39.849 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-08-11T15:53:39.849 INFO:tasks.cephfs_test_runner:Ran 1 test in 90.054s 2023-08-11T15:53:39.849 INFO:tasks.cephfs_test_runner: 2023-08-11T15:53:39.849 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-08-11T15:53:39.850 INFO:tasks.cephfs_test_runner: 2023-08-11T15:53:39.850 INFO:tasks.cephfs_test_runner:====================================================================== 2023-08-11T15:53:39.850 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-08-11T15:53:39.850 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-08-11T15:53:39.851 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-08-11T15:53:39.851 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-08-11T15:53:39.851 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-08-11T15:53:39.851 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/cephfs/mount.py", line 1566, in setfattr 2023-08-11T15:53:39.851 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-08-11T15:53:39.852 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-08-11T15:53:39.852 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-08-11T15:53:39.852 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/remote.py", line 522, in run 2023-08-11T15:53:39.852 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-08-11T15:53:39.852 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 455, in run 2023-08-11T15:53:39.853 INFO:tasks.cephfs_test_runner: r.wait() 2023-08-11T15:53:39.853 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 161, in wait 2023-08-11T15:53:39.853 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-08-11T15:53:39.853 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-08-11T15:53:39.853 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-08-11T15:53:39.854 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi027 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-08-11T15:53:39.854 INFO:tasks.cephfs_test_runner: 2023-08-11T15:53:39.854 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-08-11T15:53:39.921 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7de88daded054cecbc1edfe153bc8472 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-08-11T15:53:39.923 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-08-11T15:53:39.937 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-08-11T15:53:39.958 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-08-11T15:53:39.958 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:53:39.959 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:53:40.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053651,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-11T15:53:40.036 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-08-11T15:53:40.037 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:53:40.096 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053651,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-11T15:53:40.096 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:53:40.097 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:53:40.157 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-11T15:53:40.157 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-08-11T15:53:40.158 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:53:40.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-11T15:53:40.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:40.212 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-08-11T15:53:40.265 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-11T15:53:40.265 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-11T15:53:40.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:40.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-11T15:53:40.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:40.370 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-11T15:53:40.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:40.461 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-08-11T15:53:40.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:40.579 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-08-11T15:53:40.642 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-11T15:53:40.643 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-11T15:53:40.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:40.644 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-11T15:53:40.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:40.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-11T15:53:40.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:53:40.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2023-08-11T15:53:41.001 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-08-11T15:53:41.011 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-11T15:53:41.011 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:53:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.221+0000 7fec43f62640 1 Processor -- start 2023-08-11T15:53:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.221+0000 7fec43f62640 1 -- start start 2023-08-11T15:53:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.223+0000 7fec43f62640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fec3c0cb0c0 0x7fec3c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.223+0000 7fec43f62640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c0cbb70 0x7fec3c0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.223+0000 7fec43f62640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3c0cc020 con 0x7fec3c156e50 2023-08-11T15:53:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.223+0000 7fec43f62640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3c0ca460 con 0x7fec3c0cb0c0 2023-08-11T15:53:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.223+0000 7fec43f62640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c070470 con 0x7fec3c0cbb70 2023-08-11T15:53:41.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.223+0000 7fec424ff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c0cbb70 0x7fec3c0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.224+0000 7fec424ff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c0cbb70 0x7fec3c0708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:41608/0 (socket says 172.21.15.27:41608) 2023-08-11T15:53:41.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.224+0000 7fec414fd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fec3c0cb0c0 0x7fec3c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.224+0000 7fec424ff640 1 -- 172.21.15.27:0/2075888161 learned_addr learned my addr 172.21.15.27:0/2075888161 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:41.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.224+0000 7fec41cfe640 1 --1- 172.21.15.27:0/2075888161 >> v1:172.21.15.119:6789/0 conn(0x7fec3c156e50 0x7fec3c0ca9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:42722/0 (socket says 172.21.15.27:42722) 2023-08-11T15:53:41.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.224+0000 7fec40cfc640 1 -- 172.21.15.27:0/2075888161 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455827408 0 0) 0x7fec3c0cc020 con 0x7fec3c156e50 2023-08-11T15:53:41.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.224+0000 7fec40cfc640 1 -- 172.21.15.27:0/2075888161 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec28003520 con 0x7fec3c156e50 2023-08-11T15:53:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.225+0000 7fec424ff640 1 -- 172.21.15.27:0/2075888161 >> v2:172.21.15.119:3300/0 conn(0x7fec3c0cb0c0 msgr2=0x7fec3c15a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.225+0000 7fec424ff640 1 --2- 172.21.15.27:0/2075888161 >> v2:172.21.15.119:3300/0 conn(0x7fec3c0cb0c0 0x7fec3c15a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.225+0000 7fec424ff640 1 -- 172.21.15.27:0/2075888161 >> v1:172.21.15.119:6789/0 conn(0x7fec3c156e50 legacy=0x7fec3c0ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.225+0000 7fec424ff640 1 -- 172.21.15.27:0/2075888161 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c0cc270 con 0x7fec3c0cbb70 2023-08-11T15:53:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.225+0000 7fec414fd640 1 --2- 172.21.15.27:0/2075888161 >> v2:172.21.15.119:3300/0 conn(0x7fec3c0cb0c0 0x7fec3c15a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.226+0000 7fec424ff640 1 --2- 172.21.15.27:0/2075888161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c0cbb70 0x7fec3c0708c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fec34009e50 tx=0x7fec3402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d42df478d966001c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.226+0000 7fec40cfc640 1 -- 172.21.15.27:0/2075888161 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3402cb00 con 0x7fec3c0cbb70 2023-08-11T15:53:41.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.226+0000 7fec40cfc640 1 -- 172.21.15.27:0/2075888161 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3402cc60 con 0x7fec3c0cbb70 2023-08-11T15:53:41.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.226+0000 7fec40cfc640 1 -- 172.21.15.27:0/2075888161 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec340343f0 con 0x7fec3c0cbb70 2023-08-11T15:53:41.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.227+0000 7fec43f62640 1 -- 172.21.15.27:0/2075888161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c0cbb70 msgr2=0x7fec3c0708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.227+0000 7fec43f62640 1 --2- 172.21.15.27:0/2075888161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c0cbb70 0x7fec3c0708c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fec34009e50 tx=0x7fec3402bf60 comp rx=0 tx=0).stop 2023-08-11T15:53:41.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.227+0000 7fec43f62640 1 -- 172.21.15.27:0/2075888161 shutdown_connections 2023-08-11T15:53:41.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.227+0000 7fec43f62640 1 --2- 172.21.15.27:0/2075888161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c0cbb70 0x7fec3c0708c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.227+0000 7fec43f62640 1 --2- 172.21.15.27:0/2075888161 >> v2:172.21.15.119:3300/0 conn(0x7fec3c0cb0c0 0x7fec3c15a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.227+0000 7fec43f62640 1 -- 172.21.15.27:0/2075888161 >> 172.21.15.27:0/2075888161 conn(0x7fec3c0afb70 msgr2=0x7fec3c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.227+0000 7fec43f62640 1 -- 172.21.15.27:0/2075888161 shutdown_connections 2023-08-11T15:53:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.228+0000 7fec43f62640 1 -- 172.21.15.27:0/2075888161 wait complete. 2023-08-11T15:53:41.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.230+0000 7fec43f62640 1 Processor -- start 2023-08-11T15:53:41.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.230+0000 7fec43f62640 1 -- start start 2023-08-11T15:53:41.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.231+0000 7fec43f62640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec3c0cb0c0 0x7fec3c14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.231+0000 7fec43f62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0cbb70 0x7fec3c0c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.231+0000 7fec43f62640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c156e50 0x7fec3c0c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.231+0000 7fec43f62640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0b0d00 con 0x7fec3c0cbb70 2023-08-11T15:53:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.231+0000 7fec43f62640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0b0f70 con 0x7fec3c0cb0c0 2023-08-11T15:53:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.231+0000 7fec43f62640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0b3970 con 0x7fec3c156e50 2023-08-11T15:53:41.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.231+0000 7fec414fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0cbb70 0x7fec3c0c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.231+0000 7fec414fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0cbb70 0x7fec3c0c2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51066/0 (socket says 172.21.15.27:51066) 2023-08-11T15:53:41.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.231+0000 7fec414fd640 1 -- 172.21.15.27:0/428698043 learned_addr learned my addr 172.21.15.27:0/428698043 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:41.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec424ff640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c156e50 0x7fec3c0c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec41cfe640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec3c0cb0c0 0x7fec3c14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec414fd640 1 -- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c156e50 msgr2=0x7fec3c0c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec414fd640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c156e50 0x7fec3c0c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec414fd640 1 -- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec3c0cb0c0 msgr2=0x7fec3c14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec414fd640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec3c0cb0c0 0x7fec3c14fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec414fd640 1 -- 172.21.15.27:0/428698043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c00b6a0 con 0x7fec3c0cbb70 2023-08-11T15:53:41.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec414fd640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0cbb70 0x7fec3c0c2b20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00ed40 tx=0x7fec2c00d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec327fc640 1 -- 172.21.15.27:0/428698043 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c010640 con 0x7fec3c0cbb70 2023-08-11T15:53:41.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec327fc640 1 -- 172.21.15.27:0/428698043 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c0107a0 con 0x7fec3c0cbb70 2023-08-11T15:53:41.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec327fc640 1 -- 172.21.15.27:0/428698043 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c010b10 con 0x7fec3c0cbb70 2023-08-11T15:53:41.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec43f62640 1 -- 172.21.15.27:0/428698043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec34009b30 con 0x7fec3c0cbb70 2023-08-11T15:53:41.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.232+0000 7fec43f62640 1 -- 172.21.15.27:0/428698043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c0b3c50 con 0x7fec3c0cbb70 2023-08-11T15:53:41.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.234+0000 7fec43f62640 1 -- 172.21.15.27:0/428698043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3c0b10e0 con 0x7fec3c0cbb70 2023-08-11T15:53:41.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.235+0000 7fec327fc640 1 -- 172.21.15.27:0/428698043 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fec2c010c70 con 0x7fec3c0cbb70 2023-08-11T15:53:41.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.236+0000 7fec327fc640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fec2006a050 0x7fec2006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.236+0000 7fec327fc640 1 -- 172.21.15.27:0/428698043 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fec2c0889f0 con 0x7fec3c0cbb70 2023-08-11T15:53:41.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.237+0000 7fec41cfe640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fec2006a050 0x7fec2006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.238+0000 7fec41cfe640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fec2006a050 0x7fec2006c510 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fec38001230 tx=0x7fec38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.240+0000 7fec327fc640 1 -- 172.21.15.27:0/428698043 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fec2c0542a0 con 0x7fec3c0cbb70 2023-08-11T15:53:41.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.430+0000 7fec43f62640 1 -- 172.21.15.27:0/428698043 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec3c0cc830 con 0x7fec2006a050 2023-08-11T15:53:41.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.438+0000 7fec327fc640 1 -- 172.21.15.27:0/428698043 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70554 (secure 0 0 0) 0x7fec38001c50 con 0x7fec2006a050 2023-08-11T15:53:41.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:41.442 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:53:41.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 -- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fec2006a050 msgr2=0x7fec2006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fec2006a050 0x7fec2006c510 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fec38001230 tx=0x7fec38009040 comp rx=0 tx=0).stop 2023-08-11T15:53:41.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 -- 172.21.15.27:0/428698043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0cbb70 msgr2=0x7fec3c0c2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0cbb70 0x7fec3c0c2b20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00ed40 tx=0x7fec2c00d720 comp rx=0 tx=0).stop 2023-08-11T15:53:41.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 -- 172.21.15.27:0/428698043 shutdown_connections 2023-08-11T15:53:41.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fec2006a050 0x7fec2006c510 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec3c156e50 0x7fec3c0c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0cbb70 0x7fec3c0c2b20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 --2- 172.21.15.27:0/428698043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec3c0cb0c0 0x7fec3c14fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 -- 172.21.15.27:0/428698043 >> 172.21.15.27:0/428698043 conn(0x7fec3c0afb70 msgr2=0x7fec3c0ca9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:41.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.441+0000 7fec43f62640 1 -- 172.21.15.27:0/428698043 shutdown_connections 2023-08-11T15:53:41.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.442+0000 7fec43f62640 1 -- 172.21.15.27:0/428698043 wait complete. 2023-08-11T15:53:41.462 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-08-11T15:53:40.772257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233648,"kb_used_data":5200,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754741072,"statfs":{"total":773094113280,"available":772854857728,"internally_reserved":0,"allocated":5324800,"data_stored":3024172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6418,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.217970"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.572082+0000","last_change":"2023-08-11T15:50:54.667601+0000","last_active":"2023-08-11T15:53:34.572082+0000","last_peered":"2023-08-11T15:53:34.572082+0000","last_clean":"2023-08-11T15:53:34.572082+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:53:34.572082+0000","last_undegraded":"2023-08-11T15:53:34.572082+0000","last_fullsized":"2023-08-11T15:53:34.572082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:14:58.155069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.566887+0000","last_change":"2023-08-11T15:50:54.662968+0000","last_active":"2023-08-11T15:53:34.566887+0000","last_peered":"2023-08-11T15:53:34.566887+0000","last_clean":"2023-08-11T15:53:34.566887+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:53:34.566887+0000","last_undegraded":"2023-08-11T15:53:34.566887+0000","last_fullsized":"2023-08-11T15:53:34.566887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:52:28.027008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.571589+0000","last_change":"2023-08-11T15:50:55.272783+0000","last_active":"2023-08-11T15:53:34.571589+0000","last_peered":"2023-08-11T15:53:34.571589+0000","last_clean":"2023-08-11T15:53:34.571589+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:53:34.571589+0000","last_undegraded":"2023-08-11T15:53:34.571589+0000","last_fullsized":"2023-08-11T15:53:34.571589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:55:04.564299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.566357+0000","last_change":"2023-08-11T15:50:55.270974+0000","last_active":"2023-08-11T15:53:34.566357+0000","last_peered":"2023-08-11T15:53:34.566357+0000","last_clean":"2023-08-11T15:53:34.566357+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:53:34.566357+0000","last_undegraded":"2023-08-11T15:53:34.566357+0000","last_fullsized":"2023-08-11T15:53:34.566357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:51:10.789686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.567805+0000","last_change":"2023-08-11T15:50:54.660862+0000","last_active":"2023-08-11T15:53:34.567805+0000","last_peered":"2023-08-11T15:53:34.567805+0000","last_clean":"2023-08-11T15:53:34.567805+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:53:34.567805+0000","last_undegraded":"2023-08-11T15:53:34.567805+0000","last_fullsized":"2023-08-11T15:53:34.567805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:38:26.058144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.558797+0000","last_change":"2023-08-11T15:50:54.660575+0000","last_active":"2023-08-11T15:53:34.558797+0000","last_peered":"2023-08-11T15:53:34.558797+0000","last_clean":"2023-08-11T15:53:34.558797+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:53:34.558797+0000","last_undegraded":"2023-08-11T15:53:34.558797+0000","last_fullsized":"2023-08-11T15:53:34.558797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:03.278614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":58,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.568881+0000","last_change":"2023-08-11T15:50:54.669675+0000","last_active":"2023-08-11T15:53:34.568881+0000","last_peered":"2023-08-11T15:53:34.568881+0000","last_clean":"2023-08-11T15:53:34.568881+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:53:34.568881+0000","last_undegraded":"2023-08-11T15:53:34.568881+0000","last_fullsized":"2023-08-11T15:53:34.568881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:51.224613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.568370+0000","last_change":"2023-08-11T15:50:54.663730+0000","last_active":"2023-08-11T15:53:34.568370+0000","last_peered":"2023-08-11T15:53:34.568370+0000","last_clean":"2023-08-11T15:53:34.568370+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:53:34.568370+0000","last_undegraded":"2023-08-11T15:53:34.568370+0000","last_fullsized":"2023-08-11T15:53:34.568370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:16:33.835392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":214,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.569244+0000","last_change":"2023-08-11T15:50:48.426393+0000","last_active":"2023-08-11T15:53:34.569244+0000","last_peered":"2023-08-11T15:53:34.569244+0000","last_clean":"2023-08-11T15:53:34.569244+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:53:34.569244+0000","last_undegraded":"2023-08-11T15:53:34.569244+0000","last_fullsized":"2023-08-11T15:53:34.569244+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_clean_scrub_stamp":"2023-08-11T15:50:47.133560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:22:12.270566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542182,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1105920,"data_stored":819283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.093,"15min":1.1160000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.216,"15min":1.222},"min":{"1min":0.80900000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5509999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3129999999999999,"15min":1.327},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.7370000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.6339999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.177,"15min":1.175},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.232,"5min":1.254,"15min":1.258},"min":{"1min":0.87,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5169999999999999,"5min":2.02,"15min":2.02},"last":0.97299999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2909999999999999,"15min":1.3069999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.671,"5min":1.869,"15min":1.869},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4890000000000001,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.103}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4219999999999999,"15min":1.4279999999999999},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.885,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.395,"15min":1.4139999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6910000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.6759999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.373,"15min":1.381},"min":{"1min":0.93500000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.212},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5449999999999999,"15min":1.5509999999999999},"min":{"1min":1.2170000000000001,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":1.9330000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.762}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.5029999999999999,"15min":1.5209999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.768,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.591},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.544,"15min":1.5580000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.861,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.321}]}]},{"osd":6,"up_from":14,"seq":60129542182,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":229497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0609999999999999,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1339999999999999,"15min":1.1359999999999999},"min":{"1min":0.751,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.181}]},{"osd":1,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.115,"15min":1.1140000000000001},"min":{"1min":0.73599999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.591,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.66100000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.95199999999999996}]},{"osd":2,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2390000000000001,"15min":1.24},"min":{"1min":0.85599999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.3180000000000001,"15min":1.3400000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5600000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.296}]},{"osd":3,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.369,"15min":1.377},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.77,"5min":2.774,"15min":2.774},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7330000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.383}]},{"osd":4,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.319,"15min":1.327},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8879999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":1.694},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.468,"15min":1.4910000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.665,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.4350000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.496,"15min":1.5069999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8149999999999999,"5min":3.371,"15min":3.371},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.486,"15min":1.504},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0009999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.546}]},{"osd":7,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.615,"15min":1.6479999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.762,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":1.5},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.6739999999999999,"15min":1.704},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.921,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.4690000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":228895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0389999999999999,"15min":1.0269999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.1499999999999999,"15min":1.163},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2569999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1419999999999999,"15min":1.1399999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.401},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.93999999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.621,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.196}]},{"osd":2,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.262,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.615,"5min":1.827,"15min":1.827},"last":1.129},{"interface":"front","average":{"1min":1.194,"5min":1.351,"15min":1.377},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8300000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.5860000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.353,"15min":1.353},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.702,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.339},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.444},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.782,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.476}]},{"osd":4,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4379999999999999,"15min":1.466},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8779999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.679},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5880000000000001,"15min":1.6080000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.847,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":1.8009999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.4690000000000001,"15min":1.492},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.796,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.6120000000000001,"15min":1.6379999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.2160000000000002,"5min":3.9870000000000001,"15min":3.9870000000000001},"last":1.7749999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.5820000000000001,"15min":1.605},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9990000000000001,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":1.718},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.637,"15min":1.677},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.855,"5min":3.367,"15min":3.367},"last":1.633}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":234445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0669999999999999,"15min":1.0609999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5700000000000001,"5min":1.776,"15min":1.776},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.115,"15min":1.1259999999999999},"min":{"1min":0.64900000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.4930000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.93500000000000005}]},{"osd":1,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1200000000000001,"15min":1.129},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5089999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.008},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1879999999999999,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6659999999999999,"5min":1.756,"15min":1.756},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.26,"15min":1.2769999999999999},"min":{"1min":0.89700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3720000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2310000000000001,"15min":1.2230000000000001},"min":{"1min":0.88900000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.294}]},{"osd":3,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.82199999999999995,"5min":0.75,"15min":0.75},"max":{"1min":1.524,"5min":1.736,"15min":1.736},"last":1.216},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.357,"15min":1.3640000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.738,"5min":2.121,"15min":2.121},"last":1.5329999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2929999999999999,"15min":1.2949999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8759999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.746,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6419999999999999,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.3380000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3819999999999999,"15min":1.3839999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.825,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.917},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.9450000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.7390000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.405,"15min":1.415},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.617,"5min":2.286,"15min":2.286},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.577,"15min":1.5940000000000001},"min":{"1min":1.008,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.9139999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":2.0489999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542182,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":229463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0589999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97999999999999998},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.85299999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0249999999999999,"15min":1.0449999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.458,"5min":1.78,"15min":1.78},"last":1.4630000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.97599999999999998,"15min":0.99199999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.24,"5min":1.623,"15min":1.623},"last":0.79400000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.55,"5min":3.7799999999999998,"15min":3.7799999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.202,"15min":1.2290000000000001},"min":{"1min":0.628,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5920000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1060000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3620000000000001,"15min":1.3799999999999999},"min":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.573},{"interface":"front","average":{"1min":1.286,"5min":1.369,"15min":1.383},"min":{"1min":0.88500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.7190000000000001,"5min":2.573,"15min":2.573},"last":1.53}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.5629999999999999,"15min":1.6000000000000001},"min":{"1min":0.99399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.514,"15min":1.5700000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6299999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":1.3109999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.534,"15min":1.5580000000000001},"min":{"1min":1.0569999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.873,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.6060000000000001,"15min":1.6459999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.758,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":1.1870000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.6499999999999999,"15min":1.6899999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.825,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":1.677},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.6379999999999999,"15min":1.6879999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.738,"5min":3.4340000000000002,"15min":3.4340000000000002},"last":1.6220000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":234467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89900000000000002,"15min":0.90300000000000002},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3009999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.755},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.024,"15min":1.036},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.405,"5min":1.865,"15min":1.865},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.383,"5min":1.468,"15min":1.468},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.079,"15min":1.091},"min":{"1min":0.64900000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.85099999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4570000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.2},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.879,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4139999999999999,"15min":1.4179999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.8759999999999999,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.93100000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.847,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.026}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.508,"15min":1.5229999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9730000000000001,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.304,"5min":1.4399999999999999,"15min":1.462},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6990000000000001,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.0489999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7829999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.472,"5min":1.5069999999999999,"15min":1.5129999999999999},"min":{"1min":0.97799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.0289999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.97399999999999998}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.6060000000000001,"15min":1.6279999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9279999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.593,"15min":1.6140000000000001},"min":{"1min":1.054,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.907,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":1.0169999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":512000,"data_stored":228895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.575,"15min":1.575},"last":0.999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0089999999999999,"15min":1.0149999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.167}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.026,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.782,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.0569999999999999,"15min":1.0609999999999999},"min":{"1min":0.64000000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.377,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.651}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.282,"15min":1.292},"min":{"1min":0.70999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6200000000000001,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":1.736},{"interface":"front","average":{"1min":1.3,"5min":1.2709999999999999,"15min":1.266},"min":{"1min":0.62,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.962,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":1.579}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.411,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.7410000000000001,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4199999999999999,"15min":1.4359999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.994,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":1.3720000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1059999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.25},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5009999999999999,"15min":1.51},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.8700000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.4370000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.524,"15min":1.518},"min":{"1min":1.0529999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.3500000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.786},{"interface":"front","average":{"1min":1.405,"5min":1.458,"15min":1.4670000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.919,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.7030000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.5209999999999999,"15min":1.524},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.0449999999999999,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.837},{"interface":"front","average":{"1min":1.621,"5min":1.617,"15min":1.617},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":2.4529999999999998,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.9570000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1105920,"data_stored":819227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.83799999999999997},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.109,"5min":1.288,"15min":1.288},"last":1.26},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":0.92100000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.998}]},{"osd":2,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.381},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0249999999999999,"15min":1.0129999999999999},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.048}]},{"osd":3,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.51200000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3080000000000001,"5min":1.522,"15min":1.522},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.137,"15min":1.149},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6879999999999999,"5min":1.728,"15min":1.728},"last":1.589}]},{"osd":4,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.323,"15min":1.3360000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.724,"5min":2.23,"15min":2.23},"last":1.095},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2869999999999999,"15min":1.298},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.49}]},{"osd":5,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.333,"15min":1.337},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.889,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":2.1299999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.4330000000000001,"15min":1.4490000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.7549999999999999,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":1.141}]},{"osd":6,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.3460000000000001,"15min":1.3700000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.677,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.413,"15min":1.4239999999999999},"min":{"1min":0.877,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":2.2509999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4990000000000001,"15min":1.5069999999999999},"min":{"1min":0.86199999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.99,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.177},{"interface":"front","average":{"1min":1.337,"5min":1.468,"15min":1.49},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.7729999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.639}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:53:41.466 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:53:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.703+0000 7fe7b6a08640 1 Processor -- start 2023-08-11T15:53:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.703+0000 7fe7b6a08640 1 -- start start 2023-08-11T15:53:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.705+0000 7fe7b6a08640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7b00cc280 0x7fe7b00caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.705+0000 7fe7b6a08640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b00ad2a0 0x7fe7b00c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.705+0000 7fe7b6a08640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b006b910 con 0x7fe7b015c520 2023-08-11T15:53:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.705+0000 7fe7b6a08640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b006bbf0 con 0x7fe7b00cc280 2023-08-11T15:53:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.705+0000 7fe7b6a08640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b006bed0 con 0x7fe7b00ad2a0 2023-08-11T15:53:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.706+0000 7fe7b5205640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7b00cc280 0x7fe7b00caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.706+0000 7fe7b5205640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7b00cc280 0x7fe7b00caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51074/0 (socket says 172.21.15.27:51074) 2023-08-11T15:53:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.706+0000 7fe7b5205640 1 -- 172.21.15.27:0/1590565918 learned_addr learned my addr 172.21.15.27:0/1590565918 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.706+0000 7fe7b6207640 1 --2- 172.21.15.27:0/1590565918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b00ad2a0 0x7fe7b00c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.706+0000 7fe7b5205640 1 -- 172.21.15.27:0/1590565918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b00ad2a0 msgr2=0x7fe7b00c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.706+0000 7fe7b5205640 1 --2- 172.21.15.27:0/1590565918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b00ad2a0 0x7fe7b00c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.706+0000 7fe7b5205640 1 -- 172.21.15.27:0/1590565918 >> v1:172.21.15.119:6789/0 conn(0x7fe7b015c520 legacy=0x7fe7b00cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.706+0000 7fe7b5205640 1 -- 172.21.15.27:0/1590565918 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b006c1b0 con 0x7fe7b00cc280 2023-08-11T15:53:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.708+0000 7fe7b5205640 1 --2- 172.21.15.27:0/1590565918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 0x7fe7b00caed0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00c9f00 tx=0x7fe7a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c103b23caf59761 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.708+0000 7fe7b4a04640 1 -- 172.21.15.27:0/1590565918 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a002e070 con 0x7fe7b00cc280 2023-08-11T15:53:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.708+0000 7fe7b4a04640 1 -- 172.21.15.27:0/1590565918 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a00315a0 con 0x7fe7b00cc280 2023-08-11T15:53:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.708+0000 7fe7b4a04640 1 -- 172.21.15.27:0/1590565918 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0031930 con 0x7fe7b00cc280 2023-08-11T15:53:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.709+0000 7fe7b6a08640 1 -- 172.21.15.27:0/1590565918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 msgr2=0x7fe7b00caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.709+0000 7fe7b6a08640 1 --2- 172.21.15.27:0/1590565918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 0x7fe7b00caed0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00c9f00 tx=0x7fe7a002b840 comp rx=0 tx=0).stop 2023-08-11T15:53:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.709+0000 7fe7b6a08640 1 -- 172.21.15.27:0/1590565918 shutdown_connections 2023-08-11T15:53:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.709+0000 7fe7b6a08640 1 --2- 172.21.15.27:0/1590565918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b00ad2a0 0x7fe7b00c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.709+0000 7fe7b6a08640 1 --2- 172.21.15.27:0/1590565918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 0x7fe7b00caed0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.709+0000 7fe7b6a08640 1 -- 172.21.15.27:0/1590565918 >> 172.21.15.27:0/1590565918 conn(0x7fe7b00afb70 msgr2=0x7fe7b00aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:41.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.709+0000 7fe7b6a08640 1 -- 172.21.15.27:0/1590565918 shutdown_connections 2023-08-11T15:53:41.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.709+0000 7fe7b6a08640 1 -- 172.21.15.27:0/1590565918 wait complete. 2023-08-11T15:53:41.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.711+0000 7fe7b6a08640 1 Processor -- start 2023-08-11T15:53:41.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.711+0000 7fe7b6a08640 1 -- start start 2023-08-11T15:53:41.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.712+0000 7fe7b6a08640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7b00ad2a0 0x7fe7b0158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.712+0000 7fe7b6a08640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 0x7fe7b00c2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.712+0000 7fe7b6a08640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b015c520 0x7fe7b0070610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.712+0000 7fe7b6a08640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b0159220 con 0x7fe7b00cc280 2023-08-11T15:53:41.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.712+0000 7fe7b6a08640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b00b15a0 con 0x7fe7b00ad2a0 2023-08-11T15:53:41.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.712+0000 7fe7b6a08640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b00c1bc0 con 0x7fe7b015c520 2023-08-11T15:53:41.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.712+0000 7fe7b5205640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 0x7fe7b00c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b5205640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 0x7fe7b00c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51076/0 (socket says 172.21.15.27:51076) 2023-08-11T15:53:41.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b5205640 1 -- 172.21.15.27:0/926892936 learned_addr learned my addr 172.21.15.27:0/926892936 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:41.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b5a06640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7b00ad2a0 0x7fe7b0158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b6207640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b015c520 0x7fe7b0070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b5205640 1 -- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b015c520 msgr2=0x7fe7b0070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b5205640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b015c520 0x7fe7b0070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b5205640 1 -- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7b00ad2a0 msgr2=0x7fe7b0158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b5205640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7b00ad2a0 0x7fe7b0158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b5205640 1 -- 172.21.15.27:0/926892936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00094d0 con 0x7fe7b00cc280 2023-08-11T15:53:41.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b6207640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b015c520 0x7fe7b0070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:41.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.713+0000 7fe7b5205640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 0x7fe7b00c2220 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe7a002b880 tx=0x7fe7a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.714+0000 7fe7a67fc640 1 -- 172.21.15.27:0/926892936 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0031f10 con 0x7fe7b00cc280 2023-08-11T15:53:41.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.714+0000 7fe7a67fc640 1 -- 172.21.15.27:0/926892936 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a0032860 con 0x7fe7b00cc280 2023-08-11T15:53:41.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.714+0000 7fe7b6a08640 1 -- 172.21.15.27:0/926892936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b00b1840 con 0x7fe7b00cc280 2023-08-11T15:53:41.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.714+0000 7fe7a67fc640 1 -- 172.21.15.27:0/926892936 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0032c10 con 0x7fe7b00cc280 2023-08-11T15:53:41.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.714+0000 7fe7b6a08640 1 -- 172.21.15.27:0/926892936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b00c1ea0 con 0x7fe7b00cc280 2023-08-11T15:53:41.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.715+0000 7fe7b6a08640 1 -- 172.21.15.27:0/926892936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b00b1d60 con 0x7fe7b00cc280 2023-08-11T15:53:41.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.716+0000 7fe7a67fc640 1 -- 172.21.15.27:0/926892936 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe7a004d3f0 con 0x7fe7b00cc280 2023-08-11T15:53:41.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.717+0000 7fe7a67fc640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe79406a050 0x7fe79406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.717+0000 7fe7a67fc640 1 -- 172.21.15.27:0/926892936 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fe7a00a9ea0 con 0x7fe7b00cc280 2023-08-11T15:53:41.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.718+0000 7fe7b5a06640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe79406a050 0x7fe79406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.719+0000 7fe7b5a06640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe79406a050 0x7fe79406c510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac006fd0 tx=0x7fe7ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.721+0000 7fe7a67fc640 1 -- 172.21.15.27:0/926892936 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fe7b00b1d60 con 0x7fe7b00cc280 2023-08-11T15:53:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.917+0000 7fe7b6a08640 1 -- 172.21.15.27:0/926892936 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7b00ad6f0 con 0x7fe79406a050 2023-08-11T15:53:41.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.926+0000 7fe7a67fc640 1 -- 172.21.15.27:0/926892936 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70554 (secure 0 0 0) 0x7fe7ac00bda0 con 0x7fe79406a050 2023-08-11T15:53:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:41.930 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:53:41.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.928+0000 7fe7b6a08640 1 -- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe79406a050 msgr2=0x7fe79406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.928+0000 7fe7b6a08640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe79406a050 0x7fe79406c510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac006fd0 tx=0x7fe7ac008040 comp rx=0 tx=0).stop 2023-08-11T15:53:41.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.928+0000 7fe7b6a08640 1 -- 172.21.15.27:0/926892936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 msgr2=0x7fe7b00c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.928+0000 7fe7b6a08640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 0x7fe7b00c2220 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe7a002b880 tx=0x7fe7a00046c0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.928+0000 7fe7b6a08640 1 -- 172.21.15.27:0/926892936 shutdown_connections 2023-08-11T15:53:41.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.928+0000 7fe7b6a08640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fe79406a050 0x7fe79406c510 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.928+0000 7fe7b6a08640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7b015c520 0x7fe7b0070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.928+0000 7fe7b6a08640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b00cc280 0x7fe7b00c2220 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.928+0000 7fe7b6a08640 1 --2- 172.21.15.27:0/926892936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7b00ad2a0 0x7fe7b0158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.928+0000 7fe7b6a08640 1 -- 172.21.15.27:0/926892936 >> 172.21.15.27:0/926892936 conn(0x7fe7b00afb70 msgr2=0x7fe7b00cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:41.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.929+0000 7fe7b6a08640 1 -- 172.21.15.27:0/926892936 shutdown_connections 2023-08-11T15:53:41.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:41.929+0000 7fe7b6a08640 1 -- 172.21.15.27:0/926892936 wait complete. 2023-08-11T15:53:41.951 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-08-11T15:53:40.772257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233648,"kb_used_data":5200,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754741072,"statfs":{"total":773094113280,"available":772854857728,"internally_reserved":0,"allocated":5324800,"data_stored":3024172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6418,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.217970"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.572082+0000","last_change":"2023-08-11T15:50:54.667601+0000","last_active":"2023-08-11T15:53:34.572082+0000","last_peered":"2023-08-11T15:53:34.572082+0000","last_clean":"2023-08-11T15:53:34.572082+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:53:34.572082+0000","last_undegraded":"2023-08-11T15:53:34.572082+0000","last_fullsized":"2023-08-11T15:53:34.572082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:14:58.155069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.566887+0000","last_change":"2023-08-11T15:50:54.662968+0000","last_active":"2023-08-11T15:53:34.566887+0000","last_peered":"2023-08-11T15:53:34.566887+0000","last_clean":"2023-08-11T15:53:34.566887+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:53:34.566887+0000","last_undegraded":"2023-08-11T15:53:34.566887+0000","last_fullsized":"2023-08-11T15:53:34.566887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:52:28.027008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.571589+0000","last_change":"2023-08-11T15:50:55.272783+0000","last_active":"2023-08-11T15:53:34.571589+0000","last_peered":"2023-08-11T15:53:34.571589+0000","last_clean":"2023-08-11T15:53:34.571589+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:53:34.571589+0000","last_undegraded":"2023-08-11T15:53:34.571589+0000","last_fullsized":"2023-08-11T15:53:34.571589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:55:04.564299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.566357+0000","last_change":"2023-08-11T15:50:55.270974+0000","last_active":"2023-08-11T15:53:34.566357+0000","last_peered":"2023-08-11T15:53:34.566357+0000","last_clean":"2023-08-11T15:53:34.566357+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:53:34.566357+0000","last_undegraded":"2023-08-11T15:53:34.566357+0000","last_fullsized":"2023-08-11T15:53:34.566357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:51:10.789686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.567805+0000","last_change":"2023-08-11T15:50:54.660862+0000","last_active":"2023-08-11T15:53:34.567805+0000","last_peered":"2023-08-11T15:53:34.567805+0000","last_clean":"2023-08-11T15:53:34.567805+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:53:34.567805+0000","last_undegraded":"2023-08-11T15:53:34.567805+0000","last_fullsized":"2023-08-11T15:53:34.567805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:38:26.058144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.558797+0000","last_change":"2023-08-11T15:50:54.660575+0000","last_active":"2023-08-11T15:53:34.558797+0000","last_peered":"2023-08-11T15:53:34.558797+0000","last_clean":"2023-08-11T15:53:34.558797+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:53:34.558797+0000","last_undegraded":"2023-08-11T15:53:34.558797+0000","last_fullsized":"2023-08-11T15:53:34.558797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:03.278614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":58,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.568881+0000","last_change":"2023-08-11T15:50:54.669675+0000","last_active":"2023-08-11T15:53:34.568881+0000","last_peered":"2023-08-11T15:53:34.568881+0000","last_clean":"2023-08-11T15:53:34.568881+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:53:34.568881+0000","last_undegraded":"2023-08-11T15:53:34.568881+0000","last_fullsized":"2023-08-11T15:53:34.568881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:51.224613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.568370+0000","last_change":"2023-08-11T15:50:54.663730+0000","last_active":"2023-08-11T15:53:34.568370+0000","last_peered":"2023-08-11T15:53:34.568370+0000","last_clean":"2023-08-11T15:53:34.568370+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:53:34.568370+0000","last_undegraded":"2023-08-11T15:53:34.568370+0000","last_fullsized":"2023-08-11T15:53:34.568370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:16:33.835392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":214,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.569244+0000","last_change":"2023-08-11T15:50:48.426393+0000","last_active":"2023-08-11T15:53:34.569244+0000","last_peered":"2023-08-11T15:53:34.569244+0000","last_clean":"2023-08-11T15:53:34.569244+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:53:34.569244+0000","last_undegraded":"2023-08-11T15:53:34.569244+0000","last_fullsized":"2023-08-11T15:53:34.569244+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_clean_scrub_stamp":"2023-08-11T15:50:47.133560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:22:12.270566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542182,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1105920,"data_stored":819283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.093,"15min":1.1160000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.216,"15min":1.222},"min":{"1min":0.80900000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5509999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3129999999999999,"15min":1.327},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.7370000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.6339999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.177,"15min":1.175},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.232,"5min":1.254,"15min":1.258},"min":{"1min":0.87,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5169999999999999,"5min":2.02,"15min":2.02},"last":0.97299999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2909999999999999,"15min":1.3069999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.671,"5min":1.869,"15min":1.869},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4890000000000001,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.103}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4219999999999999,"15min":1.4279999999999999},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.885,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.395,"15min":1.4139999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6910000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.6759999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.373,"15min":1.381},"min":{"1min":0.93500000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.212},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5449999999999999,"15min":1.5509999999999999},"min":{"1min":1.2170000000000001,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":1.9330000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.762}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.5029999999999999,"15min":1.5209999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.768,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.591},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.544,"15min":1.5580000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.861,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.321}]}]},{"osd":6,"up_from":14,"seq":60129542182,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":229497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0609999999999999,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1339999999999999,"15min":1.1359999999999999},"min":{"1min":0.751,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.181}]},{"osd":1,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.115,"15min":1.1140000000000001},"min":{"1min":0.73599999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.591,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.66100000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.95199999999999996}]},{"osd":2,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2390000000000001,"15min":1.24},"min":{"1min":0.85599999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.3180000000000001,"15min":1.3400000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5600000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.296}]},{"osd":3,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.369,"15min":1.377},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.77,"5min":2.774,"15min":2.774},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7330000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.383}]},{"osd":4,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.319,"15min":1.327},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8879999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":1.694},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.468,"15min":1.4910000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.665,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.4350000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.496,"15min":1.5069999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8149999999999999,"5min":3.371,"15min":3.371},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.486,"15min":1.504},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0009999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.546}]},{"osd":7,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.615,"15min":1.6479999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.762,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":1.5},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.6739999999999999,"15min":1.704},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.921,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.4690000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":228895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0389999999999999,"15min":1.0269999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.1499999999999999,"15min":1.163},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2569999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1419999999999999,"15min":1.1399999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.401},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.93999999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.621,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.196}]},{"osd":2,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.262,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.615,"5min":1.827,"15min":1.827},"last":1.129},{"interface":"front","average":{"1min":1.194,"5min":1.351,"15min":1.377},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8300000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.5860000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.353,"15min":1.353},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.702,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.339},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.444},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.782,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.476}]},{"osd":4,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4379999999999999,"15min":1.466},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8779999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.679},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5880000000000001,"15min":1.6080000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.847,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":1.8009999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.4690000000000001,"15min":1.492},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.796,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.6120000000000001,"15min":1.6379999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.2160000000000002,"5min":3.9870000000000001,"15min":3.9870000000000001},"last":1.7749999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.5820000000000001,"15min":1.605},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9990000000000001,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":1.718},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.637,"15min":1.677},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.855,"5min":3.367,"15min":3.367},"last":1.633}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":234445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0669999999999999,"15min":1.0609999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5700000000000001,"5min":1.776,"15min":1.776},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.115,"15min":1.1259999999999999},"min":{"1min":0.64900000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.4930000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.93500000000000005}]},{"osd":1,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1200000000000001,"15min":1.129},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5089999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.008},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1879999999999999,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6659999999999999,"5min":1.756,"15min":1.756},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.26,"15min":1.2769999999999999},"min":{"1min":0.89700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3720000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2310000000000001,"15min":1.2230000000000001},"min":{"1min":0.88900000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.294}]},{"osd":3,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.82199999999999995,"5min":0.75,"15min":0.75},"max":{"1min":1.524,"5min":1.736,"15min":1.736},"last":1.216},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.357,"15min":1.3640000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.738,"5min":2.121,"15min":2.121},"last":1.5329999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2929999999999999,"15min":1.2949999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8759999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.746,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6419999999999999,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.3380000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3819999999999999,"15min":1.3839999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.825,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.917},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.9450000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.7390000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.405,"15min":1.415},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.617,"5min":2.286,"15min":2.286},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.577,"15min":1.5940000000000001},"min":{"1min":1.008,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.9139999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":2.0489999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542182,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":229463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0589999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97999999999999998},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.85299999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0249999999999999,"15min":1.0449999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.458,"5min":1.78,"15min":1.78},"last":1.4630000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.97599999999999998,"15min":0.99199999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.24,"5min":1.623,"15min":1.623},"last":0.79400000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.55,"5min":3.7799999999999998,"15min":3.7799999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.202,"15min":1.2290000000000001},"min":{"1min":0.628,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5920000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1060000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3620000000000001,"15min":1.3799999999999999},"min":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.573},{"interface":"front","average":{"1min":1.286,"5min":1.369,"15min":1.383},"min":{"1min":0.88500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.7190000000000001,"5min":2.573,"15min":2.573},"last":1.53}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.5629999999999999,"15min":1.6000000000000001},"min":{"1min":0.99399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.514,"15min":1.5700000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6299999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":1.3109999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.534,"15min":1.5580000000000001},"min":{"1min":1.0569999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.873,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.6060000000000001,"15min":1.6459999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.758,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":1.1870000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.6499999999999999,"15min":1.6899999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.825,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":1.677},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.6379999999999999,"15min":1.6879999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.738,"5min":3.4340000000000002,"15min":3.4340000000000002},"last":1.6220000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":234467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89900000000000002,"15min":0.90300000000000002},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3009999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.755},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.024,"15min":1.036},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.405,"5min":1.865,"15min":1.865},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.383,"5min":1.468,"15min":1.468},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.079,"15min":1.091},"min":{"1min":0.64900000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.85099999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4570000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.2},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.879,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4139999999999999,"15min":1.4179999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.8759999999999999,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.93100000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.847,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.026}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.508,"15min":1.5229999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9730000000000001,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.304,"5min":1.4399999999999999,"15min":1.462},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6990000000000001,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.0489999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7829999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.472,"5min":1.5069999999999999,"15min":1.5129999999999999},"min":{"1min":0.97799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.0289999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.97399999999999998}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.6060000000000001,"15min":1.6279999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9279999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.593,"15min":1.6140000000000001},"min":{"1min":1.054,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.907,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":1.0169999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":512000,"data_stored":228895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.575,"15min":1.575},"last":0.999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0089999999999999,"15min":1.0149999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.167}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.026,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.782,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.0569999999999999,"15min":1.0609999999999999},"min":{"1min":0.64000000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.377,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.651}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.282,"15min":1.292},"min":{"1min":0.70999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6200000000000001,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":1.736},{"interface":"front","average":{"1min":1.3,"5min":1.2709999999999999,"15min":1.266},"min":{"1min":0.62,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.962,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":1.579}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.411,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.7410000000000001,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4199999999999999,"15min":1.4359999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.994,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":1.3720000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1059999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.25},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5009999999999999,"15min":1.51},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.8700000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.4370000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.524,"15min":1.518},"min":{"1min":1.0529999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.3500000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.786},{"interface":"front","average":{"1min":1.405,"5min":1.458,"15min":1.4670000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.919,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.7030000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.5209999999999999,"15min":1.524},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.0449999999999999,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.837},{"interface":"front","average":{"1min":1.621,"5min":1.617,"15min":1.617},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":2.4529999999999998,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.9570000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1105920,"data_stored":819227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.83799999999999997},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.109,"5min":1.288,"15min":1.288},"last":1.26},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":0.92100000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.998}]},{"osd":2,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.381},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0249999999999999,"15min":1.0129999999999999},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.048}]},{"osd":3,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.51200000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3080000000000001,"5min":1.522,"15min":1.522},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.137,"15min":1.149},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6879999999999999,"5min":1.728,"15min":1.728},"last":1.589}]},{"osd":4,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.323,"15min":1.3360000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.724,"5min":2.23,"15min":2.23},"last":1.095},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2869999999999999,"15min":1.298},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.49}]},{"osd":5,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.333,"15min":1.337},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.889,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":2.1299999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.4330000000000001,"15min":1.4490000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.7549999999999999,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":1.141}]},{"osd":6,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.3460000000000001,"15min":1.3700000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.677,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.413,"15min":1.4239999999999999},"min":{"1min":0.877,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":2.2509999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4990000000000001,"15min":1.5069999999999999},"min":{"1min":0.86199999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.99,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.177},{"interface":"front","average":{"1min":1.337,"5min":1.468,"15min":1.49},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.7729999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.639}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:53:41.954 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-08-11T15:53:41.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:53:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.187+0000 7fd43e9a4640 1 Processor -- start 2023-08-11T15:53:42.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.187+0000 7fd43e9a4640 1 -- start start 2023-08-11T15:53:42.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.188+0000 7fd43e9a4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd43806dba0 0x7fd43806e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.188+0000 7fd43e9a4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4380d0010 con 0x7fd43806f0b0 2023-08-11T15:53:42.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.188+0000 7fd43e9a4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43806b220 con 0x7fd43806ae40 2023-08-11T15:53:42.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.188+0000 7fd43e9a4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43806e570 con 0x7fd43806dba0 2023-08-11T15:53:42.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.189+0000 7fd4377fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd43806f0b0 0x7fd43815cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49620/0 (socket says 172.21.15.27:49620) 2023-08-11T15:53:42.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.189+0000 7fd4377fe640 1 -- 172.21.15.27:0/2798902341 learned_addr learned my addr 172.21.15.27:0/2798902341 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:42.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.189+0000 7fd43cf41640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd43806dba0 0x7fd43806e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.189+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000939871 0 0) 0x7fd420001490 con 0x7fd43806f0b0 2023-08-11T15:53:42.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.189+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd424003520 con 0x7fd43806f0b0 2023-08-11T15:53:42.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3077142411 0 0) 0x7fd424003520 con 0x7fd43806f0b0 2023-08-11T15:53:42.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd43806dba0 msgr2=0x7fd43806e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:42.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd43806dba0 0x7fd43806e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 >> v1:172.21.15.119:6789/0 conn(0x7fd43806ae40 legacy=0x7fd4380cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43806f590 con 0x7fd43806f0b0 2023-08-11T15:53:42.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fd420001b80 con 0x7fd43806f0b0 2023-08-11T15:53:42.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 >> v1:172.21.15.27:6789/0 conn(0x7fd43806f0b0 legacy=0x7fd43815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4240039a0 0x7fd424003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd43806dba0 0x7fd4240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd424004d00 0x7fd424005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43806e570 con 0x7fd424004d00 2023-08-11T15:53:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42400bae0 con 0x7fd4240039a0 2023-08-11T15:53:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42400bca0 con 0x7fd43806dba0 2023-08-11T15:53:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.190+0000 7fd43cf41640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd424004d00 0x7fd424005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.191+0000 7fd4377fe640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd43806dba0 0x7fd4240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.191+0000 7fd437fff640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4240039a0 0x7fd424003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.191+0000 7fd43cf41640 1 -- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd43806dba0 msgr2=0x7fd4240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:42.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.191+0000 7fd43cf41640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd43806dba0 0x7fd4240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.191+0000 7fd43cf41640 1 -- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4240039a0 msgr2=0x7fd424003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:42.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.191+0000 7fd43cf41640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4240039a0 0x7fd424003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.191+0000 7fd43cf41640 1 -- 172.21.15.27:0/2798902341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd424003520 con 0x7fd424004d00 2023-08-11T15:53:42.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.192+0000 7fd43cf41640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd424004d00 0x7fd424005180 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd430009ac0 tx=0x7fd4240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613b857d90857ac2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:42.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.192+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43002c6b0 con 0x7fd424004d00 2023-08-11T15:53:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.192+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43002c810 con 0x7fd424004d00 2023-08-11T15:53:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.192+0000 7fd436ffd640 1 -- 172.21.15.27:0/2798902341 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43002cba0 con 0x7fd424004d00 2023-08-11T15:53:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.193+0000 7fd43e9a4640 1 -- 172.21.15.27:0/2798902341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd424004d00 msgr2=0x7fd424005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.193+0000 7fd43e9a4640 1 --2- 172.21.15.27:0/2798902341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd424004d00 0x7fd424005180 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd430009ac0 tx=0x7fd4240057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.193+0000 7fd43cf41640 1 -- 172.21.15.27:0/2798902341 reap_dead start 2023-08-11T15:53:42.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.193+0000 7fd43e9a4640 1 -- 172.21.15.27:0/2798902341 shutdown_connections 2023-08-11T15:53:42.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.193+0000 7fd43e9a4640 1 -- 172.21.15.27:0/2798902341 >> 172.21.15.27:0/2798902341 conn(0x7fd4380d0220 msgr2=0x7fd4380d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:42.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.193+0000 7fd43e9a4640 1 -- 172.21.15.27:0/2798902341 shutdown_connections 2023-08-11T15:53:42.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.193+0000 7fd43e9a4640 1 -- 172.21.15.27:0/2798902341 wait complete. 2023-08-11T15:53:42.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.195+0000 7fd43e9a4640 1 Processor -- start 2023-08-11T15:53:42.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.195+0000 7fd43e9a4640 1 -- start start 2023-08-11T15:53:42.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.196+0000 7fd43e9a4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd424004d00 0x7fd438150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.196+0000 7fd43e9a4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4380c63e0 0x7fd4381e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.196+0000 7fd43e9a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4381e6e20 0x7fd43814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.196+0000 7fd43e9a4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4381e72a0 con 0x7fd4381e6e20 2023-08-11T15:53:42.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.196+0000 7fd43e9a4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43814fba0 con 0x7fd424004d00 2023-08-11T15:53:42.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.196+0000 7fd43e9a4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43814ef90 con 0x7fd4380c63e0 2023-08-11T15:53:42.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.196+0000 7fd43cf41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4381e6e20 0x7fd43814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.196+0000 7fd43cf41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4381e6e20 0x7fd43814dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51092/0 (socket says 172.21.15.27:51092) 2023-08-11T15:53:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.197+0000 7fd43cf41640 1 -- 172.21.15.27:0/4073608531 learned_addr learned my addr 172.21.15.27:0/4073608531 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.197+0000 7fd437fff640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd424004d00 0x7fd438150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.197+0000 7fd4377fe640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4380c63e0 0x7fd4381e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.197+0000 7fd43cf41640 1 -- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4380c63e0 msgr2=0x7fd4381e68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.197+0000 7fd43cf41640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4380c63e0 0x7fd4381e68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.197+0000 7fd43cf41640 1 -- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd424004d00 msgr2=0x7fd438150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.197+0000 7fd43cf41640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd424004d00 0x7fd438150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.197+0000 7fd43cf41640 1 -- 172.21.15.27:0/4073608531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43814f270 con 0x7fd4381e6e20 2023-08-11T15:53:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.197+0000 7fd43cf41640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4381e6e20 0x7fd43814dd50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd430037040 tx=0x7fd430004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.198+0000 7fd434ff9640 1 -- 172.21.15.27:0/4073608531 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430003810 con 0x7fd4381e6e20 2023-08-11T15:53:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.198+0000 7fd434ff9640 1 -- 172.21.15.27:0/4073608531 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd430003990 con 0x7fd4381e6e20 2023-08-11T15:53:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.198+0000 7fd434ff9640 1 -- 172.21.15.27:0/4073608531 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430003d40 con 0x7fd4381e6e20 2023-08-11T15:53:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.198+0000 7fd43e9a4640 1 -- 172.21.15.27:0/4073608531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43814f550 con 0x7fd4381e6e20 2023-08-11T15:53:42.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.198+0000 7fd43e9a4640 1 -- 172.21.15.27:0/4073608531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4380c6840 con 0x7fd4381e6e20 2023-08-11T15:53:42.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.199+0000 7fd43e9a4640 1 -- 172.21.15.27:0/4073608531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43814fd50 con 0x7fd4381e6e20 2023-08-11T15:53:42.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.200+0000 7fd434ff9640 1 -- 172.21.15.27:0/4073608531 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd43003e440 con 0x7fd4381e6e20 2023-08-11T15:53:42.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.201+0000 7fd434ff9640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd41806a050 0x7fd41806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.201+0000 7fd434ff9640 1 -- 172.21.15.27:0/4073608531 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd430000e40 con 0x7fd4381e6e20 2023-08-11T15:53:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.202+0000 7fd437fff640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd41806a050 0x7fd41806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.203+0000 7fd437fff640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd41806a050 0x7fd41806c510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd428007fd0 tx=0x7fd428009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.205+0000 7fd434ff9640 1 -- 172.21.15.27:0/4073608531 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fd43814fd50 con 0x7fd4381e6e20 2023-08-11T15:53:42.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.398+0000 7fd43e9a4640 1 -- 172.21.15.27:0/4073608531 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd43806fb50 con 0x7fd41806a050 2023-08-11T15:53:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.406+0000 7fd434ff9640 1 -- 172.21.15.27:0/4073608531 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70554 (secure 0 0 0) 0x7fd4280013b0 con 0x7fd41806a050 2023-08-11T15:53:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:42.411 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:53:42.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.409+0000 7fd43e9a4640 1 -- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd41806a050 msgr2=0x7fd41806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.409+0000 7fd43e9a4640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd41806a050 0x7fd41806c510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd428007fd0 tx=0x7fd428009040 comp rx=0 tx=0).stop 2023-08-11T15:53:42.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.409+0000 7fd43e9a4640 1 -- 172.21.15.27:0/4073608531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4381e6e20 msgr2=0x7fd43814dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.409+0000 7fd43e9a4640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4381e6e20 0x7fd43814dd50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd430037040 tx=0x7fd430004730 comp rx=0 tx=0).stop 2023-08-11T15:53:42.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.409+0000 7fd43e9a4640 1 -- 172.21.15.27:0/4073608531 shutdown_connections 2023-08-11T15:53:42.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.409+0000 7fd43e9a4640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fd41806a050 0x7fd41806c510 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.409+0000 7fd43e9a4640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4380c63e0 0x7fd4381e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.409+0000 7fd43e9a4640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4381e6e20 0x7fd43814dd50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.409+0000 7fd43e9a4640 1 --2- 172.21.15.27:0/4073608531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd424004d00 0x7fd438150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.409+0000 7fd43e9a4640 1 -- 172.21.15.27:0/4073608531 >> 172.21.15.27:0/4073608531 conn(0x7fd4380d0220 msgr2=0x7fd4380b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:42.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.410+0000 7fd43e9a4640 1 -- 172.21.15.27:0/4073608531 shutdown_connections 2023-08-11T15:53:42.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.410+0000 7fd43e9a4640 1 -- 172.21.15.27:0/4073608531 wait complete. 2023-08-11T15:53:42.434 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-08-11T15:53:40.772257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233648,"kb_used_data":5200,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754741072,"statfs":{"total":773094113280,"available":772854857728,"internally_reserved":0,"allocated":5324800,"data_stored":3024172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6418,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.217970"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.572082+0000","last_change":"2023-08-11T15:50:54.667601+0000","last_active":"2023-08-11T15:53:34.572082+0000","last_peered":"2023-08-11T15:53:34.572082+0000","last_clean":"2023-08-11T15:53:34.572082+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:53:34.572082+0000","last_undegraded":"2023-08-11T15:53:34.572082+0000","last_fullsized":"2023-08-11T15:53:34.572082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:14:58.155069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.566887+0000","last_change":"2023-08-11T15:50:54.662968+0000","last_active":"2023-08-11T15:53:34.566887+0000","last_peered":"2023-08-11T15:53:34.566887+0000","last_clean":"2023-08-11T15:53:34.566887+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:53:34.566887+0000","last_undegraded":"2023-08-11T15:53:34.566887+0000","last_fullsized":"2023-08-11T15:53:34.566887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:52:28.027008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.571589+0000","last_change":"2023-08-11T15:50:55.272783+0000","last_active":"2023-08-11T15:53:34.571589+0000","last_peered":"2023-08-11T15:53:34.571589+0000","last_clean":"2023-08-11T15:53:34.571589+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:53:34.571589+0000","last_undegraded":"2023-08-11T15:53:34.571589+0000","last_fullsized":"2023-08-11T15:53:34.571589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:55:04.564299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.566357+0000","last_change":"2023-08-11T15:50:55.270974+0000","last_active":"2023-08-11T15:53:34.566357+0000","last_peered":"2023-08-11T15:53:34.566357+0000","last_clean":"2023-08-11T15:53:34.566357+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:53:34.566357+0000","last_undegraded":"2023-08-11T15:53:34.566357+0000","last_fullsized":"2023-08-11T15:53:34.566357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:51:10.789686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.567805+0000","last_change":"2023-08-11T15:50:54.660862+0000","last_active":"2023-08-11T15:53:34.567805+0000","last_peered":"2023-08-11T15:53:34.567805+0000","last_clean":"2023-08-11T15:53:34.567805+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:53:34.567805+0000","last_undegraded":"2023-08-11T15:53:34.567805+0000","last_fullsized":"2023-08-11T15:53:34.567805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:38:26.058144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.558797+0000","last_change":"2023-08-11T15:50:54.660575+0000","last_active":"2023-08-11T15:53:34.558797+0000","last_peered":"2023-08-11T15:53:34.558797+0000","last_clean":"2023-08-11T15:53:34.558797+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:53:34.558797+0000","last_undegraded":"2023-08-11T15:53:34.558797+0000","last_fullsized":"2023-08-11T15:53:34.558797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:03.278614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":58,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.568881+0000","last_change":"2023-08-11T15:50:54.669675+0000","last_active":"2023-08-11T15:53:34.568881+0000","last_peered":"2023-08-11T15:53:34.568881+0000","last_clean":"2023-08-11T15:53:34.568881+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:53:34.568881+0000","last_undegraded":"2023-08-11T15:53:34.568881+0000","last_fullsized":"2023-08-11T15:53:34.568881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:51.224613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.568370+0000","last_change":"2023-08-11T15:50:54.663730+0000","last_active":"2023-08-11T15:53:34.568370+0000","last_peered":"2023-08-11T15:53:34.568370+0000","last_clean":"2023-08-11T15:53:34.568370+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:53:34.568370+0000","last_undegraded":"2023-08-11T15:53:34.568370+0000","last_fullsized":"2023-08-11T15:53:34.568370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:16:33.835392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":214,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-08-11T15:53:34.569244+0000","last_change":"2023-08-11T15:50:48.426393+0000","last_active":"2023-08-11T15:53:34.569244+0000","last_peered":"2023-08-11T15:53:34.569244+0000","last_clean":"2023-08-11T15:53:34.569244+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:53:34.569244+0000","last_undegraded":"2023-08-11T15:53:34.569244+0000","last_fullsized":"2023-08-11T15:53:34.569244+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_clean_scrub_stamp":"2023-08-11T15:50:47.133560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:22:12.270566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542182,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1105920,"data_stored":819283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.093,"15min":1.1160000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.216,"15min":1.222},"min":{"1min":0.80900000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5509999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3129999999999999,"15min":1.327},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.7370000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.6339999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.177,"15min":1.175},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.232,"5min":1.254,"15min":1.258},"min":{"1min":0.87,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5169999999999999,"5min":2.02,"15min":2.02},"last":0.97299999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2909999999999999,"15min":1.3069999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.671,"5min":1.869,"15min":1.869},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4890000000000001,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.103}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4219999999999999,"15min":1.4279999999999999},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.885,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.395,"15min":1.4139999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6910000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.6759999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.373,"15min":1.381},"min":{"1min":0.93500000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.212},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5449999999999999,"15min":1.5509999999999999},"min":{"1min":1.2170000000000001,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":1.9330000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.762}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.5029999999999999,"15min":1.5209999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.768,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.591},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.544,"15min":1.5580000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.861,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.321}]}]},{"osd":6,"up_from":14,"seq":60129542182,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":229497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0609999999999999,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1339999999999999,"15min":1.1359999999999999},"min":{"1min":0.751,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.181}]},{"osd":1,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.115,"15min":1.1140000000000001},"min":{"1min":0.73599999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.591,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.66100000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.95199999999999996}]},{"osd":2,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2390000000000001,"15min":1.24},"min":{"1min":0.85599999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.3180000000000001,"15min":1.3400000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5600000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.296}]},{"osd":3,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.369,"15min":1.377},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.77,"5min":2.774,"15min":2.774},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7330000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.383}]},{"osd":4,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.319,"15min":1.327},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8879999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":1.694},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.468,"15min":1.4910000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.665,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.4350000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.496,"15min":1.5069999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8149999999999999,"5min":3.371,"15min":3.371},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.486,"15min":1.504},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0009999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.546}]},{"osd":7,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.615,"15min":1.6479999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.762,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":1.5},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.6739999999999999,"15min":1.704},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.921,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.4690000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":228895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0389999999999999,"15min":1.0269999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.1499999999999999,"15min":1.163},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2569999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1419999999999999,"15min":1.1399999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.401},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.93999999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.621,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.196}]},{"osd":2,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.262,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.615,"5min":1.827,"15min":1.827},"last":1.129},{"interface":"front","average":{"1min":1.194,"5min":1.351,"15min":1.377},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8300000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.5860000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.353,"15min":1.353},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.702,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.339},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.444},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.782,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.476}]},{"osd":4,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4379999999999999,"15min":1.466},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8779999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.679},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5880000000000001,"15min":1.6080000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.847,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":1.8009999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.4690000000000001,"15min":1.492},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.796,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.6120000000000001,"15min":1.6379999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.2160000000000002,"5min":3.9870000000000001,"15min":3.9870000000000001},"last":1.7749999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.5820000000000001,"15min":1.605},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9990000000000001,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":1.718},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.637,"15min":1.677},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.855,"5min":3.367,"15min":3.367},"last":1.633}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":234445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0669999999999999,"15min":1.0609999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5700000000000001,"5min":1.776,"15min":1.776},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.115,"15min":1.1259999999999999},"min":{"1min":0.64900000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.4930000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.93500000000000005}]},{"osd":1,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1200000000000001,"15min":1.129},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5089999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.008},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1879999999999999,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6659999999999999,"5min":1.756,"15min":1.756},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.26,"15min":1.2769999999999999},"min":{"1min":0.89700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3720000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2310000000000001,"15min":1.2230000000000001},"min":{"1min":0.88900000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.294}]},{"osd":3,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.82199999999999995,"5min":0.75,"15min":0.75},"max":{"1min":1.524,"5min":1.736,"15min":1.736},"last":1.216},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.357,"15min":1.3640000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.738,"5min":2.121,"15min":2.121},"last":1.5329999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2929999999999999,"15min":1.2949999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8759999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.746,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6419999999999999,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.3380000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3819999999999999,"15min":1.3839999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.825,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.917},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.9450000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.7390000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.405,"15min":1.415},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.617,"5min":2.286,"15min":2.286},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.577,"15min":1.5940000000000001},"min":{"1min":1.008,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.9139999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":2.0489999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542182,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":229463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0589999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97999999999999998},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.85299999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0249999999999999,"15min":1.0449999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.458,"5min":1.78,"15min":1.78},"last":1.4630000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.97599999999999998,"15min":0.99199999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.24,"5min":1.623,"15min":1.623},"last":0.79400000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.55,"5min":3.7799999999999998,"15min":3.7799999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.202,"15min":1.2290000000000001},"min":{"1min":0.628,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5920000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1060000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3620000000000001,"15min":1.3799999999999999},"min":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.573},{"interface":"front","average":{"1min":1.286,"5min":1.369,"15min":1.383},"min":{"1min":0.88500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.7190000000000001,"5min":2.573,"15min":2.573},"last":1.53}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.5629999999999999,"15min":1.6000000000000001},"min":{"1min":0.99399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.514,"15min":1.5700000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6299999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":1.3109999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.534,"15min":1.5580000000000001},"min":{"1min":1.0569999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.873,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.6060000000000001,"15min":1.6459999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.758,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":1.1870000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.6499999999999999,"15min":1.6899999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.825,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":1.677},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.6379999999999999,"15min":1.6879999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.738,"5min":3.4340000000000002,"15min":3.4340000000000002},"last":1.6220000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":234467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89900000000000002,"15min":0.90300000000000002},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3009999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.755},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.024,"15min":1.036},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.405,"5min":1.865,"15min":1.865},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.383,"5min":1.468,"15min":1.468},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.079,"15min":1.091},"min":{"1min":0.64900000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.85099999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4570000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.2},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.879,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4139999999999999,"15min":1.4179999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.8759999999999999,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.93100000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.847,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.026}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.508,"15min":1.5229999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9730000000000001,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.304,"5min":1.4399999999999999,"15min":1.462},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6990000000000001,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.0489999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7829999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.472,"5min":1.5069999999999999,"15min":1.5129999999999999},"min":{"1min":0.97799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.0289999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.97399999999999998}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.6060000000000001,"15min":1.6279999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9279999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.593,"15min":1.6140000000000001},"min":{"1min":1.054,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.907,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":1.0169999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":512000,"data_stored":228895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.575,"15min":1.575},"last":0.999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0089999999999999,"15min":1.0149999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.167}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.026,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.782,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.0569999999999999,"15min":1.0609999999999999},"min":{"1min":0.64000000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.377,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.651}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.282,"15min":1.292},"min":{"1min":0.70999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6200000000000001,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":1.736},{"interface":"front","average":{"1min":1.3,"5min":1.2709999999999999,"15min":1.266},"min":{"1min":0.62,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.962,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":1.579}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.411,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.7410000000000001,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4199999999999999,"15min":1.4359999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.994,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":1.3720000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1059999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.25},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5009999999999999,"15min":1.51},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.8700000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.4370000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.524,"15min":1.518},"min":{"1min":1.0529999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.3500000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.786},{"interface":"front","average":{"1min":1.405,"5min":1.458,"15min":1.4670000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.919,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.7030000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.5209999999999999,"15min":1.524},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.0449999999999999,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.837},{"interface":"front","average":{"1min":1.621,"5min":1.617,"15min":1.617},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":2.4529999999999998,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.9570000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1105920,"data_stored":819227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.83799999999999997},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.109,"5min":1.288,"15min":1.288},"last":1.26},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":0.92100000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.998}]},{"osd":2,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.381},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0249999999999999,"15min":1.0129999999999999},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.048}]},{"osd":3,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.51200000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3080000000000001,"5min":1.522,"15min":1.522},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.137,"15min":1.149},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6879999999999999,"5min":1.728,"15min":1.728},"last":1.589}]},{"osd":4,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.323,"15min":1.3360000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.724,"5min":2.23,"15min":2.23},"last":1.095},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2869999999999999,"15min":1.298},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.49}]},{"osd":5,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.333,"15min":1.337},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.889,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":2.1299999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.4330000000000001,"15min":1.4490000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.7549999999999999,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":1.141}]},{"osd":6,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.3460000000000001,"15min":1.3700000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.677,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.413,"15min":1.4239999999999999},"min":{"1min":0.877,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":2.2509999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4990000000000001,"15min":1.5069999999999999},"min":{"1min":0.86199999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.99,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.177},{"interface":"front","average":{"1min":1.337,"5min":1.468,"15min":1.49},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.7729999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.639}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:53:42.436 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:53:42.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.664+0000 7fee2b457640 1 Processor -- start 2023-08-11T15:53:42.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.664+0000 7fee2b457640 1 -- start start 2023-08-11T15:53:42.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.666+0000 7fee2b457640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee240cb430 0x7fee240b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.666+0000 7fee2b457640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee240c8a30 con 0x7fee2406f5d0 2023-08-11T15:53:42.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.666+0000 7fee2b457640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2406c0c0 con 0x7fee2415a140 2023-08-11T15:53:42.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.666+0000 7fee2b457640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2406c3a0 con 0x7fee240cb430 2023-08-11T15:53:42.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.666+0000 7fee299f4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fee2406f5d0 0x7fee240c8ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49630/0 (socket says 172.21.15.27:49630) 2023-08-11T15:53:42.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.666+0000 7fee299f4640 1 -- 172.21.15.27:0/2507692826 learned_addr learned my addr 172.21.15.27:0/2507692826 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:42.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.666+0000 7fee289f2640 1 --2- 172.21.15.27:0/2507692826 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee240cb430 0x7fee240b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.667+0000 7fee1bfff640 1 -- 172.21.15.27:0/2507692826 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280952892 0 0) 0x7fee240c8a30 con 0x7fee2406f5d0 2023-08-11T15:53:42.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.667+0000 7fee1bfff640 1 -- 172.21.15.27:0/2507692826 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee10003520 con 0x7fee2406f5d0 2023-08-11T15:53:42.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.667+0000 7fee1bfff640 1 -- 172.21.15.27:0/2507692826 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856698121 0 0) 0x7fee2406c0c0 con 0x7fee2415a140 2023-08-11T15:53:42.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.667+0000 7fee1bfff640 1 -- 172.21.15.27:0/2507692826 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee240c8a30 con 0x7fee2415a140 2023-08-11T15:53:42.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.667+0000 7fee289f2640 1 -- 172.21.15.27:0/2507692826 >> v1:172.21.15.119:6789/0 conn(0x7fee2415a140 legacy=0x7fee2406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.667+0000 7fee1bfff640 1 -- 172.21.15.27:0/2507692826 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497721371 0 0) 0x7fee10003520 con 0x7fee2406f5d0 2023-08-11T15:53:42.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.667+0000 7fee289f2640 1 -- 172.21.15.27:0/2507692826 >> v1:172.21.15.27:6789/0 conn(0x7fee2406f5d0 legacy=0x7fee240c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.667+0000 7fee289f2640 1 -- 172.21.15.27:0/2507692826 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2406c680 con 0x7fee240cb430 2023-08-11T15:53:42.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.669+0000 7fee289f2640 1 --2- 172.21.15.27:0/2507692826 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee240cb430 0x7fee240b0e70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fee140097d0 tx=0x7fee1402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10276b34d77a1ae server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:42.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.669+0000 7fee1bfff640 1 -- 172.21.15.27:0/2507692826 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c001670 con 0x7fee240cb430 2023-08-11T15:53:42.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.669+0000 7fee1bfff640 1 -- 172.21.15.27:0/2507692826 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1402e050 con 0x7fee240cb430 2023-08-11T15:53:42.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.669+0000 7fee1bfff640 1 -- 172.21.15.27:0/2507692826 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14030710 con 0x7fee240cb430 2023-08-11T15:53:42.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.670+0000 7fee2b457640 1 -- 172.21.15.27:0/2507692826 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee240cb430 msgr2=0x7fee240b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.670+0000 7fee2b457640 1 --2- 172.21.15.27:0/2507692826 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee240cb430 0x7fee240b0e70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fee140097d0 tx=0x7fee1402b800 comp rx=0 tx=0).stop 2023-08-11T15:53:42.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.670+0000 7fee2b457640 1 -- 172.21.15.27:0/2507692826 shutdown_connections 2023-08-11T15:53:42.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.670+0000 7fee2b457640 1 --2- 172.21.15.27:0/2507692826 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee240cb430 0x7fee240b0e70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.670+0000 7fee2b457640 1 -- 172.21.15.27:0/2507692826 >> 172.21.15.27:0/2507692826 conn(0x7fee240afb70 msgr2=0x7fee240aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:42.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.670+0000 7fee2b457640 1 -- 172.21.15.27:0/2507692826 shutdown_connections 2023-08-11T15:53:42.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.670+0000 7fee2b457640 1 -- 172.21.15.27:0/2507692826 wait complete. 2023-08-11T15:53:42.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.672+0000 7fee2b457640 1 Processor -- start 2023-08-11T15:53:42.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.672+0000 7fee2b457640 1 -- start start 2023-08-11T15:53:42.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.673+0000 7fee2b457640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee2406f5d0 0x7fee240c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.673+0000 7fee2b457640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee240cb430 0x7fee24158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.673+0000 7fee2b457640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2415a140 0x7fee2414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.673+0000 7fee2b457640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee240c65c0 con 0x7fee2415a140 2023-08-11T15:53:42.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.673+0000 7fee2b457640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee240c68a0 con 0x7fee240cb430 2023-08-11T15:53:42.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.673+0000 7fee2b457640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee240c6b80 con 0x7fee2406f5d0 2023-08-11T15:53:42.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.673+0000 7fee299f4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2415a140 0x7fee2414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.673+0000 7fee299f4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2415a140 0x7fee2414b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51106/0 (socket says 172.21.15.27:51106) 2023-08-11T15:53:42.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.673+0000 7fee299f4640 1 -- 172.21.15.27:0/4272917229 learned_addr learned my addr 172.21.15.27:0/4272917229 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:42.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.674+0000 7fee291f3640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee2406f5d0 0x7fee240c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.674+0000 7fee289f2640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee240cb430 0x7fee24158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.674+0000 7fee299f4640 1 -- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee2406f5d0 msgr2=0x7fee240c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.674+0000 7fee299f4640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee2406f5d0 0x7fee240c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.674+0000 7fee299f4640 1 -- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee240cb430 msgr2=0x7fee24158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.674+0000 7fee299f4640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee240cb430 0x7fee24158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.674+0000 7fee299f4640 1 -- 172.21.15.27:0/4272917229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee14009400 con 0x7fee2415a140 2023-08-11T15:53:42.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.674+0000 7fee299f4640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2415a140 0x7fee2414b3c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00c6d0 tx=0x7fee1c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:42.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.675+0000 7fee19ffb640 1 -- 172.21.15.27:0/4272917229 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c005ad0 con 0x7fee2415a140 2023-08-11T15:53:42.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.675+0000 7fee19ffb640 1 -- 172.21.15.27:0/4272917229 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c005c30 con 0x7fee2415a140 2023-08-11T15:53:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.675+0000 7fee19ffb640 1 -- 172.21.15.27:0/4272917229 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c007070 con 0x7fee2415a140 2023-08-11T15:53:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.675+0000 7fee2b457640 1 -- 172.21.15.27:0/4272917229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee240c6e60 con 0x7fee2415a140 2023-08-11T15:53:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.675+0000 7fee2b457640 1 -- 172.21.15.27:0/4272917229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee240c7660 con 0x7fee2415a140 2023-08-11T15:53:42.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.676+0000 7fee2b457640 1 -- 172.21.15.27:0/4272917229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee240c7380 con 0x7fee2415a140 2023-08-11T15:53:42.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.677+0000 7fee19ffb640 1 -- 172.21.15.27:0/4272917229 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fee1c0071d0 con 0x7fee2415a140 2023-08-11T15:53:42.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.678+0000 7fee19ffb640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee0806a050 0x7fee0806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:42.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.678+0000 7fee19ffb640 1 -- 172.21.15.27:0/4272917229 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fee1c088130 con 0x7fee2415a140 2023-08-11T15:53:42.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.679+0000 7fee291f3640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee0806a050 0x7fee0806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:42.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.680+0000 7fee291f3640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee0806a050 0x7fee0806c510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fee200015d0 tx=0x7fee20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:42.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.682+0000 7fee19ffb640 1 -- 172.21.15.27:0/4272917229 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fee240c7380 con 0x7fee2415a140 2023-08-11T15:53:42.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.880+0000 7fee2b457640 1 -- 172.21.15.27:0/4272917229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee240c7380 con 0x7fee2415a140 2023-08-11T15:53:42.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.882+0000 7fee19ffb640 1 -- 172.21.15.27:0/4272917229 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14971 (secure 0 0 0) 0x7fee240c7380 con 0x7fee2415a140 2023-08-11T15:53:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"fsid":"bee60691-4c46-4397-a560-d178aa4db950","created":"2023-08-11T15:50:33.625327+0000","modified":"2023-08-11T15:53:37.787934+0000","last_up_change":"2023-08-11T15:50:46.104102+0000","last_in_change":"2023-08-11T15:50:37.090267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:50:46.794326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-11T15:50:53.535004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6070354-9b1d-4274-9672-2ae236bbb57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6801","nonce":3363308208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6803","nonce":3363308208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6807","nonce":3363308208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3363308208},{"type":"v1","addr":"172.21.15.27:6805","nonce":3363308208}]},"public_addr":"172.21.15.27:6801/3363308208","cluster_addr":"172.21.15.27:6803/3363308208","heartbeat_back_addr":"172.21.15.27:6807/3363308208","heartbeat_front_addr":"172.21.15.27:6805/3363308208","state":["exists","up"]},{"osd":1,"uuid":"2eed2b1c-7d9f-40fb-a154-cbf8b99dd988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6809","nonce":889585576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6811","nonce":889585576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6815","nonce":889585576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":889585576},{"type":"v1","addr":"172.21.15.27:6813","nonce":889585576}]},"public_addr":"172.21.15.27:6809/889585576","cluster_addr":"172.21.15.27:6811/889585576","heartbeat_back_addr":"172.21.15.27:6815/889585576","heartbeat_front_addr":"172.21.15.27:6813/889585576","state":["exists","up"]},{"osd":2,"uuid":"1b1cef5e-9225-4bf6-a05a-1ef4ce488827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6823","nonce":2059564233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6827","nonce":2059564233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6831","nonce":2059564233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2059564233},{"type":"v1","addr":"172.21.15.27:6829","nonce":2059564233}]},"public_addr":"172.21.15.27:6823/2059564233","cluster_addr":"172.21.15.27:6827/2059564233","heartbeat_back_addr":"172.21.15.27:6831/2059564233","heartbeat_front_addr":"172.21.15.27:6829/2059564233","state":["exists","up"]},{"osd":3,"uuid":"79b66624-0fba-44ac-8abf-98da48240e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6817","nonce":4099583167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6819","nonce":4099583167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6826","nonce":4099583167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":4099583167},{"type":"v1","addr":"172.21.15.27:6822","nonce":4099583167}]},"public_addr":"172.21.15.27:6817/4099583167","cluster_addr":"172.21.15.27:6819/4099583167","heartbeat_back_addr":"172.21.15.27:6826/4099583167","heartbeat_front_addr":"172.21.15.27:6822/4099583167","state":["exists","up"]},{"osd":4,"uuid":"b7d2aeeb-47c1-4156-9720-82225a6b196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6809","nonce":2943646764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6811","nonce":2943646764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6815","nonce":2943646764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2943646764},{"type":"v1","addr":"172.21.15.119:6813","nonce":2943646764}]},"public_addr":"172.21.15.119:6809/2943646764","cluster_addr":"172.21.15.119:6811/2943646764","heartbeat_back_addr":"172.21.15.119:6815/2943646764","heartbeat_front_addr":"172.21.15.119:6813/2943646764","state":["exists","up"]},{"osd":5,"uuid":"4623837e-0e1a-42c4-8394-ec0fa2ba4a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6817","nonce":1811178752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6819","nonce":1811178752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6823","nonce":1811178752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1811178752},{"type":"v1","addr":"172.21.15.119:6821","nonce":1811178752}]},"public_addr":"172.21.15.119:6817/1811178752","cluster_addr":"172.21.15.119:6819/1811178752","heartbeat_back_addr":"172.21.15.119:6823/1811178752","heartbeat_front_addr":"172.21.15.119:6821/1811178752","state":["exists","up"]},{"osd":6,"uuid":"f0604e90-c352-436c-92de-01c1d363207b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6801","nonce":2672123554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6803","nonce":2672123554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6807","nonce":2672123554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2672123554},{"type":"v1","addr":"172.21.15.119:6805","nonce":2672123554}]},"public_addr":"172.21.15.119:6801/2672123554","cluster_addr":"172.21.15.119:6803/2672123554","heartbeat_back_addr":"172.21.15.119:6807/2672123554","heartbeat_front_addr":"172.21.15.119:6805/2672123554","state":["exists","up"]},{"osd":7,"uuid":"0ae153ef-2a91-4eaa-ad38-adcf4dbba3d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6825","nonce":1341885104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6827","nonce":1341885104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6831","nonce":1341885104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1341885104},{"type":"v1","addr":"172.21.15.119:6829","nonce":1341885104}]},"public_addr":"172.21.15.119:6825/1341885104","cluster_addr":"172.21.15.119:6827/1341885104","heartbeat_back_addr":"172.21.15.119:6831/1341885104","heartbeat_front_addr":"172.21.15.119:6829/1341885104","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.378141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.782954+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.245511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.522745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.205104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.444194+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.442588+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:50:42.487396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/899405615":"2023-08-12T15:53:34.547127+0000","172.21.15.27:6834/899405615":"2023-08-12T15:53:34.547127+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:53:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.885+0000 7fee2b457640 1 -- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee0806a050 msgr2=0x7fee0806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.885+0000 7fee2b457640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee0806a050 0x7fee0806c510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fee200015d0 tx=0x7fee20009040 comp rx=0 tx=0).stop 2023-08-11T15:53:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.885+0000 7fee2b457640 1 -- 172.21.15.27:0/4272917229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2415a140 msgr2=0x7fee2414b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.885+0000 7fee2b457640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2415a140 0x7fee2414b3c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00c6d0 tx=0x7fee1c00cb90 comp rx=0 tx=0).stop 2023-08-11T15:53:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.885+0000 7fee2b457640 1 -- 172.21.15.27:0/4272917229 shutdown_connections 2023-08-11T15:53:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.885+0000 7fee2b457640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fee0806a050 0x7fee0806c510 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.885+0000 7fee2b457640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2415a140 0x7fee2414b3c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.885+0000 7fee2b457640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee240cb430 0x7fee24158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.885+0000 7fee2b457640 1 --2- 172.21.15.27:0/4272917229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee2406f5d0 0x7fee240c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.885+0000 7fee2b457640 1 -- 172.21.15.27:0/4272917229 >> 172.21.15.27:0/4272917229 conn(0x7fee240afb70 msgr2=0x7fee2406b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.886+0000 7fee2b457640 1 -- 172.21.15.27:0/4272917229 shutdown_connections 2023-08-11T15:53:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:42.886+0000 7fee2b457640 1 -- 172.21.15.27:0/4272917229 wait complete. 2023-08-11T15:53:43.908 INFO:tasks.ceph:Scrubbing osd.0 2023-08-11T15:53:43.909 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-08-11T15:53:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.161+0000 7fcf4c426640 1 Processor -- start 2023-08-11T15:53:44.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.162+0000 7fcf4c426640 1 -- start start 2023-08-11T15:53:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.164+0000 7fcf4c426640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcf440cb0c0 0x7fcf4415a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.164+0000 7fcf4c426640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf440cbb70 0x7fcf440708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.164+0000 7fcf4c426640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf440cc020 con 0x7fcf44156e50 2023-08-11T15:53:44.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.164+0000 7fcf4c426640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf440ca460 con 0x7fcf440cb0c0 2023-08-11T15:53:44.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.164+0000 7fcf4c426640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf44070470 con 0x7fcf440cbb70 2023-08-11T15:53:44.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.164+0000 7fcf499c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcf440cb0c0 0x7fcf4415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.164+0000 7fcf4a9c3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf440cbb70 0x7fcf440708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.164+0000 7fcf4a1c2640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcf44156e50 0x7fcf440ca9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.27:42748/0 (socket says 172.21.15.27:42748) 2023-08-11T15:53:44.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.164+0000 7fcf4a1c2640 1 -- 172.21.15.27:0/799643269 learned_addr learned my addr 172.21.15.27:0/799643269 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:44.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.165+0000 7fcf491c0640 1 -- 172.21.15.27:0/799643269 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335573839 0 0) 0x7fcf440cc020 con 0x7fcf44156e50 2023-08-11T15:53:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.165+0000 7fcf491c0640 1 -- 172.21.15.27:0/799643269 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf30003520 con 0x7fcf44156e50 2023-08-11T15:53:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.165+0000 7fcf4a9c3640 1 -- 172.21.15.27:0/799643269 >> v2:172.21.15.119:3300/0 conn(0x7fcf440cb0c0 msgr2=0x7fcf4415a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.165+0000 7fcf4a9c3640 1 --2- 172.21.15.27:0/799643269 >> v2:172.21.15.119:3300/0 conn(0x7fcf440cb0c0 0x7fcf4415a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.165+0000 7fcf4a9c3640 1 -- 172.21.15.27:0/799643269 >> v1:172.21.15.119:6789/0 conn(0x7fcf44156e50 legacy=0x7fcf440ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.165+0000 7fcf4a9c3640 1 -- 172.21.15.27:0/799643269 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440cc270 con 0x7fcf440cbb70 2023-08-11T15:53:44.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.167+0000 7fcf4a9c3640 1 --2- 172.21.15.27:0/799643269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf440cbb70 0x7fcf440708c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009d90 tx=0x7fcf3c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d92ee85bc88eb51 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.167+0000 7fcf491c0640 1 -- 172.21.15.27:0/799643269 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c02cc50 con 0x7fcf440cbb70 2023-08-11T15:53:44.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.168+0000 7fcf491c0640 1 -- 172.21.15.27:0/799643269 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c02cdb0 con 0x7fcf440cbb70 2023-08-11T15:53:44.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.168+0000 7fcf491c0640 1 -- 172.21.15.27:0/799643269 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c034660 con 0x7fcf440cbb70 2023-08-11T15:53:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.168+0000 7fcf4c426640 1 -- 172.21.15.27:0/799643269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf440cbb70 msgr2=0x7fcf440708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.168+0000 7fcf4c426640 1 --2- 172.21.15.27:0/799643269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf440cbb70 0x7fcf440708c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009d90 tx=0x7fcf3c02bea0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.168+0000 7fcf4c426640 1 -- 172.21.15.27:0/799643269 shutdown_connections 2023-08-11T15:53:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.168+0000 7fcf4c426640 1 --2- 172.21.15.27:0/799643269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf440cbb70 0x7fcf440708c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.168+0000 7fcf4c426640 1 --2- 172.21.15.27:0/799643269 >> v2:172.21.15.119:3300/0 conn(0x7fcf440cb0c0 0x7fcf4415a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.168+0000 7fcf4c426640 1 -- 172.21.15.27:0/799643269 >> 172.21.15.27:0/799643269 conn(0x7fcf440afb70 msgr2=0x7fcf440aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:44.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.169+0000 7fcf4c426640 1 -- 172.21.15.27:0/799643269 shutdown_connections 2023-08-11T15:53:44.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.169+0000 7fcf4c426640 1 -- 172.21.15.27:0/799643269 wait complete. 2023-08-11T15:53:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.170+0000 7fcf4c426640 1 Processor -- start 2023-08-11T15:53:44.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.171+0000 7fcf4c426640 1 -- start start 2023-08-11T15:53:44.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf4c426640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf440cb0c0 0x7fcf4414d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf4c426640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440cbb70 0x7fcf440b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf4c426640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf44156e50 0x7fcf441512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf4c426640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4414da50 con 0x7fcf440cbb70 2023-08-11T15:53:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf4c426640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440b1ea0 con 0x7fcf440cb0c0 2023-08-11T15:53:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf4c426640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440b2180 con 0x7fcf44156e50 2023-08-11T15:53:44.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf499c1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440cbb70 0x7fcf440b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf499c1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440cbb70 0x7fcf440b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51120/0 (socket says 172.21.15.27:51120) 2023-08-11T15:53:44.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf499c1640 1 -- 172.21.15.27:0/1341312647 learned_addr learned my addr 172.21.15.27:0/1341312647 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:44.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf4a9c3640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf44156e50 0x7fcf441512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.172+0000 7fcf4a1c2640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf440cb0c0 0x7fcf4414d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf499c1640 1 -- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf44156e50 msgr2=0x7fcf441512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf499c1640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf44156e50 0x7fcf441512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf499c1640 1 -- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf440cb0c0 msgr2=0x7fcf4414d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf499c1640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf440cb0c0 0x7fcf4414d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf499c1640 1 -- 172.21.15.27:0/1341312647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c009a70 con 0x7fcf440cbb70 2023-08-11T15:53:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf4a1c2640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf440cb0c0 0x7fcf4414d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:44.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf4a9c3640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcf44156e50 0x7fcf441512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:44.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf499c1640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440cbb70 0x7fcf440b1670 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400e870 tx=0x7fcf3400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf3affd640 1 -- 172.21.15.27:0/1341312647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf340094f0 con 0x7fcf440cbb70 2023-08-11T15:53:44.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf3affd640 1 -- 172.21.15.27:0/1341312647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3400c940 con 0x7fcf440cbb70 2023-08-11T15:53:44.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf440b2460 con 0x7fcf440cbb70 2023-08-11T15:53:44.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.173+0000 7fcf3affd640 1 -- 172.21.15.27:0/1341312647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400ccf0 con 0x7fcf440cbb70 2023-08-11T15:53:44.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.174+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf440b1bb0 con 0x7fcf440cbb70 2023-08-11T15:53:44.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.174+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf440cc830 con 0x7fcf440cbb70 2023-08-11T15:53:44.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.176+0000 7fcf3affd640 1 -- 172.21.15.27:0/1341312647 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcf34013470 con 0x7fcf440cbb70 2023-08-11T15:53:44.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.177+0000 7fcf3affd640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf2806a050 0x7fcf2806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.177+0000 7fcf3affd640 1 -- 172.21.15.27:0/1341312647 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fcf340885c0 con 0x7fcf440cbb70 2023-08-11T15:53:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.177+0000 7fcf3affd640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] conn(0x7fcf28070970 0x7fcf28072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.177+0000 7fcf3affd640 1 -- 172.21.15.27:0/1341312647 --> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf28073480 con 0x7fcf28070970 2023-08-11T15:53:44.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.177+0000 7fcf3affd640 1 -- 172.21.15.27:0/1341312647 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf3408aba0 con 0x7fcf440cbb70 2023-08-11T15:53:44.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.177+0000 7fcf4a9c3640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] conn(0x7fcf28070970 0x7fcf28072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.177+0000 7fcf4a1c2640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf2806a050 0x7fcf2806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.178+0000 7fcf4a9c3640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] conn(0x7fcf28070970 0x7fcf28072db0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.178+0000 7fcf4a1c2640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf2806a050 0x7fcf2806c510 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcf40001230 tx=0x7fcf40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.179+0000 7fcf3affd640 1 -- 172.21.15.27:0/1341312647 <== osd.0 v2:172.21.15.27:6800/3363308208 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcf28073480 con 0x7fcf28070970 2023-08-11T15:53:44.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.193+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 --> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcf440cc550 con 0x7fcf28070970 2023-08-11T15:53:44.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.193+0000 7fcf3affd640 1 -- 172.21.15.27:0/1341312647 <== osd.0 v2:172.21.15.27:6800/3363308208 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcf3c02cb70 con 0x7fcf28070970 2023-08-11T15:53:44.199 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:44.200 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-11T15:53:44.200 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:44.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 >> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] conn(0x7fcf28070970 msgr2=0x7fcf28072db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4c426640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.27:6800/3363308208,v1:172.21.15.27:6801/3363308208] conn(0x7fcf28070970 0x7fcf28072db0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf2806a050 msgr2=0x7fcf2806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4c426640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fcf2806a050 0x7fcf2806c510 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcf40001230 tx=0x7fcf40009040 comp rx=0 tx=0).stop 2023-08-11T15:53:44.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440cbb70 msgr2=0x7fcf440b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4c426640 1 --2- 172.21.15.27:0/1341312647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440cbb70 0x7fcf440b1670 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400e870 tx=0x7fcf3400ed30 comp rx=0 tx=0).stop 2023-08-11T15:53:44.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4a9c3640 1 -- 172.21.15.27:0/1341312647 reap_dead start 2023-08-11T15:53:44.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 shutdown_connections 2023-08-11T15:53:44.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 >> 172.21.15.27:0/1341312647 conn(0x7fcf440afb70 msgr2=0x7fcf440ca9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:44.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 shutdown_connections 2023-08-11T15:53:44.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.194+0000 7fcf4c426640 1 -- 172.21.15.27:0/1341312647 wait complete. 2023-08-11T15:53:44.213 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-08-11T15:53:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.444+0000 7f8e796ec640 1 Processor -- start 2023-08-11T15:53:44.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.445+0000 7f8e796ec640 1 -- start start 2023-08-11T15:53:44.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.446+0000 7f8e796ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e740b55c0 0x7f8e740cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.446+0000 7f8e796ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8e740cf930 0x7f8e740d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.446+0000 7f8e796ec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e740cda30 con 0x7f8e7415c620 2023-08-11T15:53:44.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.446+0000 7f8e796ec640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e740ccc10 con 0x7f8e740cf930 2023-08-11T15:53:44.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.446+0000 7f8e796ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740cfda0 con 0x7f8e740b55c0 2023-08-11T15:53:44.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.446+0000 7f8e72ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8e7415c620 0x7f8e740b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49644/0 (socket says 172.21.15.27:49644) 2023-08-11T15:53:44.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.446+0000 7f8e72ffd640 1 -- 172.21.15.27:0/856066292 learned_addr learned my addr 172.21.15.27:0/856066292 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:44.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e727fc640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e740b55c0 0x7f8e740cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423401246 0 0) 0x7f8e740cda30 con 0x7f8e7415c620 2023-08-11T15:53:44.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e60003520 con 0x7f8e7415c620 2023-08-11T15:53:44.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e737fe640 1 --2- 172.21.15.27:0/856066292 >> v2:172.21.15.119:3300/0 conn(0x7f8e740cf930 0x7f8e740d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2794544621 0 0) 0x7f8e60003520 con 0x7f8e7415c620 2023-08-11T15:53:44.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e740b55c0 msgr2=0x7f8e740cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:44.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e740b55c0 0x7f8e740cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 >> v2:172.21.15.119:3300/0 conn(0x7f8e740cf930 msgr2=0x7f8e740d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:44.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 --2- 172.21.15.27:0/856066292 >> v2:172.21.15.119:3300/0 conn(0x7f8e740cf930 0x7f8e740d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e74161110 con 0x7f8e7415c620 2023-08-11T15:53:44.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f8e640019f0 con 0x7f8e7415c620 2023-08-11T15:53:44.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e737fe640 1 --2- 172.21.15.27:0/856066292 >> v2:172.21.15.119:3300/0 conn(0x7f8e740cf930 0x7f8e740d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:44.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 >> v1:172.21.15.27:6789/0 conn(0x7f8e7415c620 legacy=0x7f8e740b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e60003970 0x7f8e60003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e740b55c0 0x7f8e600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e60004d00 0x7f8e60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740ccc10 con 0x7f8e60003970 2023-08-11T15:53:44.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740cfda0 con 0x7f8e60004d00 2023-08-11T15:53:44.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.447+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6000b980 con 0x7f8e740b55c0 2023-08-11T15:53:44.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.448+0000 7f8e72ffd640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e60003970 0x7f8e60003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.448+0000 7f8e727fc640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e740b55c0 0x7f8e600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.448+0000 7f8e72ffd640 1 -- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e740b55c0 msgr2=0x7f8e600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:44.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.448+0000 7f8e72ffd640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e740b55c0 0x7f8e600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.448+0000 7f8e72ffd640 1 -- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e60004d00 msgr2=0x7f8e60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:44.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.448+0000 7f8e72ffd640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e60004d00 0x7f8e60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.448+0000 7f8e72ffd640 1 -- 172.21.15.27:0/856066292 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e60003520 con 0x7f8e60003970 2023-08-11T15:53:44.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.448+0000 7f8e727fc640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e740b55c0 0x7f8e600047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:44.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.449+0000 7f8e72ffd640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e60003970 0x7f8e60003d90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400a870 tx=0x7f8e600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856220b9c51eb67b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.450+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64034630 con 0x7f8e60003970 2023-08-11T15:53:44.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.450+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6402f070 con 0x7f8e60003970 2023-08-11T15:53:44.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.450+0000 7f8e71ffb640 1 -- 172.21.15.27:0/856066292 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e640349f0 con 0x7f8e60003970 2023-08-11T15:53:44.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.450+0000 7f8e796ec640 1 -- 172.21.15.27:0/856066292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e60003970 msgr2=0x7f8e60003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.450+0000 7f8e796ec640 1 --2- 172.21.15.27:0/856066292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e60003970 0x7f8e60003d90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400a870 tx=0x7f8e600057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.450+0000 7f8e737fe640 1 -- 172.21.15.27:0/856066292 reap_dead start 2023-08-11T15:53:44.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.450+0000 7f8e796ec640 1 -- 172.21.15.27:0/856066292 shutdown_connections 2023-08-11T15:53:44.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.450+0000 7f8e796ec640 1 -- 172.21.15.27:0/856066292 >> 172.21.15.27:0/856066292 conn(0x7f8e74071110 msgr2=0x7f8e740714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:44.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.450+0000 7f8e796ec640 1 -- 172.21.15.27:0/856066292 shutdown_connections 2023-08-11T15:53:44.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.450+0000 7f8e796ec640 1 -- 172.21.15.27:0/856066292 wait complete. 2023-08-11T15:53:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.452+0000 7f8e796ec640 1 Processor -- start 2023-08-11T15:53:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.452+0000 7f8e796ec640 1 -- start start 2023-08-11T15:53:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.453+0000 7f8e796ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e60003970 0x7f8e74148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.453+0000 7f8e796ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e740cbdc0 0x7f8e7414a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.453+0000 7f8e796ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e740a9480 0x7f8e740a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.453+0000 7f8e796ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e74149f60 con 0x7f8e740cbdc0 2023-08-11T15:53:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.453+0000 7f8e796ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740c8c80 con 0x7f8e740a9480 2023-08-11T15:53:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.453+0000 7f8e796ec640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e74149040 con 0x7f8e60003970 2023-08-11T15:53:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.454+0000 7f8e727fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e740cbdc0 0x7f8e7414a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.454+0000 7f8e727fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e740cbdc0 0x7f8e7414a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51130/0 (socket says 172.21.15.27:51130) 2023-08-11T15:53:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.454+0000 7f8e727fc640 1 -- 172.21.15.27:0/176118510 learned_addr learned my addr 172.21.15.27:0/176118510 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.454+0000 7f8e72ffd640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e60003970 0x7f8e74148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.454+0000 7f8e737fe640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e740a9480 0x7f8e740a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.454+0000 7f8e727fc640 1 -- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e60003970 msgr2=0x7f8e74148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.454+0000 7f8e727fc640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e60003970 0x7f8e74148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.454+0000 7f8e727fc640 1 -- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e740a9480 msgr2=0x7f8e740a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.454+0000 7f8e727fc640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e740a9480 0x7f8e740a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.454+0000 7f8e727fc640 1 -- 172.21.15.27:0/176118510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e741492e0 con 0x7f8e740cbdc0 2023-08-11T15:53:44.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.455+0000 7f8e727fc640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e740cbdc0 0x7f8e7414a380 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0108e0 tx=0x7f8e5c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.455+0000 7f8e6b7fe640 1 -- 172.21.15.27:0/176118510 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c00ec20 con 0x7f8e740cbdc0 2023-08-11T15:53:44.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.455+0000 7f8e6b7fe640 1 -- 172.21.15.27:0/176118510 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5c00ed80 con 0x7f8e740cbdc0 2023-08-11T15:53:44.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.455+0000 7f8e6b7fe640 1 -- 172.21.15.27:0/176118510 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c01c600 con 0x7f8e740cbdc0 2023-08-11T15:53:44.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.455+0000 7f8e796ec640 1 -- 172.21.15.27:0/176118510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e741495c0 con 0x7f8e740cbdc0 2023-08-11T15:53:44.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.455+0000 7f8e796ec640 1 -- 172.21.15.27:0/176118510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e74149b80 con 0x7f8e740cbdc0 2023-08-11T15:53:44.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.456+0000 7f8e796ec640 1 -- 172.21.15.27:0/176118510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e74149d40 con 0x7f8e740cbdc0 2023-08-11T15:53:44.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.457+0000 7f8e6b7fe640 1 -- 172.21.15.27:0/176118510 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8e5c01c760 con 0x7f8e740cbdc0 2023-08-11T15:53:44.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.458+0000 7f8e6b7fe640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8e5406a050 0x7f8e5406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.458+0000 7f8e6b7fe640 1 -- 172.21.15.27:0/176118510 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f8e5c088b50 con 0x7f8e740cbdc0 2023-08-11T15:53:44.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.459+0000 7f8e72ffd640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8e5406a050 0x7f8e5406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.460+0000 7f8e72ffd640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8e5406a050 0x7f8e5406c510 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8e64034000 tx=0x7f8e64009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.463+0000 7f8e6b7fe640 1 -- 172.21.15.27:0/176118510 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f8e5c057eb0 con 0x7f8e740cbdc0 2023-08-11T15:53:44.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.657+0000 7f8e796ec640 1 -- 172.21.15.27:0/176118510 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e740cd100 con 0x7f8e5406a050 2023-08-11T15:53:44.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.659+0000 7f8e6b7fe640 1 -- 172.21.15.27:0/176118510 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8e64001930 con 0x7f8e5406a050 2023-08-11T15:53:44.661 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 0 to deep-scrub 2023-08-11T15:53:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.660+0000 7f8e796ec640 1 -- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8e5406a050 msgr2=0x7f8e5406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.660+0000 7f8e796ec640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8e5406a050 0x7f8e5406c510 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8e64034000 tx=0x7f8e64009030 comp rx=0 tx=0).stop 2023-08-11T15:53:44.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.660+0000 7f8e796ec640 1 -- 172.21.15.27:0/176118510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e740cbdc0 msgr2=0x7f8e7414a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.660+0000 7f8e796ec640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e740cbdc0 0x7f8e7414a380 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0108e0 tx=0x7f8e5c010da0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.661+0000 7f8e796ec640 1 -- 172.21.15.27:0/176118510 shutdown_connections 2023-08-11T15:53:44.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.661+0000 7f8e796ec640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f8e5406a050 0x7f8e5406c510 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.661+0000 7f8e796ec640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e740a9480 0x7f8e740a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.661+0000 7f8e796ec640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e740cbdc0 0x7f8e7414a380 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.661+0000 7f8e796ec640 1 --2- 172.21.15.27:0/176118510 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e60003970 0x7f8e74148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.661+0000 7f8e796ec640 1 -- 172.21.15.27:0/176118510 >> 172.21.15.27:0/176118510 conn(0x7f8e74071110 msgr2=0x7f8e7415bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:44.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.661+0000 7f8e796ec640 1 -- 172.21.15.27:0/176118510 shutdown_connections 2023-08-11T15:53:44.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.661+0000 7f8e796ec640 1 -- 172.21.15.27:0/176118510 wait complete. 2023-08-11T15:53:44.681 INFO:tasks.ceph:Scrubbing osd.1 2023-08-11T15:53:44.682 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-08-11T15:53:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.920+0000 7f70affff640 1 Processor -- start 2023-08-11T15:53:44.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.920+0000 7f70affff640 1 -- start start 2023-08-11T15:53:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.922+0000 7f70affff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f70b01614e0 0x7f70b006bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.922+0000 7f70affff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b006c5e0 con 0x7f70b006b7b0 2023-08-11T15:53:44.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.922+0000 7f70affff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b01619b0 con 0x7f70b015e4a0 2023-08-11T15:53:44.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.922+0000 7f70affff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b006c1a0 con 0x7f70b01614e0 2023-08-11T15:53:44.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.922+0000 7f70af7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f70b01614e0 0x7f70b006bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.922+0000 7f70a7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f70b006b7b0 0x7f70b0160dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49652/0 (socket says 172.21.15.27:49652) 2023-08-11T15:53:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.922+0000 7f70a7fff640 1 -- 172.21.15.27:0/574977511 learned_addr learned my addr 172.21.15.27:0/574977511 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.923+0000 7f70af7fe640 1 -- 172.21.15.27:0/574977511 >> v1:172.21.15.119:6789/0 conn(0x7f70b015e4a0 legacy=0x7f70b00cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.923+0000 7f70af7fe640 1 -- 172.21.15.27:0/574977511 >> v1:172.21.15.27:6789/0 conn(0x7f70b006b7b0 legacy=0x7f70b0160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.923+0000 7f70af7fe640 1 -- 172.21.15.27:0/574977511 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b0161c90 con 0x7f70b01614e0 2023-08-11T15:53:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.924+0000 7f70af7fe640 1 --2- 172.21.15.27:0/574977511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70b01614e0 0x7f70b006bc60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009d90 tx=0x7f70a802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8b1cc403eefda3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.924+0000 7f70ae7fc640 1 -- 172.21.15.27:0/574977511 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a802cc60 con 0x7f70b01614e0 2023-08-11T15:53:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.924+0000 7f70ae7fc640 1 -- 172.21.15.27:0/574977511 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a802cdc0 con 0x7f70b01614e0 2023-08-11T15:53:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.924+0000 7f70ae7fc640 1 -- 172.21.15.27:0/574977511 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8034660 con 0x7f70b01614e0 2023-08-11T15:53:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.925+0000 7f70affff640 1 -- 172.21.15.27:0/574977511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70b01614e0 msgr2=0x7f70b006bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.925+0000 7f70affff640 1 --2- 172.21.15.27:0/574977511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70b01614e0 0x7f70b006bc60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009d90 tx=0x7f70a802beb0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.925+0000 7f70affff640 1 -- 172.21.15.27:0/574977511 shutdown_connections 2023-08-11T15:53:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.925+0000 7f70affff640 1 --2- 172.21.15.27:0/574977511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70b01614e0 0x7f70b006bc60 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.925+0000 7f70affff640 1 -- 172.21.15.27:0/574977511 >> 172.21.15.27:0/574977511 conn(0x7f70b00afb70 msgr2=0x7f70b00aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.925+0000 7f70affff640 1 -- 172.21.15.27:0/574977511 shutdown_connections 2023-08-11T15:53:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.925+0000 7f70affff640 1 -- 172.21.15.27:0/574977511 wait complete. 2023-08-11T15:53:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.927+0000 7f70affff640 1 Processor -- start 2023-08-11T15:53:44.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.927+0000 7f70affff640 1 -- start start 2023-08-11T15:53:44.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70affff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70b006b7b0 0x7f70b00c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70affff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70b015e4a0 0x7f70b00b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70affff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b01614e0 0x7f70b0150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70affff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00c16b0 con 0x7f70b006b7b0 2023-08-11T15:53:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70affff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00c6710 con 0x7f70b01614e0 2023-08-11T15:53:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70affff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00b3c60 con 0x7f70b015e4a0 2023-08-11T15:53:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70aeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70b006b7b0 0x7f70b00c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70aeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70b006b7b0 0x7f70b00c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51140/0 (socket says 172.21.15.27:51140) 2023-08-11T15:53:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70aeffd640 1 -- 172.21.15.27:0/3647389187 learned_addr learned my addr 172.21.15.27:0/3647389187 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70aeffd640 1 -- 172.21.15.27:0/3647389187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70b015e4a0 msgr2=0x7f70b00b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70aeffd640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70b015e4a0 0x7f70b00b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70aeffd640 1 -- 172.21.15.27:0/3647389187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b01614e0 msgr2=0x7f70b0150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70aeffd640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b01614e0 0x7f70b0150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.929+0000 7f70aeffd640 1 -- 172.21.15.27:0/3647389187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a8009a70 con 0x7f70b006b7b0 2023-08-11T15:53:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.930+0000 7f70aeffd640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70b006b7b0 0x7f70b00c5a90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f70a0004640 tx=0x7f70a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.930+0000 7f70acff9640 1 -- 172.21.15.27:0/3647389187 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0011790 con 0x7f70b006b7b0 2023-08-11T15:53:44.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.930+0000 7f70acff9640 1 -- 172.21.15.27:0/3647389187 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a00118f0 con 0x7f70b006b7b0 2023-08-11T15:53:44.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.930+0000 7f70acff9640 1 -- 172.21.15.27:0/3647389187 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0011ce0 con 0x7f70b006b7b0 2023-08-11T15:53:44.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.930+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b0150a70 con 0x7f70b006b7b0 2023-08-11T15:53:44.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.930+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b00c6430 con 0x7f70b006b7b0 2023-08-11T15:53:44.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.931+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70b00caaa0 con 0x7f70b006b7b0 2023-08-11T15:53:44.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.932+0000 7f70acff9640 1 -- 172.21.15.27:0/3647389187 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f70a001c460 con 0x7f70b006b7b0 2023-08-11T15:53:44.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.933+0000 7f70acff9640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f709406a050 0x7f709406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.933+0000 7f70acff9640 1 -- 172.21.15.27:0/3647389187 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f70a0088330 con 0x7f70b006b7b0 2023-08-11T15:53:44.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.934+0000 7f70acff9640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] conn(0x7f7094070970 0x7f7094072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:44.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.934+0000 7f70acff9640 1 -- 172.21.15.27:0/3647389187 --> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7094073480 con 0x7f7094070970 2023-08-11T15:53:44.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.934+0000 7f70acff9640 1 -- 172.21.15.27:0/3647389187 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70a008a910 con 0x7f70b006b7b0 2023-08-11T15:53:44.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.934+0000 7f70a7fff640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f709406a050 0x7f709406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.934+0000 7f70af7fe640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] conn(0x7f7094070970 0x7f7094072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:44.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.935+0000 7f70af7fe640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] conn(0x7f7094070970 0x7f7094072db0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.935+0000 7f70a7fff640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f709406a050 0x7f709406c510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7098005880 tx=0x7f7098007a00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:44.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.935+0000 7f70acff9640 1 -- 172.21.15.27:0/3647389187 <== osd.1 v2:172.21.15.27:6808/889585576 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f7094073480 con 0x7f7094070970 2023-08-11T15:53:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.950+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 --> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f70b00ca7c0 con 0x7f7094070970 2023-08-11T15:53:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70acff9640 1 -- 172.21.15.27:0/3647389187 <== osd.1 v2:172.21.15.27:6808/889585576 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f70a802ca60 con 0x7f7094070970 2023-08-11T15:53:44.954 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:44.954 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-11T15:53:44.954 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 >> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] conn(0x7f7094070970 msgr2=0x7f7094072db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70affff640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.27:6808/889585576,v1:172.21.15.27:6809/889585576] conn(0x7f7094070970 0x7f7094072db0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f709406a050 msgr2=0x7f709406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70affff640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f709406a050 0x7f709406c510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7098005880 tx=0x7f7098007a00 comp rx=0 tx=0).stop 2023-08-11T15:53:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70b006b7b0 msgr2=0x7f70b00c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70affff640 1 --2- 172.21.15.27:0/3647389187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70b006b7b0 0x7f70b00c5a90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f70a0004640 tx=0x7f70a000c8e0 comp rx=0 tx=0).stop 2023-08-11T15:53:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70af7fe640 1 -- 172.21.15.27:0/3647389187 reap_dead start 2023-08-11T15:53:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 shutdown_connections 2023-08-11T15:53:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 >> 172.21.15.27:0/3647389187 conn(0x7f70b00afb70 msgr2=0x7f70b00cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.951+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 shutdown_connections 2023-08-11T15:53:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:44.952+0000 7f70affff640 1 -- 172.21.15.27:0/3647389187 wait complete. 2023-08-11T15:53:44.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-08-11T15:53:45.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.198+0000 7f29cce87640 1 Processor -- start 2023-08-11T15:53:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.198+0000 7f29cce87640 1 -- start start 2023-08-11T15:53:45.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.200+0000 7f29cce87640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29c80cb430 0x7f29c80b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.200+0000 7f29cce87640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 0x7f29c806fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.200+0000 7f29cce87640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c80c8a30 con 0x7f29c815a140 2023-08-11T15:53:45.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.200+0000 7f29cce87640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c806ffa0 con 0x7f29c80cb430 2023-08-11T15:53:45.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.200+0000 7f29cce87640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c806c0c0 con 0x7f29c806f5d0 2023-08-11T15:53:45.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.201+0000 7f29bdd9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29c80cb430 0x7f29c80b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.201+0000 7f29bdd9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29c80cb430 0x7f29c80b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51154/0 (socket says 172.21.15.27:51154) 2023-08-11T15:53:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.201+0000 7f29bdd9b640 1 -- 172.21.15.27:0/520668308 learned_addr learned my addr 172.21.15.27:0/520668308 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.201+0000 7f29c6d9d640 1 --2- 172.21.15.27:0/520668308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 0x7f29c806fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.201+0000 7f29bdd9b640 1 -- 172.21.15.27:0/520668308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 msgr2=0x7f29c806fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.201+0000 7f29bdd9b640 1 --2- 172.21.15.27:0/520668308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 0x7f29c806fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.201+0000 7f29bdd9b640 1 -- 172.21.15.27:0/520668308 >> v1:172.21.15.119:6789/0 conn(0x7f29c815a140 legacy=0x7f29c806b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.201+0000 7f29bdd9b640 1 -- 172.21.15.27:0/520668308 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c806c3a0 con 0x7f29c80cb430 2023-08-11T15:53:45.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.202+0000 7f29bdd9b640 1 --2- 172.21.15.27:0/520668308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 0x7f29c80b0e70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009870 tx=0x7f29b002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c50fe22a1b202 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:45.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.203+0000 7f29c5d9b640 1 -- 172.21.15.27:0/520668308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0030840 con 0x7f29c80cb430 2023-08-11T15:53:45.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.203+0000 7f29c5d9b640 1 -- 172.21.15.27:0/520668308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b00309a0 con 0x7f29c80cb430 2023-08-11T15:53:45.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.203+0000 7f29c5d9b640 1 -- 172.21.15.27:0/520668308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0030d30 con 0x7f29c80cb430 2023-08-11T15:53:45.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.204+0000 7f29cce87640 1 -- 172.21.15.27:0/520668308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 msgr2=0x7f29c80b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.204+0000 7f29cce87640 1 --2- 172.21.15.27:0/520668308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 0x7f29c80b0e70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009870 tx=0x7f29b002b980 comp rx=0 tx=0).stop 2023-08-11T15:53:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.204+0000 7f29cce87640 1 -- 172.21.15.27:0/520668308 shutdown_connections 2023-08-11T15:53:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.204+0000 7f29cce87640 1 --2- 172.21.15.27:0/520668308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 0x7f29c806fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.204+0000 7f29cce87640 1 --2- 172.21.15.27:0/520668308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 0x7f29c80b0e70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.204+0000 7f29cce87640 1 -- 172.21.15.27:0/520668308 >> 172.21.15.27:0/520668308 conn(0x7f29c80afb70 msgr2=0x7f29c80aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.204+0000 7f29cce87640 1 -- 172.21.15.27:0/520668308 shutdown_connections 2023-08-11T15:53:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.204+0000 7f29cce87640 1 -- 172.21.15.27:0/520668308 wait complete. 2023-08-11T15:53:45.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.206+0000 7f29cce87640 1 Processor -- start 2023-08-11T15:53:45.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.206+0000 7f29cce87640 1 -- start start 2023-08-11T15:53:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29cce87640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 0x7f29c8155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29cce87640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 0x7f29c815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29cce87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c815a140 0x7f29c815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29cce87640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c815c850 con 0x7f29c80cb430 2023-08-11T15:53:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29cce87640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c81564d0 con 0x7f29c815a140 2023-08-11T15:53:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29cce87640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80ae930 con 0x7f29c806f5d0 2023-08-11T15:53:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29bdd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 0x7f29c815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29bdd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 0x7f29c815bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51158/0 (socket says 172.21.15.27:51158) 2023-08-11T15:53:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29bdd9b640 1 -- 172.21.15.27:0/3914078651 learned_addr learned my addr 172.21.15.27:0/3914078651 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29c6d9d640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c815a140 0x7f29c815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.207+0000 7f29c659c640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 0x7f29c8155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29bdd9b640 1 -- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 msgr2=0x7f29c8155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29bdd9b640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 0x7f29c8155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29bdd9b640 1 -- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c815a140 msgr2=0x7f29c815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29bdd9b640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c815a140 0x7f29c815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29bdd9b640 1 -- 172.21.15.27:0/3914078651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b00094d0 con 0x7f29c80cb430 2023-08-11T15:53:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29c659c640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 0x7f29c8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29c6d9d640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c815a140 0x7f29c815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29bdd9b640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 0x7f29c815bc70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f29b002b9c0 tx=0x7f29b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29bf7fe640 1 -- 172.21.15.27:0/3914078651 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0039c90 con 0x7f29c80cb430 2023-08-11T15:53:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29bf7fe640 1 -- 172.21.15.27:0/3914078651 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0039df0 con 0x7f29c80cb430 2023-08-11T15:53:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29bf7fe640 1 -- 172.21.15.27:0/3914078651 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00446c0 con 0x7f29c80cb430 2023-08-11T15:53:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.208+0000 7f29cce87640 1 -- 172.21.15.27:0/3914078651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c8156720 con 0x7f29c80cb430 2023-08-11T15:53:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.209+0000 7f29cce87640 1 -- 172.21.15.27:0/3914078651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c8156ce0 con 0x7f29c80cb430 2023-08-11T15:53:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.210+0000 7f29cce87640 1 -- 172.21.15.27:0/3914078651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c8073990 con 0x7f29c80cb430 2023-08-11T15:53:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.211+0000 7f29bf7fe640 1 -- 172.21.15.27:0/3914078651 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f29b002f390 con 0x7f29c80cb430 2023-08-11T15:53:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.212+0000 7f29bf7fe640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f29a806a050 0x7f29a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.212+0000 7f29bf7fe640 1 -- 172.21.15.27:0/3914078651 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f29b003e070 con 0x7f29c80cb430 2023-08-11T15:53:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.213+0000 7f29c659c640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f29a806a050 0x7f29a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.214+0000 7f29c659c640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f29a806a050 0x7f29a806c510 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f29b8001610 tx=0x7f29b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.217+0000 7f29bf7fe640 1 -- 172.21.15.27:0/3914078651 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f29b0076760 con 0x7f29c80cb430 2023-08-11T15:53:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.411+0000 7f29cce87640 1 -- 172.21.15.27:0/3914078651 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f29c806c680 con 0x7f29a806a050 2023-08-11T15:53:45.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.412+0000 7f29bf7fe640 1 -- 172.21.15.27:0/3914078651 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f29b800d490 con 0x7f29a806a050 2023-08-11T15:53:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 1 to deep-scrub 2023-08-11T15:53:45.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.414+0000 7f29cce87640 1 -- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f29a806a050 msgr2=0x7f29a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.414+0000 7f29cce87640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f29a806a050 0x7f29a806c510 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f29b8001610 tx=0x7f29b8009040 comp rx=0 tx=0).stop 2023-08-11T15:53:45.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.414+0000 7f29cce87640 1 -- 172.21.15.27:0/3914078651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 msgr2=0x7f29c815bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.414+0000 7f29cce87640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 0x7f29c815bc70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f29b002b9c0 tx=0x7f29b0004730 comp rx=0 tx=0).stop 2023-08-11T15:53:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.414+0000 7f29cce87640 1 -- 172.21.15.27:0/3914078651 shutdown_connections 2023-08-11T15:53:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.414+0000 7f29cce87640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f29a806a050 0x7f29a806c510 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.414+0000 7f29cce87640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c815a140 0x7f29c815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.414+0000 7f29cce87640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29c80cb430 0x7f29c815bc70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.414+0000 7f29cce87640 1 --2- 172.21.15.27:0/3914078651 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29c806f5d0 0x7f29c8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.414+0000 7f29cce87640 1 -- 172.21.15.27:0/3914078651 >> 172.21.15.27:0/3914078651 conn(0x7f29c80afb70 msgr2=0x7f29c80aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.415+0000 7f29cce87640 1 -- 172.21.15.27:0/3914078651 shutdown_connections 2023-08-11T15:53:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.415+0000 7f29cce87640 1 -- 172.21.15.27:0/3914078651 wait complete. 2023-08-11T15:53:45.437 INFO:tasks.ceph:Scrubbing osd.2 2023-08-11T15:53:45.438 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-08-11T15:53:45.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.668+0000 7f654018b640 1 Processor -- start 2023-08-11T15:53:45.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.668+0000 7f654018b640 1 -- start start 2023-08-11T15:53:45.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.670+0000 7f654018b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f653815a140 0x7f653806b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.670+0000 7f654018b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f65380cb430 0x7f65380b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.670+0000 7f654018b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65380c8a30 con 0x7f653806f5d0 2023-08-11T15:53:45.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.670+0000 7f654018b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653806c0c0 con 0x7f653815a140 2023-08-11T15:53:45.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.670+0000 7f654018b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653806c3a0 con 0x7f65380cb430 2023-08-11T15:53:45.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.670+0000 7f653e728640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f653806f5d0 0x7f65380c8ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49656/0 (socket says 172.21.15.27:49656) 2023-08-11T15:53:45.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.670+0000 7f653e728640 1 -- 172.21.15.27:0/135565531 learned_addr learned my addr 172.21.15.27:0/135565531 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:45.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.670+0000 7f653df27640 1 --2- 172.21.15.27:0/135565531 >> v2:172.21.15.119:3300/0 conn(0x7f653815a140 0x7f653806b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.670+0000 7f653d726640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f65380cb430 0x7f65380b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187991621 0 0) 0x7f65380c8a30 con 0x7f653806f5d0 2023-08-11T15:53:45.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6524003520 con 0x7f653806f5d0 2023-08-11T15:53:45.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 409045398 0 0) 0x7f6524003520 con 0x7f653806f5d0 2023-08-11T15:53:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f65380cb430 msgr2=0x7f65380b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f65380cb430 0x7f65380b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 >> v2:172.21.15.119:3300/0 conn(0x7f653815a140 msgr2=0x7f653806b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 --2- 172.21.15.27:0/135565531 >> v2:172.21.15.119:3300/0 conn(0x7f653815a140 0x7f653806b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653806c680 con 0x7f653806f5d0 2023-08-11T15:53:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f65300019f0 con 0x7f653806f5d0 2023-08-11T15:53:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653df27640 1 --2- 172.21.15.27:0/135565531 >> v2:172.21.15.119:3300/0 conn(0x7f653815a140 0x7f653806b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 >> v1:172.21.15.27:6789/0 conn(0x7f653806f5d0 legacy=0x7f65380c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6524003970 0x7f6524003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65240047c0 0x7f6524004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f65380cb430 0x7f6524005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653806c0c0 con 0x7f6524003970 2023-08-11T15:53:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653806c3a0 con 0x7f65240047c0 2023-08-11T15:53:45.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.671+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652400b980 con 0x7f65380cb430 2023-08-11T15:53:45.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.672+0000 7f653df27640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6524003970 0x7f6524003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.672+0000 7f653e728640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f65380cb430 0x7f6524005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.672+0000 7f653df27640 1 -- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f65380cb430 msgr2=0x7f6524005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:45.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.672+0000 7f653df27640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f65380cb430 0x7f6524005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.672+0000 7f653df27640 1 -- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65240047c0 msgr2=0x7f6524004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:45.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.672+0000 7f653df27640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65240047c0 0x7f6524004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.672+0000 7f653df27640 1 -- 172.21.15.27:0/135565531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6524003520 con 0x7f6524003970 2023-08-11T15:53:45.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.672+0000 7f653e728640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f65380cb430 0x7f6524005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:45.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.673+0000 7f653df27640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6524003970 0x7f6524003d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6534009c80 tx=0x7f65240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca676e016f3add9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:45.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.673+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653402c980 con 0x7f6524003970 2023-08-11T15:53:45.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.674+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653402cae0 con 0x7f6524003970 2023-08-11T15:53:45.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.674+0000 7f653cf25640 1 -- 172.21.15.27:0/135565531 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653402ce90 con 0x7f6524003970 2023-08-11T15:53:45.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.674+0000 7f654018b640 1 -- 172.21.15.27:0/135565531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6524003970 msgr2=0x7f6524003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.674+0000 7f654018b640 1 --2- 172.21.15.27:0/135565531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6524003970 0x7f6524003d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6534009c80 tx=0x7f65240057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.674+0000 7f653e728640 1 -- 172.21.15.27:0/135565531 reap_dead start 2023-08-11T15:53:45.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.674+0000 7f654018b640 1 -- 172.21.15.27:0/135565531 shutdown_connections 2023-08-11T15:53:45.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.674+0000 7f654018b640 1 -- 172.21.15.27:0/135565531 >> 172.21.15.27:0/135565531 conn(0x7f65380afb70 msgr2=0x7f65380aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:45.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.674+0000 7f654018b640 1 -- 172.21.15.27:0/135565531 shutdown_connections 2023-08-11T15:53:45.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.674+0000 7f654018b640 1 -- 172.21.15.27:0/135565531 wait complete. 2023-08-11T15:53:45.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.676+0000 7f654018b640 1 Processor -- start 2023-08-11T15:53:45.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.676+0000 7f654018b640 1 -- start start 2023-08-11T15:53:45.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.677+0000 7f654018b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6524003970 0x7f65380c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.677+0000 7f654018b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6538154900 0x7f65381535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.677+0000 7f654018b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6538153ae0 0x7f6538156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.677+0000 7f654018b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65380c1830 con 0x7f6538154900 2023-08-11T15:53:45.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.677+0000 7f654018b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65380c1b10 con 0x7f6524003970 2023-08-11T15:53:45.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.677+0000 7f654018b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65380c44e0 con 0x7f6538153ae0 2023-08-11T15:53:45.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.677+0000 7f653d726640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6538154900 0x7f65381535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653d726640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6538154900 0x7f65381535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51164/0 (socket says 172.21.15.27:51164) 2023-08-11T15:53:45.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653d726640 1 -- 172.21.15.27:0/1995759350 learned_addr learned my addr 172.21.15.27:0/1995759350 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:45.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653df27640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6524003970 0x7f65380c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653d726640 1 -- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6538153ae0 msgr2=0x7f6538156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653e728640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6538153ae0 0x7f6538156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653d726640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6538153ae0 0x7f6538156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653d726640 1 -- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6524003970 msgr2=0x7f65380c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653d726640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6524003970 0x7f65380c3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653d726640 1 -- 172.21.15.27:0/1995759350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6538158260 con 0x7f6538154900 2023-08-11T15:53:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653e728640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6538153ae0 0x7f6538156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.678+0000 7f653d726640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6538154900 0x7f65381535a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f65280038f0 tx=0x7f6528003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.679+0000 7f652e7fc640 1 -- 172.21.15.27:0/1995759350 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528014040 con 0x7f6538154900 2023-08-11T15:53:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.679+0000 7f652e7fc640 1 -- 172.21.15.27:0/1995759350 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6528009c90 con 0x7f6538154900 2023-08-11T15:53:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.679+0000 7f652e7fc640 1 -- 172.21.15.27:0/1995759350 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528004110 con 0x7f6538154900 2023-08-11T15:53:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.679+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6538158540 con 0x7f6538154900 2023-08-11T15:53:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.679+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6538153f60 con 0x7f6538154900 2023-08-11T15:53:45.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.680+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f653806bdb0 con 0x7f6538154900 2023-08-11T15:53:45.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.681+0000 7f652e7fc640 1 -- 172.21.15.27:0/1995759350 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6528018420 con 0x7f6538154900 2023-08-11T15:53:45.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.682+0000 7f652e7fc640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f651c06a080 0x7f651c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.682+0000 7f652e7fc640 1 -- 172.21.15.27:0/1995759350 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f652808c0f0 con 0x7f6538154900 2023-08-11T15:53:45.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.682+0000 7f652e7fc640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] conn(0x7f651c0709a0 0x7f651c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.682+0000 7f652e7fc640 1 -- 172.21.15.27:0/1995759350 --> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f651c0734b0 con 0x7f651c0709a0 2023-08-11T15:53:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.682+0000 7f652e7fc640 1 -- 172.21.15.27:0/1995759350 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f652808e6d0 con 0x7f6538154900 2023-08-11T15:53:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.682+0000 7f653e728640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] conn(0x7f651c0709a0 0x7f651c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.683+0000 7f653df27640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f651c06a080 0x7f651c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.683+0000 7f653e728640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] conn(0x7f651c0709a0 0x7f651c072de0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.684+0000 7f653df27640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f651c06a080 0x7f651c06c540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6534036000 tx=0x7f6534030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.684+0000 7f652e7fc640 1 -- 172.21.15.27:0/1995759350 <== osd.2 v2:172.21.15.27:6821/2059564233 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f651c0734b0 con 0x7f651c0709a0 2023-08-11T15:53:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.699+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 --> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f65380c1970 con 0x7f651c0709a0 2023-08-11T15:53:45.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.699+0000 7f652e7fc640 1 -- 172.21.15.27:0/1995759350 <== osd.2 v2:172.21.15.27:6821/2059564233 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f653000d040 con 0x7f651c0709a0 2023-08-11T15:53:45.702 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:45.702 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-11T15:53:45.703 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:45.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 >> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] conn(0x7f651c0709a0 msgr2=0x7f651c072de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f654018b640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.27:6821/2059564233,v1:172.21.15.27:6823/2059564233] conn(0x7f651c0709a0 0x7f651c072de0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f651c06a080 msgr2=0x7f651c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f654018b640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f651c06a080 0x7f651c06c540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6534036000 tx=0x7f6534030000 comp rx=0 tx=0).stop 2023-08-11T15:53:45.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6538154900 msgr2=0x7f65381535a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f654018b640 1 --2- 172.21.15.27:0/1995759350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6538154900 0x7f65381535a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f65280038f0 tx=0x7f6528003cf0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f653e728640 1 -- 172.21.15.27:0/1995759350 reap_dead start 2023-08-11T15:53:45.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 shutdown_connections 2023-08-11T15:53:45.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 >> 172.21.15.27:0/1995759350 conn(0x7f65380afb70 msgr2=0x7f65380cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:45.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 shutdown_connections 2023-08-11T15:53:45.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.700+0000 7f654018b640 1 -- 172.21.15.27:0/1995759350 wait complete. 2023-08-11T15:53:45.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-08-11T15:53:45.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.948+0000 7f356e68e640 1 Processor -- start 2023-08-11T15:53:45.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.948+0000 7f356e68e640 1 -- start start 2023-08-11T15:53:45.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.950+0000 7f356e68e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3568156e50 0x7f356815a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.950+0000 7f356e68e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35680cbfb0 con 0x7f35680cb0c0 2023-08-11T15:53:45.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.950+0000 7f356e68e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35680cc270 con 0x7f35680cbb70 2023-08-11T15:53:45.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.950+0000 7f356e68e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680ca460 con 0x7f3568156e50 2023-08-11T15:53:45.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.950+0000 7f3567fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3568156e50 0x7f356815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.950+0000 7f356cc2b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f35680cb0c0 0x7f35680708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49658/0 (socket says 172.21.15.27:49658) 2023-08-11T15:53:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.950+0000 7f356cc2b640 1 -- 172.21.15.27:0/3407932454 learned_addr learned my addr 172.21.15.27:0/3407932454 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:45.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.951+0000 7f3567fff640 1 -- 172.21.15.27:0/3407932454 >> v1:172.21.15.119:6789/0 conn(0x7f35680cbb70 legacy=0x7f35680ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:45.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.951+0000 7f3567fff640 1 -- 172.21.15.27:0/3407932454 >> v1:172.21.15.27:6789/0 conn(0x7f35680cb0c0 legacy=0x7f35680708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.951+0000 7f3567fff640 1 -- 172.21.15.27:0/3407932454 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35680cc550 con 0x7f3568156e50 2023-08-11T15:53:45.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.951+0000 7f3566ffd640 1 -- 172.21.15.27:0/3407932454 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538093442 0 0) 0x7f35680cbfb0 con 0x7f35680cb0c0 2023-08-11T15:53:45.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.952+0000 7f3567fff640 1 --2- 172.21.15.27:0/3407932454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3568156e50 0x7f356815a180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3558009790 tx=0x7f355802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3215a0378b41460 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:45.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.952+0000 7f3566ffd640 1 -- 172.21.15.27:0/3407932454 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355802d040 con 0x7f3568156e50 2023-08-11T15:53:45.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.952+0000 7f3566ffd640 1 -- 172.21.15.27:0/3407932454 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3558031810 con 0x7f3568156e50 2023-08-11T15:53:45.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.952+0000 7f3566ffd640 1 -- 172.21.15.27:0/3407932454 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558031ba0 con 0x7f3568156e50 2023-08-11T15:53:45.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.953+0000 7f356e68e640 1 -- 172.21.15.27:0/3407932454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3568156e50 msgr2=0x7f356815a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.953+0000 7f356e68e640 1 --2- 172.21.15.27:0/3407932454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3568156e50 0x7f356815a180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3558009790 tx=0x7f355802b8b0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.953+0000 7f356e68e640 1 -- 172.21.15.27:0/3407932454 shutdown_connections 2023-08-11T15:53:45.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.953+0000 7f356e68e640 1 --2- 172.21.15.27:0/3407932454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3568156e50 0x7f356815a180 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.953+0000 7f356e68e640 1 -- 172.21.15.27:0/3407932454 >> 172.21.15.27:0/3407932454 conn(0x7f35680afb70 msgr2=0x7f35680aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:45.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.953+0000 7f356e68e640 1 -- 172.21.15.27:0/3407932454 shutdown_connections 2023-08-11T15:53:45.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.953+0000 7f356e68e640 1 -- 172.21.15.27:0/3407932454 wait complete. 2023-08-11T15:53:45.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.955+0000 7f356e68e640 1 Processor -- start 2023-08-11T15:53:45.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.955+0000 7f356e68e640 1 -- start start 2023-08-11T15:53:45.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.956+0000 7f356e68e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35680cb0c0 0x7f35681f9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.956+0000 7f356e68e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680cbb70 0x7f35680c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.956+0000 7f356e68e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3568156e50 0x7f35680aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.956+0000 7f356e68e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680ae970 con 0x7f35680cbb70 2023-08-11T15:53:45.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.956+0000 7f356e68e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680adae0 con 0x7f3568156e50 2023-08-11T15:53:45.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.956+0000 7f356e68e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35680adcf0 con 0x7f35680cb0c0 2023-08-11T15:53:45.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.956+0000 7f35677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680cbb70 0x7f35680c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.957+0000 7f35677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680cbb70 0x7f35680c4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51170/0 (socket says 172.21.15.27:51170) 2023-08-11T15:53:45.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.957+0000 7f35677fe640 1 -- 172.21.15.27:0/1935224143 learned_addr learned my addr 172.21.15.27:0/1935224143 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:45.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.957+0000 7f3567fff640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35680cb0c0 0x7f35681f9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.957+0000 7f356cc2b640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3568156e50 0x7f35680aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.957+0000 7f35677fe640 1 -- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35680cb0c0 msgr2=0x7f35681f9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.957+0000 7f35677fe640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35680cb0c0 0x7f35681f9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.957+0000 7f35677fe640 1 -- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3568156e50 msgr2=0x7f35680aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:45.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.957+0000 7f35677fe640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3568156e50 0x7f35680aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:45.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.957+0000 7f35677fe640 1 -- 172.21.15.27:0/1935224143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580093a0 con 0x7f35680cbb70 2023-08-11T15:53:45.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.957+0000 7f35677fe640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680cbb70 0x7f35680c4150 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f355c00c7c0 tx=0x7f355c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:45.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.958+0000 7f3564ff9640 1 -- 172.21.15.27:0/1935224143 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c00aa50 con 0x7f35680cbb70 2023-08-11T15:53:45.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.958+0000 7f3564ff9640 1 -- 172.21.15.27:0/1935224143 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c00abb0 con 0x7f35680cbb70 2023-08-11T15:53:45.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.958+0000 7f3564ff9640 1 -- 172.21.15.27:0/1935224143 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c012420 con 0x7f35680cbb70 2023-08-11T15:53:45.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.958+0000 7f356e68e640 1 -- 172.21.15.27:0/1935224143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35680adfd0 con 0x7f35680cbb70 2023-08-11T15:53:45.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.958+0000 7f356e68e640 1 -- 172.21.15.27:0/1935224143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35680ae590 con 0x7f35680cbb70 2023-08-11T15:53:45.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.959+0000 7f356e68e640 1 -- 172.21.15.27:0/1935224143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35680aeab0 con 0x7f35680cbb70 2023-08-11T15:53:45.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.960+0000 7f3564ff9640 1 -- 172.21.15.27:0/1935224143 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f355c012580 con 0x7f35680cbb70 2023-08-11T15:53:45.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.961+0000 7f3564ff9640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355406a080 0x7f355406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:45.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.961+0000 7f3564ff9640 1 -- 172.21.15.27:0/1935224143 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f355c088680 con 0x7f35680cbb70 2023-08-11T15:53:45.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.962+0000 7f3567fff640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355406a080 0x7f355406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:45.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.963+0000 7f3567fff640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355406a080 0x7f355406c540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3558036000 tx=0x7f355803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:45.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:45.965+0000 7f3564ff9640 1 -- 172.21.15.27:0/1935224143 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f35680aeab0 con 0x7f35680cbb70 2023-08-11T15:53:46.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.160+0000 7f356e68e640 1 -- 172.21.15.27:0/1935224143 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f35680cc830 con 0x7f355406a080 2023-08-11T15:53:46.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.161+0000 7f3564ff9640 1 -- 172.21.15.27:0/1935224143 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3558031be0 con 0x7f355406a080 2023-08-11T15:53:46.164 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 2 to deep-scrub 2023-08-11T15:53:46.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 -- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355406a080 msgr2=0x7f355406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355406a080 0x7f355406c540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3558036000 tx=0x7f355803c040 comp rx=0 tx=0).stop 2023-08-11T15:53:46.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 -- 172.21.15.27:0/1935224143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680cbb70 msgr2=0x7f35680c4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680cbb70 0x7f35680c4150 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f355c00c7c0 tx=0x7f355c00cc80 comp rx=0 tx=0).stop 2023-08-11T15:53:46.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 -- 172.21.15.27:0/1935224143 shutdown_connections 2023-08-11T15:53:46.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f355406a080 0x7f355406c540 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3568156e50 0x7f35680aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680cbb70 0x7f35680c4150 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 --2- 172.21.15.27:0/1935224143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35680cb0c0 0x7f35681f9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 -- 172.21.15.27:0/1935224143 >> 172.21.15.27:0/1935224143 conn(0x7f35680afb70 msgr2=0x7f35680aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:46.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 -- 172.21.15.27:0/1935224143 shutdown_connections 2023-08-11T15:53:46.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.164+0000 7f356e68e640 1 -- 172.21.15.27:0/1935224143 wait complete. 2023-08-11T15:53:46.185 INFO:tasks.ceph:Scrubbing osd.3 2023-08-11T15:53:46.185 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-08-11T15:53:46.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.432+0000 7f406ea08640 1 Processor -- start 2023-08-11T15:53:46.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.432+0000 7f406ea08640 1 -- start start 2023-08-11T15:53:46.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.433+0000 7f406ea08640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f406806f5f0 0x7f406815b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.433+0000 7f406ea08640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40680c8650 0x7f40680c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.433+0000 7f406ea08640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40680718b0 con 0x7f406815a1a0 2023-08-11T15:53:46.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.433+0000 7f406ea08640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406806fd80 con 0x7f406806f5f0 2023-08-11T15:53:46.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.433+0000 7f406ea08640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4068070c50 con 0x7f40680c8650 2023-08-11T15:53:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.434+0000 7f406da06640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f406815a1a0 0x7f406815e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49668/0 (socket says 172.21.15.27:49668) 2023-08-11T15:53:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.434+0000 7f406da06640 1 -- 172.21.15.27:0/668742631 learned_addr learned my addr 172.21.15.27:0/668742631 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:46.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.434+0000 7f406e207640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40680c8650 0x7f40680c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.434+0000 7f406d205640 1 --2- 172.21.15.27:0/668742631 >> v2:172.21.15.119:3300/0 conn(0x7f406806f5f0 0x7f406815b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.434+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114067671 0 0) 0x7f40680718b0 con 0x7f406815a1a0 2023-08-11T15:53:46.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.434+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4054003520 con 0x7f406815a1a0 2023-08-11T15:53:46.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1908364464 0 0) 0x7f4054003520 con 0x7f406815a1a0 2023-08-11T15:53:46.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40680c8650 msgr2=0x7f40680c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:46.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40680c8650 0x7f40680c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 >> v2:172.21.15.119:3300/0 conn(0x7f406806f5f0 msgr2=0x7f406815b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:46.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 --2- 172.21.15.27:0/668742631 >> v2:172.21.15.119:3300/0 conn(0x7f406806f5f0 0x7f406815b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406815a8b0 con 0x7f406815a1a0 2023-08-11T15:53:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f40640017f0 con 0x7f406815a1a0 2023-08-11T15:53:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 >> v1:172.21.15.27:6789/0 conn(0x7f406815a1a0 legacy=0x7f406815e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4054003970 0x7f4054003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40680c8650 0x7f40540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4054004d00 0x7f4054005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405400b980 con 0x7f4054004d00 2023-08-11T15:53:46.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405400bac0 con 0x7f4054003970 2023-08-11T15:53:46.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405400bc00 con 0x7f40680c8650 2023-08-11T15:53:46.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406e207640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4054004d00 0x7f4054005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406da06640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4054003970 0x7f4054003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406e207640 1 -- 172.21.15.27:0/668742631 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40680c8650 msgr2=0x7f40540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:46.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.435+0000 7f406e207640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40680c8650 0x7f40540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.436+0000 7f406e207640 1 -- 172.21.15.27:0/668742631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4054003970 msgr2=0x7f4054003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:46.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.436+0000 7f406e207640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4054003970 0x7f4054003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.436+0000 7f406e207640 1 -- 172.21.15.27:0/668742631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4054003520 con 0x7f4054004d00 2023-08-11T15:53:46.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.437+0000 7f406e207640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4054004d00 0x7f4054005180 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4060009fc0 tx=0x7f40540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ef60f7d7d2ffe4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:46.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.437+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406002cc30 con 0x7f4054004d00 2023-08-11T15:53:46.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.437+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406000f040 con 0x7f4054004d00 2023-08-11T15:53:46.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.437+0000 7f406ca04640 1 -- 172.21.15.27:0/668742631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060034440 con 0x7f4054004d00 2023-08-11T15:53:46.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.438+0000 7f406ea08640 1 -- 172.21.15.27:0/668742631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4054004d00 msgr2=0x7f4054005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.438+0000 7f406ea08640 1 --2- 172.21.15.27:0/668742631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4054004d00 0x7f4054005180 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4060009fc0 tx=0x7f40540057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.438+0000 7f406e207640 1 -- 172.21.15.27:0/668742631 reap_dead start 2023-08-11T15:53:46.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.438+0000 7f406ea08640 1 -- 172.21.15.27:0/668742631 shutdown_connections 2023-08-11T15:53:46.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.438+0000 7f406ea08640 1 -- 172.21.15.27:0/668742631 >> 172.21.15.27:0/668742631 conn(0x7f40680afb70 msgr2=0x7f40680aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:46.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.438+0000 7f406ea08640 1 -- 172.21.15.27:0/668742631 shutdown_connections 2023-08-11T15:53:46.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.438+0000 7f406ea08640 1 -- 172.21.15.27:0/668742631 wait complete. 2023-08-11T15:53:46.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.440+0000 7f406ea08640 1 Processor -- start 2023-08-11T15:53:46.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.440+0000 7f406ea08640 1 -- start start 2023-08-11T15:53:46.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.441+0000 7f406ea08640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4054004d00 0x7f40681536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.441+0000 7f406ea08640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f406814b040 0x7f406814aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.441+0000 7f406ea08640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40680c99f0 0x7f40680c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.441+0000 7f406ea08640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4068154990 con 0x7f40680c99f0 2023-08-11T15:53:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.441+0000 7f406ea08640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40680ca3b0 con 0x7f4054004d00 2023-08-11T15:53:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.441+0000 7f406ea08640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40680c50d0 con 0x7f406814b040 2023-08-11T15:53:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.441+0000 7f406e207640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40680c99f0 0x7f40680c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.441+0000 7f406e207640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40680c99f0 0x7f40680c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51194/0 (socket says 172.21.15.27:51194) 2023-08-11T15:53:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.441+0000 7f406e207640 1 -- 172.21.15.27:0/1286072888 learned_addr learned my addr 172.21.15.27:0/1286072888 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406da06640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4054004d00 0x7f40681536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406d205640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f406814b040 0x7f406814aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406e207640 1 -- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f406814b040 msgr2=0x7f406814aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406e207640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f406814b040 0x7f406814aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406e207640 1 -- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4054004d00 msgr2=0x7f40681536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406e207640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4054004d00 0x7f40681536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406e207640 1 -- 172.21.15.27:0/1286072888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068153da0 con 0x7f40680c99f0 2023-08-11T15:53:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406da06640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4054004d00 0x7f40681536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406d205640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f406814b040 0x7f406814aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406e207640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40680c99f0 0x7f40680c9e70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4060037040 tx=0x7f4060009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:46.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f405e7fc640 1 -- 172.21.15.27:0/1286072888 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060034ad0 con 0x7f40680c99f0 2023-08-11T15:53:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f405e7fc640 1 -- 172.21.15.27:0/1286072888 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4060034c30 con 0x7f40680c99f0 2023-08-11T15:53:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f405e7fc640 1 -- 172.21.15.27:0/1286072888 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060003990 con 0x7f40680c99f0 2023-08-11T15:53:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.442+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4068153ff0 con 0x7f40680c99f0 2023-08-11T15:53:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.443+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40681545b0 con 0x7f40680c99f0 2023-08-11T15:53:46.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.444+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40680715d0 con 0x7f40680c99f0 2023-08-11T15:53:46.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.445+0000 7f405e7fc640 1 -- 172.21.15.27:0/1286072888 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f406003e440 con 0x7f40680c99f0 2023-08-11T15:53:46.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.446+0000 7f405e7fc640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f404c06a080 0x7f404c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.446+0000 7f405e7fc640 1 -- 172.21.15.27:0/1286072888 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f40600aac90 con 0x7f40680c99f0 2023-08-11T15:53:46.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.446+0000 7f405e7fc640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] conn(0x7f404c0709a0 0x7f404c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.446+0000 7f405e7fc640 1 -- 172.21.15.27:0/1286072888 --> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f404c0734b0 con 0x7f404c0709a0 2023-08-11T15:53:46.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.446+0000 7f405e7fc640 1 -- 172.21.15.27:0/1286072888 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40600ab000 con 0x7f40680c99f0 2023-08-11T15:53:46.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.447+0000 7f406da06640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f404c06a080 0x7f404c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.447+0000 7f406d205640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] conn(0x7f404c0709a0 0x7f404c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.447+0000 7f406d205640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] conn(0x7f404c0709a0 0x7f404c072de0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:46.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.448+0000 7f406da06640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f404c06a080 0x7f404c06c540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4064001980 tx=0x7f4064009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:46.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.448+0000 7f405e7fc640 1 -- 172.21.15.27:0/1286072888 <== osd.3 v2:172.21.15.27:6816/4099583167 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f404c0734b0 con 0x7f404c0709a0 2023-08-11T15:53:46.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.463+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 --> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f40680712f0 con 0x7f404c0709a0 2023-08-11T15:53:46.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.463+0000 7f405e7fc640 1 -- 172.21.15.27:0/1286072888 <== osd.3 v2:172.21.15.27:6816/4099583167 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4058009b30 con 0x7f404c0709a0 2023-08-11T15:53:46.466 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-11T15:53:46.466 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.463+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 >> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] conn(0x7f404c0709a0 msgr2=0x7f404c072de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.463+0000 7f406ea08640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.27:6816/4099583167,v1:172.21.15.27:6817/4099583167] conn(0x7f404c0709a0 0x7f404c072de0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.463+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f404c06a080 msgr2=0x7f404c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.463+0000 7f406ea08640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f404c06a080 0x7f404c06c540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4064001980 tx=0x7f4064009040 comp rx=0 tx=0).stop 2023-08-11T15:53:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.463+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40680c99f0 msgr2=0x7f40680c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.463+0000 7f406ea08640 1 --2- 172.21.15.27:0/1286072888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40680c99f0 0x7f40680c9e70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4060037040 tx=0x7f4060009750 comp rx=0 tx=0).stop 2023-08-11T15:53:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.464+0000 7f406e207640 1 -- 172.21.15.27:0/1286072888 reap_dead start 2023-08-11T15:53:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.464+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 shutdown_connections 2023-08-11T15:53:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.464+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 >> 172.21.15.27:0/1286072888 conn(0x7f40680afb70 msgr2=0x7f40680c48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.464+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 shutdown_connections 2023-08-11T15:53:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.464+0000 7f406ea08640 1 -- 172.21.15.27:0/1286072888 wait complete. 2023-08-11T15:53:46.485 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-08-11T15:53:46.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.721+0000 7f3e41d12640 1 Processor -- start 2023-08-11T15:53:46.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.721+0000 7f3e41d12640 1 -- start start 2023-08-11T15:53:46.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.722+0000 7f3e41d12640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3e3c0b55c0 0x7f3e3c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.722+0000 7f3e41d12640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3e3c0cf930 0x7f3e3c0d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.722+0000 7f3e41d12640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e3c0cfd80 con 0x7f3e3c15c620 2023-08-11T15:53:46.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.722+0000 7f3e41d12640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3c0ccc10 con 0x7f3e3c0cf930 2023-08-11T15:53:46.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.722+0000 7f3e41d12640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3c0cda30 con 0x7f3e3c0b55c0 2023-08-11T15:53:46.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.723+0000 7f3e3bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3e3c0cf930 0x7f3e3c0d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.723+0000 7f3e3bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3e3c0cf930 0x7f3e3c0d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51206/0 (socket says 172.21.15.27:51206) 2023-08-11T15:53:46.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.723+0000 7f3e3bfff640 1 -- 172.21.15.27:0/1554242496 learned_addr learned my addr 172.21.15.27:0/1554242496 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:46.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.723+0000 7f3e32ffd640 1 --2- 172.21.15.27:0/1554242496 >> v2:172.21.15.119:3300/0 conn(0x7f3e3c0b55c0 0x7f3e3c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.723+0000 7f3e3bfff640 1 -- 172.21.15.27:0/1554242496 >> v2:172.21.15.119:3300/0 conn(0x7f3e3c0b55c0 msgr2=0x7f3e3c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.723+0000 7f3e3bfff640 1 --2- 172.21.15.27:0/1554242496 >> v2:172.21.15.119:3300/0 conn(0x7f3e3c0b55c0 0x7f3e3c0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.723+0000 7f3e3bfff640 1 -- 172.21.15.27:0/1554242496 >> v1:172.21.15.119:6789/0 conn(0x7f3e3c15c620 legacy=0x7f3e3c0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.723+0000 7f3e3bfff640 1 -- 172.21.15.27:0/1554242496 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c161110 con 0x7f3e3c0cf930 2023-08-11T15:53:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.725+0000 7f3e3bfff640 1 --2- 172.21.15.27:0/1554242496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 0x7f3e3c0d0b00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0b50e0 tx=0x7f3e3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336e7f6de926d7da server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:46.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.725+0000 7f3e3affd640 1 -- 172.21.15.27:0/1554242496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34031870 con 0x7f3e3c0cf930 2023-08-11T15:53:46.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.725+0000 7f3e3affd640 1 -- 172.21.15.27:0/1554242496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e340319d0 con 0x7f3e3c0cf930 2023-08-11T15:53:46.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.725+0000 7f3e3affd640 1 -- 172.21.15.27:0/1554242496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34031d60 con 0x7f3e3c0cf930 2023-08-11T15:53:46.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.726+0000 7f3e41d12640 1 -- 172.21.15.27:0/1554242496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 msgr2=0x7f3e3c0d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.726+0000 7f3e41d12640 1 --2- 172.21.15.27:0/1554242496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 0x7f3e3c0d0b00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0b50e0 tx=0x7f3e3402b910 comp rx=0 tx=0).stop 2023-08-11T15:53:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.726+0000 7f3e41d12640 1 -- 172.21.15.27:0/1554242496 shutdown_connections 2023-08-11T15:53:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.726+0000 7f3e41d12640 1 --2- 172.21.15.27:0/1554242496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 0x7f3e3c0d0b00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.726+0000 7f3e41d12640 1 --2- 172.21.15.27:0/1554242496 >> v2:172.21.15.119:3300/0 conn(0x7f3e3c0b55c0 0x7f3e3c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.726+0000 7f3e41d12640 1 -- 172.21.15.27:0/1554242496 >> 172.21.15.27:0/1554242496 conn(0x7f3e3c071110 msgr2=0x7f3e3c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.726+0000 7f3e41d12640 1 -- 172.21.15.27:0/1554242496 shutdown_connections 2023-08-11T15:53:46.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.726+0000 7f3e41d12640 1 -- 172.21.15.27:0/1554242496 wait complete. 2023-08-11T15:53:46.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.728+0000 7f3e41d12640 1 Processor -- start 2023-08-11T15:53:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.728+0000 7f3e41d12640 1 -- start start 2023-08-11T15:53:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.729+0000 7f3e41d12640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e3c0b55c0 0x7f3e3c156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.729+0000 7f3e41d12640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 0x7f3e3c0b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.729+0000 7f3e41d12640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e3c15c620 0x7f3e3c0a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.729+0000 7f3e41d12640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c1f1c10 con 0x7f3e3c0cf930 2023-08-11T15:53:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.729+0000 7f3e41d12640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c0c9a00 con 0x7f3e3c15c620 2023-08-11T15:53:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.729+0000 7f3e41d12640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c0c9ce0 con 0x7f3e3c0b55c0 2023-08-11T15:53:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.729+0000 7f3e32ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 0x7f3e3c0b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.729+0000 7f3e32ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 0x7f3e3c0b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51222/0 (socket says 172.21.15.27:51222) 2023-08-11T15:53:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.729+0000 7f3e32ffd640 1 -- 172.21.15.27:0/3109732424 learned_addr learned my addr 172.21.15.27:0/3109732424 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:46.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.729+0000 7f3e3b7fe640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e3c0b55c0 0x7f3e3c156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e3bfff640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e3c15c620 0x7f3e3c0a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e32ffd640 1 -- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e3c0b55c0 msgr2=0x7f3e3c156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e32ffd640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e3c0b55c0 0x7f3e3c156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e32ffd640 1 -- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e3c15c620 msgr2=0x7f3e3c0a8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e32ffd640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e3c15c620 0x7f3e3c0a8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e32ffd640 1 -- 172.21.15.27:0/3109732424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e340094d0 con 0x7f3e3c0cf930 2023-08-11T15:53:46.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e32ffd640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 0x7f3e3c0b4a40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400d5f0 tx=0x7f3e2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:46.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e38ff9640 1 -- 172.21.15.27:0/3109732424 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e240125f0 con 0x7f3e3c0cf930 2023-08-11T15:53:46.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e38ff9640 1 -- 172.21.15.27:0/3109732424 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e24012750 con 0x7f3e3c0cf930 2023-08-11T15:53:46.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e38ff9640 1 -- 172.21.15.27:0/3109732424 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24012b00 con 0x7f3e3c0cf930 2023-08-11T15:53:46.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.730+0000 7f3e41d12640 1 -- 172.21.15.27:0/3109732424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c0c9fc0 con 0x7f3e3c0cf930 2023-08-11T15:53:46.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.731+0000 7f3e41d12640 1 -- 172.21.15.27:0/3109732424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c0ca580 con 0x7f3e3c0cf930 2023-08-11T15:53:46.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.732+0000 7f3e41d12640 1 -- 172.21.15.27:0/3109732424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3c0ca710 con 0x7f3e3c0cf930 2023-08-11T15:53:46.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.733+0000 7f3e38ff9640 1 -- 172.21.15.27:0/3109732424 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3e24012c60 con 0x7f3e3c0cf930 2023-08-11T15:53:46.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.734+0000 7f3e38ff9640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e1c06a050 0x7f3e1c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:46.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.734+0000 7f3e38ff9640 1 -- 172.21.15.27:0/3109732424 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f3e24088910 con 0x7f3e3c0cf930 2023-08-11T15:53:46.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.734+0000 7f3e3b7fe640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e1c06a050 0x7f3e1c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:46.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.735+0000 7f3e3b7fe640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e1c06a050 0x7f3e1c06c510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005520 tx=0x7f3e2c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:46.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.738+0000 7f3e38ff9640 1 -- 172.21.15.27:0/3109732424 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f3e24054140 con 0x7f3e3c0cf930 2023-08-11T15:53:46.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.938+0000 7f3e41d12640 1 -- 172.21.15.27:0/3109732424 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e3c0c9e50 con 0x7f3e1c06a050 2023-08-11T15:53:46.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.940+0000 7f3e38ff9640 1 -- 172.21.15.27:0/3109732424 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3e3c0c9e50 con 0x7f3e1c06a050 2023-08-11T15:53:46.942 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 3 to deep-scrub 2023-08-11T15:53:46.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.942+0000 7f3e41d12640 1 -- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e1c06a050 msgr2=0x7f3e1c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.942+0000 7f3e41d12640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e1c06a050 0x7f3e1c06c510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005520 tx=0x7f3e2c009040 comp rx=0 tx=0).stop 2023-08-11T15:53:46.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.942+0000 7f3e41d12640 1 -- 172.21.15.27:0/3109732424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 msgr2=0x7f3e3c0b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:46.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.942+0000 7f3e41d12640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 0x7f3e3c0b4a40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400d5f0 tx=0x7f3e2400dab0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.942+0000 7f3e41d12640 1 -- 172.21.15.27:0/3109732424 shutdown_connections 2023-08-11T15:53:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.942+0000 7f3e41d12640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f3e1c06a050 0x7f3e1c06c510 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.942+0000 7f3e41d12640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e3c15c620 0x7f3e3c0a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.942+0000 7f3e41d12640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e3c0cf930 0x7f3e3c0b4a40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.942+0000 7f3e41d12640 1 --2- 172.21.15.27:0/3109732424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e3c0b55c0 0x7f3e3c156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.942+0000 7f3e41d12640 1 -- 172.21.15.27:0/3109732424 >> 172.21.15.27:0/3109732424 conn(0x7f3e3c071110 msgr2=0x7f3e3c15bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:46.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.943+0000 7f3e41d12640 1 -- 172.21.15.27:0/3109732424 shutdown_connections 2023-08-11T15:53:46.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:46.943+0000 7f3e41d12640 1 -- 172.21.15.27:0/3109732424 wait complete. 2023-08-11T15:53:46.963 INFO:tasks.ceph:Scrubbing osd.4 2023-08-11T15:53:46.964 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-08-11T15:53:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.206+0000 7f794f087640 1 Processor -- start 2023-08-11T15:53:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.206+0000 7f794f087640 1 -- start start 2023-08-11T15:53:47.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.208+0000 7f794f087640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f794815c620 0x7f79480b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.208+0000 7f794f087640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 0x7f79480cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.208+0000 7f794f087640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79480cda30 con 0x7f79480b55c0 2023-08-11T15:53:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.208+0000 7f794f087640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79480ccc10 con 0x7f794815c620 2023-08-11T15:53:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.208+0000 7f794f087640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7948161110 con 0x7f79480b82f0 2023-08-11T15:53:47.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.209+0000 7f7947fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 0x7f79480cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.209+0000 7f7947fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 0x7f79480cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:41862/0 (socket says 172.21.15.27:41862) 2023-08-11T15:53:47.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.209+0000 7f7947fff640 1 -- 172.21.15.27:0/1184160918 learned_addr learned my addr 172.21.15.27:0/1184160918 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:47.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.209+0000 7f794ce23640 1 --2- 172.21.15.27:0/1184160918 >> v2:172.21.15.119:3300/0 conn(0x7f794815c620 0x7f79480b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.209+0000 7f7947fff640 1 -- 172.21.15.27:0/1184160918 >> v2:172.21.15.119:3300/0 conn(0x7f794815c620 msgr2=0x7f79480b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:47.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.209+0000 7f7947fff640 1 --2- 172.21.15.27:0/1184160918 >> v2:172.21.15.119:3300/0 conn(0x7f794815c620 0x7f79480b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.209+0000 7f7947fff640 1 -- 172.21.15.27:0/1184160918 >> v1:172.21.15.119:6789/0 conn(0x7f79480b55c0 legacy=0x7f79480b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:47.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.209+0000 7f7947fff640 1 -- 172.21.15.27:0/1184160918 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79480cce20 con 0x7f79480b82f0 2023-08-11T15:53:47.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.211+0000 7f7947fff640 1 --2- 172.21.15.27:0/1184160918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 0x7f79480cf3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7934009870 tx=0x7f793402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f9f150b4be49ca server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:47.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.211+0000 7f79477fe640 1 -- 172.21.15.27:0/1184160918 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934031640 con 0x7f79480b82f0 2023-08-11T15:53:47.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.211+0000 7f79477fe640 1 -- 172.21.15.27:0/1184160918 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79340317a0 con 0x7f79480b82f0 2023-08-11T15:53:47.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.211+0000 7f79477fe640 1 -- 172.21.15.27:0/1184160918 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934031b30 con 0x7f79480b82f0 2023-08-11T15:53:47.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.212+0000 7f794f087640 1 -- 172.21.15.27:0/1184160918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 msgr2=0x7f79480cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.212+0000 7f794f087640 1 --2- 172.21.15.27:0/1184160918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 0x7f79480cf3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7934009870 tx=0x7f793402b8f0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.212+0000 7f794f087640 1 -- 172.21.15.27:0/1184160918 shutdown_connections 2023-08-11T15:53:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.212+0000 7f794f087640 1 --2- 172.21.15.27:0/1184160918 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 0x7f79480cf3f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.212+0000 7f794f087640 1 --2- 172.21.15.27:0/1184160918 >> v2:172.21.15.119:3300/0 conn(0x7f794815c620 0x7f79480b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.212+0000 7f794f087640 1 -- 172.21.15.27:0/1184160918 >> 172.21.15.27:0/1184160918 conn(0x7f7948071110 msgr2=0x7f79480714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.212+0000 7f794f087640 1 -- 172.21.15.27:0/1184160918 shutdown_connections 2023-08-11T15:53:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.212+0000 7f794f087640 1 -- 172.21.15.27:0/1184160918 wait complete. 2023-08-11T15:53:47.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.215+0000 7f794f087640 1 Processor -- start 2023-08-11T15:53:47.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.215+0000 7f794f087640 1 -- start start 2023-08-11T15:53:47.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.216+0000 7f794f087640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79480b55c0 0x7f79481596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.216+0000 7f794f087640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 0x7f794815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.216+0000 7f794f087640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f794815c620 0x7f794815f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.216+0000 7f794f087640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7948159c30 con 0x7f794815c620 2023-08-11T15:53:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.216+0000 7f794f087640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79480ca430 con 0x7f79480b55c0 2023-08-11T15:53:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.216+0000 7f794f087640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79480ca710 con 0x7f79480b82f0 2023-08-11T15:53:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.217+0000 7f794d624640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f794815c620 0x7f794815f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.217+0000 7f794d624640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f794815c620 0x7f794815f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51232/0 (socket says 172.21.15.27:51232) 2023-08-11T15:53:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.217+0000 7f794d624640 1 -- 172.21.15.27:0/1475147467 learned_addr learned my addr 172.21.15.27:0/1475147467 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.217+0000 7f7947fff640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 0x7f794815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.217+0000 7f794ce23640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79480b55c0 0x7f79481596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.217+0000 7f794d624640 1 -- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 msgr2=0x7f794815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.217+0000 7f794d624640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79480b82f0 0x7f794815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.217+0000 7f794d624640 1 -- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79480b55c0 msgr2=0x7f79481596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.217+0000 7f794d624640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79480b55c0 0x7f79481596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.217+0000 7f794d624640 1 -- 172.21.15.27:0/1475147467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340094d0 con 0x7f794815c620 2023-08-11T15:53:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.218+0000 7f794d624640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f794815c620 0x7f794815f110 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f794000be60 tx=0x7f794000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.218+0000 7f79457fa640 1 -- 172.21.15.27:0/1475147467 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940014440 con 0x7f794815c620 2023-08-11T15:53:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.218+0000 7f79457fa640 1 -- 172.21.15.27:0/1475147467 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79400145a0 con 0x7f794815c620 2023-08-11T15:53:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.218+0000 7f79457fa640 1 -- 172.21.15.27:0/1475147467 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940014990 con 0x7f794815c620 2023-08-11T15:53:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.218+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79480ca9f0 con 0x7f794815c620 2023-08-11T15:53:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.218+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79480cafb0 con 0x7f794815c620 2023-08-11T15:53:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.219+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79480cd3e0 con 0x7f794815c620 2023-08-11T15:53:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.220+0000 7f79457fa640 1 -- 172.21.15.27:0/1475147467 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f794001d3b0 con 0x7f794815c620 2023-08-11T15:53:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.221+0000 7f79457fa640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792c06a080 0x7f792c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.221+0000 7f79457fa640 1 -- 172.21.15.27:0/1475147467 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f7940088570 con 0x7f794815c620 2023-08-11T15:53:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.221+0000 7f79457fa640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] conn(0x7f792c0709a0 0x7f792c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.221+0000 7f79457fa640 1 -- 172.21.15.27:0/1475147467 --> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f792c0734b0 con 0x7f792c0709a0 2023-08-11T15:53:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.221+0000 7f79457fa640 1 -- 172.21.15.27:0/1475147467 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f794008ab50 con 0x7f794815c620 2023-08-11T15:53:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.222+0000 7f794ce23640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792c06a080 0x7f792c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.222+0000 7f7947fff640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] conn(0x7f792c0709a0 0x7f792c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.223+0000 7f7947fff640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] conn(0x7f792c0709a0 0x7f792c072de0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.223+0000 7f794ce23640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792c06a080 0x7f792c06c540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f793c009ad0 tx=0x7f793c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.225+0000 7f79457fa640 1 -- 172.21.15.27:0/1475147467 <== osd.4 v2:172.21.15.119:6808/2943646764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f792c0734b0 con 0x7f792c0709a0 2023-08-11T15:53:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.240+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 --> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f79480ca880 con 0x7f792c0709a0 2023-08-11T15:53:47.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.241+0000 7f79457fa640 1 -- 172.21.15.27:0/1475147467 <== osd.4 v2:172.21.15.119:6808/2943646764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f79480ca880 con 0x7f792c0709a0 2023-08-11T15:53:47.244 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:47.244 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-11T15:53:47.245 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.241+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] conn(0x7f792c0709a0 msgr2=0x7f792c072de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.241+0000 7f794f087640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:6808/2943646764,v1:172.21.15.119:6809/2943646764] conn(0x7f792c0709a0 0x7f792c072de0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.242+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792c06a080 msgr2=0x7f792c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.242+0000 7f794f087640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f792c06a080 0x7f792c06c540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f793c009ad0 tx=0x7f793c008040 comp rx=0 tx=0).stop 2023-08-11T15:53:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.242+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f794815c620 msgr2=0x7f794815f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.242+0000 7f794f087640 1 --2- 172.21.15.27:0/1475147467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f794815c620 0x7f794815f110 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f794000be60 tx=0x7f794000bf50 comp rx=0 tx=0).stop 2023-08-11T15:53:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.242+0000 7f794d624640 1 -- 172.21.15.27:0/1475147467 reap_dead start 2023-08-11T15:53:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.242+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 shutdown_connections 2023-08-11T15:53:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.242+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 >> 172.21.15.27:0/1475147467 conn(0x7f7948071110 msgr2=0x7f79480b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.242+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 shutdown_connections 2023-08-11T15:53:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.242+0000 7f794f087640 1 -- 172.21.15.27:0/1475147467 wait complete. 2023-08-11T15:53:47.262 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-08-11T15:53:47.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.498+0000 7fc291287640 1 Processor -- start 2023-08-11T15:53:47.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.498+0000 7fc291287640 1 -- start start 2023-08-11T15:53:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.500+0000 7fc291287640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc28c0cfb00 0x7fc28c167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.500+0000 7fc291287640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c0cda30 con 0x7fc28c0b55c0 2023-08-11T15:53:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.500+0000 7fc291287640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c161110 con 0x7fc28c15c620 2023-08-11T15:53:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.500+0000 7fc291287640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28c0ccc10 con 0x7fc28c0cfb00 2023-08-11T15:53:47.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.501+0000 7fc283fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc28c0b55c0 0x7fc28c0cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49684/0 (socket says 172.21.15.27:49684) 2023-08-11T15:53:47.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.501+0000 7fc283fff640 1 -- 172.21.15.27:0/2900559360 learned_addr learned my addr 172.21.15.27:0/2900559360 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.501+0000 7fc28b7fe640 1 --2- 172.21.15.27:0/2900559360 >> v2:172.21.15.119:3300/0 conn(0x7fc28c0cfb00 0x7fc28c167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.501+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625527415 0 0) 0x7fc28c0cda30 con 0x7fc28c0b55c0 2023-08-11T15:53:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.501+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc278003520 con 0x7fc28c0b55c0 2023-08-11T15:53:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3766771845 0 0) 0x7fc278003520 con 0x7fc28c0b55c0 2023-08-11T15:53:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 >> v2:172.21.15.119:3300/0 conn(0x7fc28c0cfb00 msgr2=0x7fc28c167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:47.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 --2- 172.21.15.27:0/2900559360 >> v2:172.21.15.119:3300/0 conn(0x7fc28c0cfb00 0x7fc28c167550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 >> v1:172.21.15.119:6789/0 conn(0x7fc28c15c620 legacy=0x7fc28c0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c0cce20 con 0x7fc28c0b55c0 2023-08-11T15:53:47.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7fc274001670 con 0x7fc28c0b55c0 2023-08-11T15:53:47.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 >> v1:172.21.15.27:6789/0 conn(0x7fc28c0b55c0 legacy=0x7fc28c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc278003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2780047c0 0x7fc278004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc278005180 0x7fc27800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0ccc10 con 0x7fc278003970 2023-08-11T15:53:47.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27800c020 con 0x7fc278005180 2023-08-11T15:53:47.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27800c1e0 con 0x7fc2780047c0 2023-08-11T15:53:47.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.502+0000 7fc28affd640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc278003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.503+0000 7fc28b7fe640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc278005180 0x7fc27800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.503+0000 7fc283fff640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2780047c0 0x7fc278004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.503+0000 7fc28affd640 1 -- 172.21.15.27:0/2900559360 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2780047c0 msgr2=0x7fc278004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:47.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.503+0000 7fc28affd640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2780047c0 0x7fc278004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.503+0000 7fc28affd640 1 -- 172.21.15.27:0/2900559360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc278005180 msgr2=0x7fc27800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:47.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.503+0000 7fc28affd640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc278005180 0x7fc27800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.503+0000 7fc28affd640 1 -- 172.21.15.27:0/2900559360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc278003520 con 0x7fc278003970 2023-08-11T15:53:47.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.503+0000 7fc28b7fe640 1 -- 172.21.15.27:0/2900559360 reap_dead start 2023-08-11T15:53:47.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.503+0000 7fc283fff640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2780047c0 0x7fc278004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:47.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.504+0000 7fc28affd640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc278003d90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc27c001750 tx=0x7fc2780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5c73e3180a09b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.504+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c0052d0 con 0x7fc278003970 2023-08-11T15:53:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.504+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27c02e050 con 0x7fc278003970 2023-08-11T15:53:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.504+0000 7fc28a7fc640 1 -- 172.21.15.27:0/2900559360 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c003830 con 0x7fc278003970 2023-08-11T15:53:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.505+0000 7fc291287640 1 -- 172.21.15.27:0/2900559360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 msgr2=0x7fc278003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.505+0000 7fc291287640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc278003d90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc27c001750 tx=0x7fc2780057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.505+0000 7fc291287640 1 -- 172.21.15.27:0/2900559360 shutdown_connections 2023-08-11T15:53:47.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.505+0000 7fc291287640 1 --2- 172.21.15.27:0/2900559360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc278003d90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.505+0000 7fc291287640 1 -- 172.21.15.27:0/2900559360 >> 172.21.15.27:0/2900559360 conn(0x7fc28c071110 msgr2=0x7fc28c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:47.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.505+0000 7fc291287640 1 -- 172.21.15.27:0/2900559360 shutdown_connections 2023-08-11T15:53:47.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.505+0000 7fc291287640 1 -- 172.21.15.27:0/2900559360 wait complete. 2023-08-11T15:53:47.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.507+0000 7fc291287640 1 Processor -- start 2023-08-11T15:53:47.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.507+0000 7fc291287640 1 -- start start 2023-08-11T15:53:47.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.508+0000 7fc291287640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc28c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.508+0000 7fc291287640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc28c0aba50 0x7fc28c0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.508+0000 7fc291287640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc28c156e30 0x7fc28c0a9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.508+0000 7fc291287640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0ab1a0 con 0x7fc278003970 2023-08-11T15:53:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.508+0000 7fc291287640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0b4fa0 con 0x7fc28c0aba50 2023-08-11T15:53:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.508+0000 7fc291287640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0b4d60 con 0x7fc28c156e30 2023-08-11T15:53:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.508+0000 7fc28affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc28c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.508+0000 7fc28affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc28c0cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51254/0 (socket says 172.21.15.27:51254) 2023-08-11T15:53:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.508+0000 7fc28affd640 1 -- 172.21.15.27:0/2630640506 learned_addr learned my addr 172.21.15.27:0/2630640506 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.509+0000 7fc28b7fe640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc28c156e30 0x7fc28c0a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.509+0000 7fc283fff640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc28c0aba50 0x7fc28c0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.509+0000 7fc28affd640 1 -- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc28c156e30 msgr2=0x7fc28c0a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.509+0000 7fc28affd640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc28c156e30 0x7fc28c0a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.509+0000 7fc28affd640 1 -- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc28c0aba50 msgr2=0x7fc28c0b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.509+0000 7fc28affd640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc28c0aba50 0x7fc28c0b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.509+0000 7fc28affd640 1 -- 172.21.15.27:0/2630640506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c0c9830 con 0x7fc278003970 2023-08-11T15:53:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.509+0000 7fc283fff640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc28c0aba50 0x7fc28c0b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.509+0000 7fc28affd640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc28c0cc310 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc27c001870 tx=0x7fc27c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.510+0000 7fc2837fe640 1 -- 172.21.15.27:0/2630640506 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c0076c0 con 0x7fc278003970 2023-08-11T15:53:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.510+0000 7fc2837fe640 1 -- 172.21.15.27:0/2630640506 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27c007a30 con 0x7fc278003970 2023-08-11T15:53:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.510+0000 7fc2837fe640 1 -- 172.21.15.27:0/2630640506 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c006c20 con 0x7fc278003970 2023-08-11T15:53:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.510+0000 7fc291287640 1 -- 172.21.15.27:0/2630640506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28c0c9b10 con 0x7fc278003970 2023-08-11T15:53:47.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.510+0000 7fc291287640 1 -- 172.21.15.27:0/2630640506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28c0ca0d0 con 0x7fc278003970 2023-08-11T15:53:47.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.511+0000 7fc291287640 1 -- 172.21.15.27:0/2630640506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28c0ca290 con 0x7fc278003970 2023-08-11T15:53:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.512+0000 7fc2837fe640 1 -- 172.21.15.27:0/2630640506 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc27c0035f0 con 0x7fc278003970 2023-08-11T15:53:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.513+0000 7fc2837fe640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc26c06a050 0x7fc26c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:47.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.513+0000 7fc2837fe640 1 -- 172.21.15.27:0/2630640506 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fc27c0a9e80 con 0x7fc278003970 2023-08-11T15:53:47.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.514+0000 7fc283fff640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc26c06a050 0x7fc26c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:47.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.515+0000 7fc283fff640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc26c06a050 0x7fc26c06c510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc274000ff0 tx=0x7fc274009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:47.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.517+0000 7fc2837fe640 1 -- 172.21.15.27:0/2630640506 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fc27c079250 con 0x7fc278003970 2023-08-11T15:53:47.586 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:53:47.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:53:47.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.714+0000 7fc291287640 1 -- 172.21.15.27:0/2630640506 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc28c0cd3e0 con 0x7fc26c06a050 2023-08-11T15:53:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.716+0000 7fc2837fe640 1 -- 172.21.15.27:0/2630640506 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc27400d040 con 0x7fc26c06a050 2023-08-11T15:53:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 4 to deep-scrub 2023-08-11T15:53:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 -- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc26c06a050 msgr2=0x7fc26c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc26c06a050 0x7fc26c06c510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc274000ff0 tx=0x7fc274009040 comp rx=0 tx=0).stop 2023-08-11T15:53:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 -- 172.21.15.27:0/2630640506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 msgr2=0x7fc28c0cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc28c0cc310 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc27c001870 tx=0x7fc27c001120 comp rx=0 tx=0).stop 2023-08-11T15:53:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 -- 172.21.15.27:0/2630640506 shutdown_connections 2023-08-11T15:53:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc26c06a050 0x7fc26c06c510 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc28c156e30 0x7fc28c0a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc28c0aba50 0x7fc28c0b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 --2- 172.21.15.27:0/2630640506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc278003970 0x7fc28c0cc310 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:47.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 -- 172.21.15.27:0/2630640506 >> 172.21.15.27:0/2630640506 conn(0x7fc28c071110 msgr2=0x7fc28c15bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:47.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 -- 172.21.15.27:0/2630640506 shutdown_connections 2023-08-11T15:53:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:47.718+0000 7fc291287640 1 -- 172.21.15.27:0/2630640506 wait complete. 2023-08-11T15:53:47.738 INFO:tasks.ceph:Scrubbing osd.5 2023-08-11T15:53:47.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-08-11T15:53:48.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.022+0000 7f13f3eba640 1 Processor -- start 2023-08-11T15:53:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.023+0000 7f13f3eba640 1 -- start start 2023-08-11T15:53:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.024+0000 7f13f3eba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13ec0cb430 0x7f13ec0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.024+0000 7f13f3eba640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13ec06f5d0 0x7f13ec06fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.024+0000 7f13f3eba640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0c8a30 con 0x7f13ec15a140 2023-08-11T15:53:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.024+0000 7f13f3eba640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec06ffa0 con 0x7f13ec0cb430 2023-08-11T15:53:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.024+0000 7f13f3eba640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec06c0c0 con 0x7f13ec06f5d0 2023-08-11T15:53:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.025+0000 7f13f1455640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13ec0cb430 0x7f13ec0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.025+0000 7f13f1455640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13ec0cb430 0x7f13ec0b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51258/0 (socket says 172.21.15.27:51258) 2023-08-11T15:53:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.025+0000 7f13f1455640 1 -- 172.21.15.27:0/3259772277 learned_addr learned my addr 172.21.15.27:0/3259772277 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.025+0000 7f13f2457640 1 --2- 172.21.15.27:0/3259772277 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13ec06f5d0 0x7f13ec06fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.025+0000 7f13f1455640 1 -- 172.21.15.27:0/3259772277 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13ec06f5d0 msgr2=0x7f13ec06fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.025+0000 7f13f1455640 1 --2- 172.21.15.27:0/3259772277 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13ec06f5d0 0x7f13ec06fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.025+0000 7f13f1455640 1 -- 172.21.15.27:0/3259772277 >> v1:172.21.15.119:6789/0 conn(0x7f13ec15a140 legacy=0x7f13ec06b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.025+0000 7f13f1455640 1 -- 172.21.15.27:0/3259772277 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec06c3a0 con 0x7f13ec0cb430 2023-08-11T15:53:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.026+0000 7f13f1455640 1 --2- 172.21.15.27:0/3259772277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13ec0cb430 0x7f13ec0b0e70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009870 tx=0x7f13dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681917f7ee8688b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.027+0000 7f13f0c54640 1 -- 172.21.15.27:0/3259772277 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc031640 con 0x7f13ec0cb430 2023-08-11T15:53:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.027+0000 7f13f0c54640 1 -- 172.21.15.27:0/3259772277 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc0317a0 con 0x7f13ec0cb430 2023-08-11T15:53:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.027+0000 7f13f0c54640 1 -- 172.21.15.27:0/3259772277 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc031b30 con 0x7f13ec0cb430 2023-08-11T15:53:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.027+0000 7f13f3eba640 1 -- 172.21.15.27:0/3259772277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13ec0cb430 msgr2=0x7f13ec0b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.027+0000 7f13f3eba640 1 --2- 172.21.15.27:0/3259772277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13ec0cb430 0x7f13ec0b0e70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009870 tx=0x7f13dc02b8f0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.028+0000 7f13f3eba640 1 -- 172.21.15.27:0/3259772277 shutdown_connections 2023-08-11T15:53:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.028+0000 7f13f3eba640 1 --2- 172.21.15.27:0/3259772277 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13ec06f5d0 0x7f13ec06fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.028+0000 7f13f3eba640 1 --2- 172.21.15.27:0/3259772277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13ec0cb430 0x7f13ec0b0e70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.028+0000 7f13f3eba640 1 -- 172.21.15.27:0/3259772277 >> 172.21.15.27:0/3259772277 conn(0x7f13ec0afb70 msgr2=0x7f13ec0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.028+0000 7f13f3eba640 1 -- 172.21.15.27:0/3259772277 shutdown_connections 2023-08-11T15:53:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.028+0000 7f13f3eba640 1 -- 172.21.15.27:0/3259772277 wait complete. 2023-08-11T15:53:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.029+0000 7f13f3eba640 1 Processor -- start 2023-08-11T15:53:48.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.030+0000 7f13f3eba640 1 -- start start 2023-08-11T15:53:48.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f3eba640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13ec06f5d0 0x7f13ec0c2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f3eba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13ec0cb430 0x7f13ec0c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f3eba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13ec15a140 0x7f13ec15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f3eba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0c8440 con 0x7f13ec0cb430 2023-08-11T15:53:48.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f3eba640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0c55b0 con 0x7f13ec15a140 2023-08-11T15:53:48.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f3eba640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0c57e0 con 0x7f13ec06f5d0 2023-08-11T15:53:48.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f1455640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13ec0cb430 0x7f13ec0c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f1455640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13ec0cb430 0x7f13ec0c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51270/0 (socket says 172.21.15.27:51270) 2023-08-11T15:53:48.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f1455640 1 -- 172.21.15.27:0/3388210012 learned_addr learned my addr 172.21.15.27:0/3388210012 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:48.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f1c56640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13ec06f5d0 0x7f13ec0c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.031+0000 7f13f2457640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13ec15a140 0x7f13ec15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13f1455640 1 -- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13ec06f5d0 msgr2=0x7f13ec0c2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13f1455640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13ec06f5d0 0x7f13ec0c2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13f1455640 1 -- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13ec15a140 msgr2=0x7f13ec15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13f1455640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13ec15a140 0x7f13ec15cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13f1455640 1 -- 172.21.15.27:0/3388210012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc0094d0 con 0x7f13ec0cb430 2023-08-11T15:53:48.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13f2457640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13ec15a140 0x7f13ec15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:48.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13f1455640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13ec0cb430 0x7f13ec0c5070 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f13dc000c00 tx=0x7f13dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13e27fc640 1 -- 172.21.15.27:0/3388210012 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc03ac60 con 0x7f13ec0cb430 2023-08-11T15:53:48.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13e27fc640 1 -- 172.21.15.27:0/3388210012 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc03adc0 con 0x7f13ec0cb430 2023-08-11T15:53:48.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13e27fc640 1 -- 172.21.15.27:0/3388210012 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc044630 con 0x7f13ec0cb430 2023-08-11T15:53:48.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.032+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec0c5ac0 con 0x7f13ec0cb430 2023-08-11T15:53:48.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.033+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec0c1d50 con 0x7f13ec0cb430 2023-08-11T15:53:48.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.034+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13ec06c680 con 0x7f13ec0cb430 2023-08-11T15:53:48.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.035+0000 7f13e27fc640 1 -- 172.21.15.27:0/3388210012 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f13dc02f050 con 0x7f13ec0cb430 2023-08-11T15:53:48.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.036+0000 7f13e27fc640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13d006a080 0x7f13d006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.036+0000 7f13e27fc640 1 -- 172.21.15.27:0/3388210012 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f13dc0aa2b0 con 0x7f13ec0cb430 2023-08-11T15:53:48.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.036+0000 7f13e27fc640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] conn(0x7f13d00709a0 0x7f13d0072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.036+0000 7f13e27fc640 1 -- 172.21.15.27:0/3388210012 --> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13d00734b0 con 0x7f13d00709a0 2023-08-11T15:53:48.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.036+0000 7f13e27fc640 1 -- 172.21.15.27:0/3388210012 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13dc0ac8f0 con 0x7f13ec0cb430 2023-08-11T15:53:48.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.036+0000 7f13f1c56640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13d006a080 0x7f13d006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.037+0000 7f13f2457640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] conn(0x7f13d00709a0 0x7f13d0072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.038+0000 7f13f2457640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] conn(0x7f13d00709a0 0x7f13d0072de0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.038+0000 7f13f1c56640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13d006a080 0x7f13d006c540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f13e800a790 tx=0x7f13e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.040+0000 7f13e27fc640 1 -- 172.21.15.27:0/3388210012 <== osd.5 v2:172.21.15.119:6816/1811178752 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f13d00734b0 con 0x7f13d00709a0 2023-08-11T15:53:48.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.055+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 --> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f13ec04e420 con 0x7f13d00709a0 2023-08-11T15:53:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.056+0000 7f13e27fc640 1 -- 172.21.15.27:0/3388210012 <== osd.5 v2:172.21.15.119:6816/1811178752 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f13ec04e420 con 0x7f13d00709a0 2023-08-11T15:53:48.059 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:48.059 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-11T15:53:48.059 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.056+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] conn(0x7f13d00709a0 msgr2=0x7f13d0072de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.056+0000 7f13f3eba640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:6816/1811178752,v1:172.21.15.119:6817/1811178752] conn(0x7f13d00709a0 0x7f13d0072de0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.056+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13d006a080 msgr2=0x7f13d006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.057+0000 7f13f3eba640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f13d006a080 0x7f13d006c540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f13e800a790 tx=0x7f13e8008040 comp rx=0 tx=0).stop 2023-08-11T15:53:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.057+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13ec0cb430 msgr2=0x7f13ec0c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.057+0000 7f13f3eba640 1 --2- 172.21.15.27:0/3388210012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13ec0cb430 0x7f13ec0c5070 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f13dc000c00 tx=0x7f13dc0046c0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.057+0000 7f13f2457640 1 -- 172.21.15.27:0/3388210012 reap_dead start 2023-08-11T15:53:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.057+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 shutdown_connections 2023-08-11T15:53:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.057+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 >> 172.21.15.27:0/3388210012 conn(0x7f13ec0afb70 msgr2=0x7f13ec0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.057+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 shutdown_connections 2023-08-11T15:53:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.057+0000 7f13f3eba640 1 -- 172.21.15.27:0/3388210012 wait complete. 2023-08-11T15:53:48.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-08-11T15:53:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.317+0000 7f63b93eb640 1 Processor -- start 2023-08-11T15:53:48.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.317+0000 7f63b93eb640 1 -- start start 2023-08-11T15:53:48.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.319+0000 7f63b93eb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63b415a020 0x7f63b415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.319+0000 7f63b93eb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f63b40c9600 0x7f63b4071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.319+0000 7f63b93eb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b40c85f0 con 0x7f63b406f5d0 2023-08-11T15:53:48.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.319+0000 7f63b93eb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b415eae0 con 0x7f63b415a020 2023-08-11T15:53:48.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.319+0000 7f63b93eb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b4071980 con 0x7f63b40c9600 2023-08-11T15:53:48.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.319+0000 7f63b3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63b415a020 0x7f63b415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.319+0000 7f63b8bea640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f63b406f5d0 0x7f63b406b870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49694/0 (socket says 172.21.15.27:49694) 2023-08-11T15:53:48.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.319+0000 7f63b8bea640 1 -- 172.21.15.27:0/145320242 learned_addr learned my addr 172.21.15.27:0/145320242 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.319+0000 7f63abfff640 1 --2- 172.21.15.27:0/145320242 >> v2:172.21.15.119:3300/0 conn(0x7f63b40c9600 0x7f63b4071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.320+0000 7f63b3fff640 1 -- 172.21.15.27:0/145320242 >> v2:172.21.15.119:3300/0 conn(0x7f63b40c9600 msgr2=0x7f63b4071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.320+0000 7f63b3fff640 1 --2- 172.21.15.27:0/145320242 >> v2:172.21.15.119:3300/0 conn(0x7f63b40c9600 0x7f63b4071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.320+0000 7f63b3fff640 1 -- 172.21.15.27:0/145320242 >> v1:172.21.15.27:6789/0 conn(0x7f63b406f5d0 legacy=0x7f63b406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.320+0000 7f63b3fff640 1 -- 172.21.15.27:0/145320242 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b40c8840 con 0x7f63b415a020 2023-08-11T15:53:48.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.320+0000 7f63b37fe640 1 -- 172.21.15.27:0/145320242 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966452289 0 0) 0x7f63b40c85f0 con 0x7f63b406f5d0 2023-08-11T15:53:48.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.320+0000 7f63abfff640 1 --2- 172.21.15.27:0/145320242 >> v2:172.21.15.119:3300/0 conn(0x7f63b40c9600 0x7f63b4071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:48.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.321+0000 7f63b3fff640 1 --2- 172.21.15.27:0/145320242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 0x7f63b415e5a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f63b40c8580 tx=0x7f63a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157162f8b86c5ea0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.321+0000 7f63b37fe640 1 -- 172.21.15.27:0/145320242 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a402e070 con 0x7f63b415a020 2023-08-11T15:53:48.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.321+0000 7f63b37fe640 1 -- 172.21.15.27:0/145320242 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a40315a0 con 0x7f63b415a020 2023-08-11T15:53:48.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.321+0000 7f63b37fe640 1 -- 172.21.15.27:0/145320242 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4031930 con 0x7f63b415a020 2023-08-11T15:53:48.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.322+0000 7f63b93eb640 1 -- 172.21.15.27:0/145320242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 msgr2=0x7f63b415e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.322+0000 7f63b93eb640 1 --2- 172.21.15.27:0/145320242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 0x7f63b415e5a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f63b40c8580 tx=0x7f63a402b840 comp rx=0 tx=0).stop 2023-08-11T15:53:48.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.322+0000 7f63b93eb640 1 -- 172.21.15.27:0/145320242 shutdown_connections 2023-08-11T15:53:48.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.322+0000 7f63b93eb640 1 --2- 172.21.15.27:0/145320242 >> v2:172.21.15.119:3300/0 conn(0x7f63b40c9600 0x7f63b4071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.322+0000 7f63b93eb640 1 --2- 172.21.15.27:0/145320242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 0x7f63b415e5a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.322+0000 7f63b93eb640 1 -- 172.21.15.27:0/145320242 >> 172.21.15.27:0/145320242 conn(0x7f63b40afb70 msgr2=0x7f63b40aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:48.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.322+0000 7f63b93eb640 1 -- 172.21.15.27:0/145320242 shutdown_connections 2023-08-11T15:53:48.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.322+0000 7f63b93eb640 1 -- 172.21.15.27:0/145320242 wait complete. 2023-08-11T15:53:48.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.324+0000 7f63b93eb640 1 Processor -- start 2023-08-11T15:53:48.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.324+0000 7f63b93eb640 1 -- start start 2023-08-11T15:53:48.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.325+0000 7f63b93eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63b406f5d0 0x7f63b40c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.325+0000 7f63b93eb640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63b40c9600 0x7f63b41f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.325+0000 7f63b93eb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 0x7f63b40c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.325+0000 7f63b93eb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b40c5f30 con 0x7f63b415a020 2023-08-11T15:53:48.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.325+0000 7f63b93eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b40c5970 con 0x7f63b406f5d0 2023-08-11T15:53:48.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.325+0000 7f63b93eb640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b40c5c50 con 0x7f63b40c9600 2023-08-11T15:53:48.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.325+0000 7f63b8bea640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 0x7f63b40c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.325+0000 7f63b8bea640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 0x7f63b40c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51290/0 (socket says 172.21.15.27:51290) 2023-08-11T15:53:48.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.325+0000 7f63b8bea640 1 -- 172.21.15.27:0/3172740239 learned_addr learned my addr 172.21.15.27:0/3172740239 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:48.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63abfff640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63b40c9600 0x7f63b41f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63b3fff640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63b406f5d0 0x7f63b40c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63b8bea640 1 -- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63b40c9600 msgr2=0x7f63b41f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63b8bea640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63b40c9600 0x7f63b41f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63b8bea640 1 -- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63b406f5d0 msgr2=0x7f63b40c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63b8bea640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63b406f5d0 0x7f63b40c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63b8bea640 1 -- 172.21.15.27:0/3172740239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a4009400 con 0x7f63b415a020 2023-08-11T15:53:48.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63abfff640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63b40c9600 0x7f63b41f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:48.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63b8bea640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 0x7f63b40c7450 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00c700 tx=0x7f63ac00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63b17fa640 1 -- 172.21.15.27:0/3172740239 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00ce10 con 0x7f63b415a020 2023-08-11T15:53:48.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63b17fa640 1 -- 172.21.15.27:0/3172740239 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac00a640 con 0x7f63b415a020 2023-08-11T15:53:48.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.326+0000 7f63b17fa640 1 -- 172.21.15.27:0/3172740239 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00aa70 con 0x7f63b415a020 2023-08-11T15:53:48.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.327+0000 7f63b93eb640 1 -- 172.21.15.27:0/3172740239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b40b0d90 con 0x7f63b415a020 2023-08-11T15:53:48.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.327+0000 7f63b93eb640 1 -- 172.21.15.27:0/3172740239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b415a400 con 0x7f63b415a020 2023-08-11T15:53:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.328+0000 7f63b93eb640 1 -- 172.21.15.27:0/3172740239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b40b12b0 con 0x7f63b415a020 2023-08-11T15:53:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.329+0000 7f63b17fa640 1 -- 172.21.15.27:0/3172740239 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f63ac00abd0 con 0x7f63b415a020 2023-08-11T15:53:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.330+0000 7f63b17fa640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f639806a050 0x7f639806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.330+0000 7f63b17fa640 1 -- 172.21.15.27:0/3172740239 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f63ac087840 con 0x7f63b415a020 2023-08-11T15:53:48.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.331+0000 7f63b3fff640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f639806a050 0x7f639806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.332+0000 7f63b3fff640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f639806a050 0x7f639806c510 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f63a4036000 tx=0x7f63a4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.334+0000 7f63b17fa640 1 -- 172.21.15.27:0/3172740239 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f63b40b12b0 con 0x7f63b415a020 2023-08-11T15:53:48.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.540+0000 7f63b93eb640 1 -- 172.21.15.27:0/3172740239 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f63b40c8e00 con 0x7f639806a050 2023-08-11T15:53:48.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.541+0000 7f63b17fa640 1 -- 172.21.15.27:0/3172740239 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f63a4031910 con 0x7f639806a050 2023-08-11T15:53:48.544 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 5 to deep-scrub 2023-08-11T15:53:48.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 -- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f639806a050 msgr2=0x7f639806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f639806a050 0x7f639806c510 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f63a4036000 tx=0x7f63a4008030 comp rx=0 tx=0).stop 2023-08-11T15:53:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 -- 172.21.15.27:0/3172740239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 msgr2=0x7f63b40c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 0x7f63b40c7450 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00c700 tx=0x7f63ac00cbc0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 -- 172.21.15.27:0/3172740239 shutdown_connections 2023-08-11T15:53:48.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f639806a050 0x7f639806c510 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63b40c9600 0x7f63b41f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b415a020 0x7f63b40c7450 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 --2- 172.21.15.27:0/3172740239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63b406f5d0 0x7f63b40c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 -- 172.21.15.27:0/3172740239 >> 172.21.15.27:0/3172740239 conn(0x7f63b40afb70 msgr2=0x7f63b40af400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:48.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 -- 172.21.15.27:0/3172740239 shutdown_connections 2023-08-11T15:53:48.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.544+0000 7f63b93eb640 1 -- 172.21.15.27:0/3172740239 wait complete. 2023-08-11T15:53:48.565 INFO:tasks.ceph:Scrubbing osd.6 2023-08-11T15:53:48.566 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-08-11T15:53:48.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.805+0000 7f9361b39640 1 Processor -- start 2023-08-11T15:53:48.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.806+0000 7f9361b39640 1 -- start start 2023-08-11T15:53:48.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.807+0000 7f9361b39640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f935c15a140 0x7f935c06b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.807+0000 7f9361b39640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f935c0cb430 0x7f935c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.807+0000 7f9361b39640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c0c8a30 con 0x7f935c06f5d0 2023-08-11T15:53:48.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.807+0000 7f9361b39640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935c06c0c0 con 0x7f935c15a140 2023-08-11T15:53:48.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.807+0000 7f9361b39640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c06c3a0 con 0x7f935c0cb430 2023-08-11T15:53:48.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.808+0000 7f935bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f935c06f5d0 0x7f935c0c8ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49698/0 (socket says 172.21.15.27:49698) 2023-08-11T15:53:48.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.808+0000 7f935bfff640 1 -- 172.21.15.27:0/4074663737 learned_addr learned my addr 172.21.15.27:0/4074663737 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:48.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.808+0000 7f935b7fe640 1 --2- 172.21.15.27:0/4074663737 >> v2:172.21.15.119:3300/0 conn(0x7f935c15a140 0x7f935c06b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.808+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469016091 0 0) 0x7f935c0c8a30 con 0x7f935c06f5d0 2023-08-11T15:53:48.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.808+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9344003520 con 0x7f935c06f5d0 2023-08-11T15:53:48.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.808+0000 7f935affd640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f935c0cb430 0x7f935c0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1445657428 0 0) 0x7f9344003520 con 0x7f935c06f5d0 2023-08-11T15:53:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f935c0cb430 msgr2=0x7f935c0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f935c0cb430 0x7f935c0b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 >> v2:172.21.15.119:3300/0 conn(0x7f935c15a140 msgr2=0x7f935c06b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:48.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 --2- 172.21.15.27:0/4074663737 >> v2:172.21.15.119:3300/0 conn(0x7f935c15a140 0x7f935c06b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c06c680 con 0x7f935c06f5d0 2023-08-11T15:53:48.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f9354001670 con 0x7f935c06f5d0 2023-08-11T15:53:48.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 >> v1:172.21.15.27:6789/0 conn(0x7f935c06f5d0 legacy=0x7f935c0c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9344003970 0x7f9344003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93440047c0 0x7f9344004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f935c0cb430 0x7f9344005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c06c0c0 con 0x7f93440047c0 2023-08-11T15:53:48.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c06c3a0 con 0x7f9344003970 2023-08-11T15:53:48.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934400b980 con 0x7f935c0cb430 2023-08-11T15:53:48.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.809+0000 7f935affd640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93440047c0 0x7f9344004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.810+0000 7f935bfff640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f935c0cb430 0x7f9344005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.810+0000 7f935affd640 1 -- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f935c0cb430 msgr2=0x7f9344005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:48.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.810+0000 7f935affd640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f935c0cb430 0x7f9344005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.810+0000 7f935affd640 1 -- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9344003970 msgr2=0x7f9344003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:48.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.810+0000 7f935affd640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9344003970 0x7f9344003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.810+0000 7f935affd640 1 -- 172.21.15.27:0/4074663737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344003520 con 0x7f93440047c0 2023-08-11T15:53:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.810+0000 7f935bfff640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f935c0cb430 0x7f9344005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:48.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.811+0000 7f935affd640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93440047c0 0x7f9344004c40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f935000ba30 tx=0x7f93440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7636caf4b5b3cb53 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.811+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350032640 con 0x7f93440047c0 2023-08-11T15:53:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.811+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93500327a0 con 0x7f93440047c0 2023-08-11T15:53:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.811+0000 7f935a7fc640 1 -- 172.21.15.27:0/4074663737 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350032b30 con 0x7f93440047c0 2023-08-11T15:53:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.812+0000 7f9361b39640 1 -- 172.21.15.27:0/4074663737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93440047c0 msgr2=0x7f9344004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.812+0000 7f9361b39640 1 --2- 172.21.15.27:0/4074663737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93440047c0 0x7f9344004c40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f935000ba30 tx=0x7f93440057f0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.812+0000 7f935bfff640 1 -- 172.21.15.27:0/4074663737 reap_dead start 2023-08-11T15:53:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.812+0000 7f9361b39640 1 -- 172.21.15.27:0/4074663737 shutdown_connections 2023-08-11T15:53:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.812+0000 7f9361b39640 1 -- 172.21.15.27:0/4074663737 >> 172.21.15.27:0/4074663737 conn(0x7f935c0afb70 msgr2=0x7f935c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.812+0000 7f9361b39640 1 -- 172.21.15.27:0/4074663737 shutdown_connections 2023-08-11T15:53:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.812+0000 7f9361b39640 1 -- 172.21.15.27:0/4074663737 wait complete. 2023-08-11T15:53:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.814+0000 7f9361b39640 1 Processor -- start 2023-08-11T15:53:48.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.814+0000 7f9361b39640 1 -- start start 2023-08-11T15:53:48.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.815+0000 7f9361b39640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93440047c0 0x7f935c14e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.815+0000 7f9361b39640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f935c0ab8e0 0x7f935c156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.815+0000 7f9361b39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f935c0abd60 0x7f935c156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.815+0000 7f9361b39640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0acfb0 con 0x7f935c0ab8e0 2023-08-11T15:53:48.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.815+0000 7f9361b39640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0ac330 con 0x7f935c0abd60 2023-08-11T15:53:48.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.815+0000 7f9361b39640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0ac610 con 0x7f93440047c0 2023-08-11T15:53:48.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.815+0000 7f935affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f935c0ab8e0 0x7f935c156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.815+0000 7f935affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f935c0ab8e0 0x7f935c156a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51312/0 (socket says 172.21.15.27:51312) 2023-08-11T15:53:48.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.815+0000 7f935affd640 1 -- 172.21.15.27:0/3336174021 learned_addr learned my addr 172.21.15.27:0/3336174021 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:48.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.816+0000 7f935b7fe640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93440047c0 0x7f935c14e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.816+0000 7f935affd640 1 -- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93440047c0 msgr2=0x7f935c14e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.816+0000 7f935affd640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93440047c0 0x7f935c14e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.816+0000 7f935bfff640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f935c0abd60 0x7f935c156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.816+0000 7f935affd640 1 -- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f935c0abd60 msgr2=0x7f935c156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.816+0000 7f935affd640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f935c0abd60 0x7f935c156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.816+0000 7f935affd640 1 -- 172.21.15.27:0/3336174021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c0ac8f0 con 0x7f935c0ab8e0 2023-08-11T15:53:48.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.816+0000 7f935b7fe640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93440047c0 0x7f935c14e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:48.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.816+0000 7f935affd640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f935c0ab8e0 0x7f935c156a90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f935002bec0 tx=0x7f9350004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.817+0000 7f9343fff640 1 -- 172.21.15.27:0/3336174021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350004220 con 0x7f935c0ab8e0 2023-08-11T15:53:48.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.817+0000 7f9343fff640 1 -- 172.21.15.27:0/3336174021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93500043a0 con 0x7f935c0ab8e0 2023-08-11T15:53:48.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.817+0000 7f9343fff640 1 -- 172.21.15.27:0/3336174021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935003ba30 con 0x7f935c0ab8e0 2023-08-11T15:53:48.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.817+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c0acbd0 con 0x7f935c0ab8e0 2023-08-11T15:53:48.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.817+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c0ad3d0 con 0x7f935c0ab8e0 2023-08-11T15:53:48.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.818+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f935c06bdb0 con 0x7f935c0ab8e0 2023-08-11T15:53:48.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.819+0000 7f9343fff640 1 -- 172.21.15.27:0/3336174021 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f935003bb90 con 0x7f935c0ab8e0 2023-08-11T15:53:48.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.820+0000 7f9343fff640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f934806a050 0x7f934806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.820+0000 7f9343fff640 1 -- 172.21.15.27:0/3336174021 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f93500aab80 con 0x7f935c0ab8e0 2023-08-11T15:53:48.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.820+0000 7f9343fff640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] conn(0x7f9348070970 0x7f9348072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:48.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.820+0000 7f9343fff640 1 -- 172.21.15.27:0/3336174021 --> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9348073480 con 0x7f9348070970 2023-08-11T15:53:48.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.820+0000 7f9343fff640 1 -- 172.21.15.27:0/3336174021 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93500af2a0 con 0x7f935c0ab8e0 2023-08-11T15:53:48.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.821+0000 7f935b7fe640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f934806a050 0x7f934806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.821+0000 7f935bfff640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] conn(0x7f9348070970 0x7f9348072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:48.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.822+0000 7f935bfff640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] conn(0x7f9348070970 0x7f9348072db0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.822+0000 7f935b7fe640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f934806a050 0x7f934806c510 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f934c009fd0 tx=0x7f934c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.824+0000 7f9343fff640 1 -- 172.21.15.27:0/3336174021 <== osd.6 v2:172.21.15.119:6800/2672123554 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9348073480 con 0x7f9348070970 2023-08-11T15:53:48.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.840+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 --> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f935c04e420 con 0x7f9348070970 2023-08-11T15:53:48.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.841+0000 7f9343fff640 1 -- 172.21.15.27:0/3336174021 <== osd.6 v2:172.21.15.119:6800/2672123554 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f935c04e420 con 0x7f9348070970 2023-08-11T15:53:48.844 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:48.844 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-11T15:53:48.845 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:48.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.842+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] conn(0x7f9348070970 msgr2=0x7f9348072db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.842+0000 7f9361b39640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:6800/2672123554,v1:172.21.15.119:6801/2672123554] conn(0x7f9348070970 0x7f9348072db0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:48.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.842+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f934806a050 msgr2=0x7f934806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.842+0000 7f9361b39640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f934806a050 0x7f934806c510 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f934c009fd0 tx=0x7f934c00b040 comp rx=0 tx=0).stop 2023-08-11T15:53:48.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.842+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f935c0ab8e0 msgr2=0x7f935c156a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:48.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.842+0000 7f9361b39640 1 --2- 172.21.15.27:0/3336174021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f935c0ab8e0 0x7f935c156a90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f935002bec0 tx=0x7f9350004730 comp rx=0 tx=0).stop 2023-08-11T15:53:48.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.842+0000 7f935bfff640 1 -- 172.21.15.27:0/3336174021 reap_dead start 2023-08-11T15:53:48.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.842+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 shutdown_connections 2023-08-11T15:53:48.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.842+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 >> 172.21.15.27:0/3336174021 conn(0x7f935c0afb70 msgr2=0x7f935c157e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:48.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.842+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 shutdown_connections 2023-08-11T15:53:48.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:48.843+0000 7f9361b39640 1 -- 172.21.15.27:0/3336174021 wait complete. 2023-08-11T15:53:48.862 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-08-11T15:53:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.091+0000 7fc26108e640 1 Processor -- start 2023-08-11T15:53:49.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.091+0000 7fc26108e640 1 -- start start 2023-08-11T15:53:49.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.093+0000 7fc26108e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc25c0cc520 0x7fc25c15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.093+0000 7fc26108e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c15d1e0 0x7fc25c0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.093+0000 7fc26108e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c15da30 con 0x7fc25c160d20 2023-08-11T15:53:49.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.093+0000 7fc26108e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c156fa0 con 0x7fc25c0cc520 2023-08-11T15:53:49.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.093+0000 7fc26108e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0ca570 con 0x7fc25c15d1e0 2023-08-11T15:53:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.093+0000 7fc25a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc25c0cc520 0x7fc25c15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.093+0000 7fc25ad9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc25c160d20 0x7fc25c0cbe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49704/0 (socket says 172.21.15.27:49704) 2023-08-11T15:53:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.093+0000 7fc25ad9d640 1 -- 172.21.15.27:0/1262896783 learned_addr learned my addr 172.21.15.27:0/1262896783 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.094+0000 7fc25b59e640 1 --2- 172.21.15.27:0/1262896783 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c15d1e0 0x7fc25c0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.094+0000 7fc25a59c640 1 -- 172.21.15.27:0/1262896783 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c15d1e0 msgr2=0x7fc25c0c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.094+0000 7fc25a59c640 1 --2- 172.21.15.27:0/1262896783 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c15d1e0 0x7fc25c0c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.094+0000 7fc25a59c640 1 -- 172.21.15.27:0/1262896783 >> v1:172.21.15.27:6789/0 conn(0x7fc25c160d20 legacy=0x7fc25c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.094+0000 7fc259d9b640 1 -- 172.21.15.27:0/1262896783 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725748101 0 0) 0x7fc25c15da30 con 0x7fc25c160d20 2023-08-11T15:53:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.094+0000 7fc25a59c640 1 -- 172.21.15.27:0/1262896783 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c0ca850 con 0x7fc25c0cc520 2023-08-11T15:53:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.094+0000 7fc25b59e640 1 --2- 172.21.15.27:0/1262896783 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c15d1e0 0x7fc25c0c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:49.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.095+0000 7fc25a59c640 1 --2- 172.21.15.27:0/1262896783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 0x7fc25c15cca0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc2440097a0 tx=0x7fc24402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd887a001feb837 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:49.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.095+0000 7fc259d9b640 1 -- 172.21.15.27:0/1262896783 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24402d040 con 0x7fc25c0cc520 2023-08-11T15:53:49.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.095+0000 7fc259d9b640 1 -- 172.21.15.27:0/1262896783 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc244031810 con 0x7fc25c0cc520 2023-08-11T15:53:49.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.095+0000 7fc259d9b640 1 -- 172.21.15.27:0/1262896783 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244031ba0 con 0x7fc25c0cc520 2023-08-11T15:53:49.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.096+0000 7fc26108e640 1 -- 172.21.15.27:0/1262896783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 msgr2=0x7fc25c15cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.096+0000 7fc26108e640 1 --2- 172.21.15.27:0/1262896783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 0x7fc25c15cca0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc2440097a0 tx=0x7fc24402b8b0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.096+0000 7fc26108e640 1 -- 172.21.15.27:0/1262896783 shutdown_connections 2023-08-11T15:53:49.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.096+0000 7fc26108e640 1 --2- 172.21.15.27:0/1262896783 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c15d1e0 0x7fc25c0c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.096+0000 7fc26108e640 1 --2- 172.21.15.27:0/1262896783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 0x7fc25c15cca0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.096+0000 7fc26108e640 1 -- 172.21.15.27:0/1262896783 >> 172.21.15.27:0/1262896783 conn(0x7fc25c0afb70 msgr2=0x7fc25c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.096+0000 7fc26108e640 1 -- 172.21.15.27:0/1262896783 shutdown_connections 2023-08-11T15:53:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.096+0000 7fc26108e640 1 -- 172.21.15.27:0/1262896783 wait complete. 2023-08-11T15:53:49.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.098+0000 7fc26108e640 1 Processor -- start 2023-08-11T15:53:49.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.098+0000 7fc26108e640 1 -- start start 2023-08-11T15:53:49.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.099+0000 7fc26108e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 0x7fc25c0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.099+0000 7fc26108e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc25c15d1e0 0x7fc25c145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.099+0000 7fc26108e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c160d20 0x7fc25c146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.099+0000 7fc26108e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0c3cd0 con 0x7fc25c0cc520 2023-08-11T15:53:49.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.099+0000 7fc26108e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c147350 con 0x7fc25c15d1e0 2023-08-11T15:53:49.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.099+0000 7fc26108e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c146740 con 0x7fc25c160d20 2023-08-11T15:53:49.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.099+0000 7fc25ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 0x7fc25c0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.099+0000 7fc25ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 0x7fc25c0c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51342/0 (socket says 172.21.15.27:51342) 2023-08-11T15:53:49.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.099+0000 7fc25ad9d640 1 -- 172.21.15.27:0/228868759 learned_addr learned my addr 172.21.15.27:0/228868759 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:49.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.100+0000 7fc25b59e640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c160d20 0x7fc25c146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.100+0000 7fc25ad9d640 1 -- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c160d20 msgr2=0x7fc25c146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.100+0000 7fc25ad9d640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c160d20 0x7fc25c146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.100+0000 7fc25ad9d640 1 -- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc25c15d1e0 msgr2=0x7fc25c145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.100+0000 7fc25a59c640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc25c15d1e0 0x7fc25c145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.100+0000 7fc25ad9d640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc25c15d1e0 0x7fc25c145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.100+0000 7fc25ad9d640 1 -- 172.21.15.27:0/228868759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc244009400 con 0x7fc25c0cc520 2023-08-11T15:53:49.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.100+0000 7fc25b59e640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c160d20 0x7fc25c146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:53:49.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.100+0000 7fc25ad9d640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 0x7fc25c0c3790 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00c520 tx=0x7fc24c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:49.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.100+0000 7fc2537fe640 1 -- 172.21.15.27:0/228868759 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c007480 con 0x7fc25c0cc520 2023-08-11T15:53:49.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.101+0000 7fc2537fe640 1 -- 172.21.15.27:0/228868759 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c0075e0 con 0x7fc25c0cc520 2023-08-11T15:53:49.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.101+0000 7fc2537fe640 1 -- 172.21.15.27:0/228868759 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0059a0 con 0x7fc25c0cc520 2023-08-11T15:53:49.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.101+0000 7fc26108e640 1 -- 172.21.15.27:0/228868759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c146a20 con 0x7fc25c0cc520 2023-08-11T15:53:49.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.101+0000 7fc26108e640 1 -- 172.21.15.27:0/228868759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c146fe0 con 0x7fc25c0cc520 2023-08-11T15:53:49.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.102+0000 7fc26108e640 1 -- 172.21.15.27:0/228868759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25c073990 con 0x7fc25c0cc520 2023-08-11T15:53:49.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.103+0000 7fc2537fe640 1 -- 172.21.15.27:0/228868759 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc24c005b00 con 0x7fc25c0cc520 2023-08-11T15:53:49.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.104+0000 7fc2537fe640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc23c06a050 0x7fc23c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.104+0000 7fc2537fe640 1 -- 172.21.15.27:0/228868759 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fc24c0878e0 con 0x7fc25c0cc520 2023-08-11T15:53:49.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.105+0000 7fc25a59c640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc23c06a050 0x7fc23c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.106+0000 7fc25a59c640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc23c06a050 0x7fc23c06c510 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc244036000 tx=0x7fc24403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:49.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.109+0000 7fc2537fe640 1 -- 172.21.15.27:0/228868759 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fc24c0531c0 con 0x7fc25c0cc520 2023-08-11T15:53:49.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.301+0000 7fc26108e640 1 -- 172.21.15.27:0/228868759 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc25c0cae10 con 0x7fc23c06a050 2023-08-11T15:53:49.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.302+0000 7fc2537fe640 1 -- 172.21.15.27:0/228868759 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc244031610 con 0x7fc23c06a050 2023-08-11T15:53:49.305 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 6 to deep-scrub 2023-08-11T15:53:49.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 -- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc23c06a050 msgr2=0x7fc23c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc23c06a050 0x7fc23c06c510 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc244036000 tx=0x7fc24403c040 comp rx=0 tx=0).stop 2023-08-11T15:53:49.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 -- 172.21.15.27:0/228868759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 msgr2=0x7fc25c0c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 0x7fc25c0c3790 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00c520 tx=0x7fc24c00c9e0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 -- 172.21.15.27:0/228868759 shutdown_connections 2023-08-11T15:53:49.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fc23c06a050 0x7fc23c06c510 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc25c160d20 0x7fc25c146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc25c15d1e0 0x7fc25c145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 --2- 172.21.15.27:0/228868759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25c0cc520 0x7fc25c0c3790 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 -- 172.21.15.27:0/228868759 >> 172.21.15.27:0/228868759 conn(0x7fc25c0afb70 msgr2=0x7fc25c0c4570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:49.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 -- 172.21.15.27:0/228868759 shutdown_connections 2023-08-11T15:53:49.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.305+0000 7fc26108e640 1 -- 172.21.15.27:0/228868759 wait complete. 2023-08-11T15:53:49.327 INFO:tasks.ceph:Scrubbing osd.7 2023-08-11T15:53:49.327 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-08-11T15:53:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.563+0000 7f1d62349640 1 Processor -- start 2023-08-11T15:53:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.563+0000 7f1d62349640 1 -- start start 2023-08-11T15:53:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.565+0000 7f1d62349640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c15c410 0x7f1d5c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.565+0000 7f1d62349640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1d5c0cd180 0x7f1d5c0b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.565+0000 7f1d62349640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5c0ca730 con 0x7f1d5c15c7f0 2023-08-11T15:53:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.565+0000 7f1d62349640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5c0ca9d0 con 0x7f1d5c0cd180 2023-08-11T15:53:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.565+0000 7f1d62349640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0cacb0 con 0x7f1d5c15c410 2023-08-11T15:53:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.566+0000 7f1d61347640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c15c410 0x7f1d5c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.566+0000 7f1d61b48640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1d5c0cd180 0x7f1d5c0b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.566+0000 7f1d61347640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c15c410 0x7f1d5c0cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.27:43916/0 (socket says 172.21.15.27:43916) 2023-08-11T15:53:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.566+0000 7f1d61347640 1 -- 172.21.15.27:0/3361178175 learned_addr learned my addr 172.21.15.27:0/3361178175 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.567+0000 7f1d61347640 1 -- 172.21.15.27:0/3361178175 >> v2:172.21.15.119:3300/0 conn(0x7f1d5c0cd180 msgr2=0x7f1d5c0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.567+0000 7f1d61347640 1 --2- 172.21.15.27:0/3361178175 >> v2:172.21.15.119:3300/0 conn(0x7f1d5c0cd180 0x7f1d5c0b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.567+0000 7f1d61347640 1 -- 172.21.15.27:0/3361178175 >> v1:172.21.15.119:6789/0 conn(0x7f1d5c15c7f0 legacy=0x7f1d5c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.567+0000 7f1d61347640 1 -- 172.21.15.27:0/3361178175 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c0caf90 con 0x7f1d5c15c410 2023-08-11T15:53:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.569+0000 7f1d61347640 1 --2- 172.21.15.27:0/3361178175 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c15c410 0x7f1d5c0cee30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009870 tx=0x7f1d5802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc401a2e7825615 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.569+0000 7f1d53fff640 1 -- 172.21.15.27:0/3361178175 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58030840 con 0x7f1d5c15c410 2023-08-11T15:53:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.570+0000 7f1d53fff640 1 -- 172.21.15.27:0/3361178175 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d580309a0 con 0x7f1d5c15c410 2023-08-11T15:53:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.570+0000 7f1d53fff640 1 -- 172.21.15.27:0/3361178175 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58030d30 con 0x7f1d5c15c410 2023-08-11T15:53:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.570+0000 7f1d62349640 1 -- 172.21.15.27:0/3361178175 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c15c410 msgr2=0x7f1d5c0cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.570+0000 7f1d62349640 1 --2- 172.21.15.27:0/3361178175 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c15c410 0x7f1d5c0cee30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009870 tx=0x7f1d5802b980 comp rx=0 tx=0).stop 2023-08-11T15:53:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.570+0000 7f1d62349640 1 -- 172.21.15.27:0/3361178175 shutdown_connections 2023-08-11T15:53:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.570+0000 7f1d62349640 1 --2- 172.21.15.27:0/3361178175 >> v2:172.21.15.119:3300/0 conn(0x7f1d5c0cd180 0x7f1d5c0b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.570+0000 7f1d62349640 1 --2- 172.21.15.27:0/3361178175 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c15c410 0x7f1d5c0cee30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.570+0000 7f1d62349640 1 -- 172.21.15.27:0/3361178175 >> 172.21.15.27:0/3361178175 conn(0x7f1d5c0d00e0 msgr2=0x7f1d5c0d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.570+0000 7f1d62349640 1 -- 172.21.15.27:0/3361178175 shutdown_connections 2023-08-11T15:53:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.570+0000 7f1d62349640 1 -- 172.21.15.27:0/3361178175 wait complete. 2023-08-11T15:53:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.572+0000 7f1d62349640 1 Processor -- start 2023-08-11T15:53:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.572+0000 7f1d62349640 1 -- start start 2023-08-11T15:53:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.573+0000 7f1d62349640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c0cd180 0x7f1d5c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.573+0000 7f1d62349640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5c15c410 0x7f1d5c1f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.573+0000 7f1d62349640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d5c15c7f0 0x7f1d5c0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.573+0000 7f1d62349640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0c45e0 con 0x7f1d5c15c410 2023-08-11T15:53:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.573+0000 7f1d62349640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1f2e10 con 0x7f1d5c15c7f0 2023-08-11T15:53:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.573+0000 7f1d62349640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0c84c0 con 0x7f1d5c0cd180 2023-08-11T15:53:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.574+0000 7f1d60b46640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5c15c410 0x7f1d5c1f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.574+0000 7f1d60b46640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5c15c410 0x7f1d5c1f28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35232/0 (socket says 172.21.15.27:35232) 2023-08-11T15:53:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.574+0000 7f1d60b46640 1 -- 172.21.15.27:0/2834110533 learned_addr learned my addr 172.21.15.27:0/2834110533 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.574+0000 7f1d61347640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c0cd180 0x7f1d5c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.574+0000 7f1d61b48640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d5c15c7f0 0x7f1d5c0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.574+0000 7f1d60b46640 1 -- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c0cd180 msgr2=0x7f1d5c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.574+0000 7f1d60b46640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1d5c0cd180 0x7f1d5c0c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.574+0000 7f1d60b46640 1 -- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d5c15c7f0 msgr2=0x7f1d5c0c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.574+0000 7f1d60b46640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d5c15c7f0 0x7f1d5c0c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.574+0000 7f1d60b46640 1 -- 172.21.15.27:0/2834110533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d580094d0 con 0x7f1d5c15c410 2023-08-11T15:53:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.575+0000 7f1d60b46640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5c15c410 0x7f1d5c1f28d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00c430 tx=0x7f1d4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.575+0000 7f1d51ffb640 1 -- 172.21.15.27:0/2834110533 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0052d0 con 0x7f1d5c15c410 2023-08-11T15:53:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.575+0000 7f1d51ffb640 1 -- 172.21.15.27:0/2834110533 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c005430 con 0x7f1d5c15c410 2023-08-11T15:53:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.575+0000 7f1d51ffb640 1 -- 172.21.15.27:0/2834110533 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c003830 con 0x7f1d5c15c410 2023-08-11T15:53:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.575+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c0c3cb0 con 0x7f1d5c15c410 2023-08-11T15:53:49.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.575+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c0c4270 con 0x7f1d5c15c410 2023-08-11T15:53:49.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.576+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d5c0cd800 con 0x7f1d5c15c410 2023-08-11T15:53:49.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.577+0000 7f1d51ffb640 1 -- 172.21.15.27:0/2834110533 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1d4c003990 con 0x7f1d5c15c410 2023-08-11T15:53:49.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.578+0000 7f1d51ffb640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d4006a050 0x7f1d4006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.579+0000 7f1d51ffb640 1 -- 172.21.15.27:0/2834110533 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f1d4c087900 con 0x7f1d5c15c410 2023-08-11T15:53:49.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.579+0000 7f1d51ffb640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] conn(0x7f1d40070970 0x7f1d40072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.579+0000 7f1d51ffb640 1 -- 172.21.15.27:0/2834110533 --> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d40073480 con 0x7f1d40070970 2023-08-11T15:53:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.579+0000 7f1d51ffb640 1 -- 172.21.15.27:0/2834110533 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d4c08c2a0 con 0x7f1d5c15c410 2023-08-11T15:53:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.579+0000 7f1d61347640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d4006a050 0x7f1d4006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.579+0000 7f1d61b48640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] conn(0x7f1d40070970 0x7f1d40072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.580+0000 7f1d61b48640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] conn(0x7f1d40070970 0x7f1d40072db0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.580+0000 7f1d61347640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d4006a050 0x7f1d4006c510 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1d58036000 tx=0x7f1d58008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.583+0000 7f1d51ffb640 1 -- 172.21.15.27:0/2834110533 <== osd.7 v2:172.21.15.119:6824/1341885104 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1d40073480 con 0x7f1d40070970 2023-08-11T15:53:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.597+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 --> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1d5c0cb270 con 0x7f1d40070970 2023-08-11T15:53:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.598+0000 7f1d51ffb640 1 -- 172.21.15.27:0/2834110533 <== osd.7 v2:172.21.15.119:6824/1341885104 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1d5400f040 con 0x7f1d40070970 2023-08-11T15:53:49.601 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-08-11T15:53:49.601 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-11T15:53:49.602 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-08-11T15:53:49.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] conn(0x7f1d40070970 msgr2=0x7f1d40072db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d62349640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:6824/1341885104,v1:172.21.15.119:6825/1341885104] conn(0x7f1d40070970 0x7f1d40072db0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d4006a050 msgr2=0x7f1d4006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d62349640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f1d4006a050 0x7f1d4006c510 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1d58036000 tx=0x7f1d58008030 comp rx=0 tx=0).stop 2023-08-11T15:53:49.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5c15c410 msgr2=0x7f1d5c1f28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d62349640 1 --2- 172.21.15.27:0/2834110533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5c15c410 0x7f1d5c1f28d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00c430 tx=0x7f1d4c00c8f0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d61b48640 1 -- 172.21.15.27:0/2834110533 reap_dead start 2023-08-11T15:53:49.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 shutdown_connections 2023-08-11T15:53:49.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 >> 172.21.15.27:0/2834110533 conn(0x7f1d5c0d00e0 msgr2=0x7f1d5c06ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:49.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 shutdown_connections 2023-08-11T15:53:49.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.599+0000 7f1d62349640 1 -- 172.21.15.27:0/2834110533 wait complete. 2023-08-11T15:53:49.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-08-11T15:53:49.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.853+0000 7f6fbe498640 1 Processor -- start 2023-08-11T15:53:49.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.853+0000 7f6fbe498640 1 -- start start 2023-08-11T15:53:49.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.854+0000 7f6fbe498640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6fb815c1f0 0x7f6fb80b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.854+0000 7f6fbe498640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6fb815ca30 0x7f6fb8160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.854+0000 7f6fbe498640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb80b5170 con 0x7f6fb815c5d0 2023-08-11T15:53:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.855+0000 7f6fbe498640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb815ce70 con 0x7f6fb815c1f0 2023-08-11T15:53:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.855+0000 7f6fbe498640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb815d080 con 0x7f6fb815ca30 2023-08-11T15:53:49.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.855+0000 7f6fbcc95640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6fb815c5d0 0x7f6fb80d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39788/0 (socket says 172.21.15.27:39788) 2023-08-11T15:53:49.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.855+0000 7f6fbcc95640 1 -- 172.21.15.27:0/3354678225 learned_addr learned my addr 172.21.15.27:0/3354678225 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:53:49.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.855+0000 7f6fbd496640 1 --2- 172.21.15.27:0/3354678225 >> v2:172.21.15.27:3300/0 conn(0x7f6fb815c1f0 0x7f6fb80b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.855+0000 7f6fbdc97640 1 --2- 172.21.15.27:0/3354678225 >> v2:172.21.15.119:3300/0 conn(0x7f6fb815ca30 0x7f6fb8160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.855+0000 7f6faffff640 1 -- 172.21.15.27:0/3354678225 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520916894 0 0) 0x7f6fb80b5170 con 0x7f6fb815c5d0 2023-08-11T15:53:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.856+0000 7f6faffff640 1 -- 172.21.15.27:0/3354678225 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa4003520 con 0x7f6fb815c5d0 2023-08-11T15:53:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.856+0000 7f6fbd496640 1 -- 172.21.15.27:0/3354678225 >> v2:172.21.15.119:3300/0 conn(0x7f6fb815ca30 msgr2=0x7f6fb8160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.856+0000 7f6fbd496640 1 --2- 172.21.15.27:0/3354678225 >> v2:172.21.15.119:3300/0 conn(0x7f6fb815ca30 0x7f6fb8160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.856+0000 7f6fbd496640 1 -- 172.21.15.27:0/3354678225 >> v1:172.21.15.27:6789/0 conn(0x7f6fb815c5d0 legacy=0x7f6fb80d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.856+0000 7f6fbd496640 1 -- 172.21.15.27:0/3354678225 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb815d360 con 0x7f6fb815c1f0 2023-08-11T15:53:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.856+0000 7f6fbdc97640 1 --2- 172.21.15.27:0/3354678225 >> v2:172.21.15.119:3300/0 conn(0x7f6fb815ca30 0x7f6fb8160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.857+0000 7f6fbd496640 1 --2- 172.21.15.27:0/3354678225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815c1f0 0x7f6fb80b7d40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80d03d0 tx=0x7f6fb402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf46e53e920b4dc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.857+0000 7f6faffff640 1 -- 172.21.15.27:0/3354678225 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb402cd10 con 0x7f6fb815c1f0 2023-08-11T15:53:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.857+0000 7f6faffff640 1 -- 172.21.15.27:0/3354678225 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb402ce70 con 0x7f6fb815c1f0 2023-08-11T15:53:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.857+0000 7f6faffff640 1 -- 172.21.15.27:0/3354678225 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4034640 con 0x7f6fb815c1f0 2023-08-11T15:53:49.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.858+0000 7f6fbe498640 1 -- 172.21.15.27:0/3354678225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815c1f0 msgr2=0x7f6fb80b7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.858+0000 7f6fbe498640 1 --2- 172.21.15.27:0/3354678225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815c1f0 0x7f6fb80b7d40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80d03d0 tx=0x7f6fb402bf60 comp rx=0 tx=0).stop 2023-08-11T15:53:49.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.858+0000 7f6fbe498640 1 -- 172.21.15.27:0/3354678225 shutdown_connections 2023-08-11T15:53:49.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.858+0000 7f6fbe498640 1 --2- 172.21.15.27:0/3354678225 >> v2:172.21.15.119:3300/0 conn(0x7f6fb815ca30 0x7f6fb8160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.858+0000 7f6fbe498640 1 --2- 172.21.15.27:0/3354678225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815c1f0 0x7f6fb80b7d40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.858+0000 7f6fbe498640 1 -- 172.21.15.27:0/3354678225 >> 172.21.15.27:0/3354678225 conn(0x7f6fb806a250 msgr2=0x7f6fb806a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:49.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.858+0000 7f6fbe498640 1 -- 172.21.15.27:0/3354678225 shutdown_connections 2023-08-11T15:53:49.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.858+0000 7f6fbe498640 1 -- 172.21.15.27:0/3354678225 wait complete. 2023-08-11T15:53:49.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.860+0000 7f6fbe498640 1 Processor -- start 2023-08-11T15:53:49.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.860+0000 7f6fbe498640 1 -- start start 2023-08-11T15:53:49.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.861+0000 7f6fbe498640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fb815c1f0 0x7f6fb814cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.861+0000 7f6fbe498640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb815c5d0 0x7f6fb81f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.861+0000 7f6fbe498640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815ca30 0x7f6fb81f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.861+0000 7f6fbe498640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80c6170 con 0x7f6fb815ca30 2023-08-11T15:53:49.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.861+0000 7f6fbe498640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb81589a0 con 0x7f6fb815c5d0 2023-08-11T15:53:49.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.861+0000 7f6fbe498640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80c8450 con 0x7f6fb815c1f0 2023-08-11T15:53:49.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.861+0000 7f6fbdc97640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815ca30 0x7f6fb81f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.861+0000 7f6fbdc97640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815ca30 0x7f6fb81f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35242/0 (socket says 172.21.15.27:35242) 2023-08-11T15:53:49.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.861+0000 7f6fbdc97640 1 -- 172.21.15.27:0/4235056839 learned_addr learned my addr 172.21.15.27:0/4235056839 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:49.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fbdc97640 1 -- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fb815c1f0 msgr2=0x7f6fb814cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fbdc97640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fb815c1f0 0x7f6fb814cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fbcc95640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb815c5d0 0x7f6fb81f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fbdc97640 1 -- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb815c5d0 msgr2=0x7f6fb81f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:49.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fbdc97640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb815c5d0 0x7f6fb81f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:49.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fbdc97640 1 -- 172.21.15.27:0/4235056839 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4009a70 con 0x7f6fb815ca30 2023-08-11T15:53:49.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fbdc97640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815ca30 0x7f6fb81f5770 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000d7d0 tx=0x7f6fb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:49.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fadffb640 1 -- 172.21.15.27:0/4235056839 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0004030 con 0x7f6fb815ca30 2023-08-11T15:53:49.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fadffb640 1 -- 172.21.15.27:0/4235056839 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb0004190 con 0x7f6fb815ca30 2023-08-11T15:53:49.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fadffb640 1 -- 172.21.15.27:0/4235056839 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0004500 con 0x7f6fb815ca30 2023-08-11T15:53:49.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.862+0000 7f6fbe498640 1 -- 172.21.15.27:0/4235056839 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb80c8730 con 0x7f6fb815ca30 2023-08-11T15:53:49.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.863+0000 7f6fbe498640 1 -- 172.21.15.27:0/4235056839 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb8158580 con 0x7f6fb815ca30 2023-08-11T15:53:49.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.864+0000 7f6fbe498640 1 -- 172.21.15.27:0/4235056839 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb80c6370 con 0x7f6fb815ca30 2023-08-11T15:53:49.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.865+0000 7f6fadffb640 1 -- 172.21.15.27:0/4235056839 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6fb00096a0 con 0x7f6fb815ca30 2023-08-11T15:53:49.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.866+0000 7f6fadffb640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f9c06a050 0x7f6f9c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:49.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.866+0000 7f6fadffb640 1 -- 172.21.15.27:0/4235056839 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6fb0088670 con 0x7f6fb815ca30 2023-08-11T15:53:49.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.867+0000 7f6fbd496640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f9c06a050 0x7f6f9c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:49.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.868+0000 7f6fbd496640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f9c06a050 0x7f6f9c06c510 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4036000 tx=0x7f6fb402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:49.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:49.870+0000 7f6fadffb640 1 -- 172.21.15.27:0/4235056839 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f6fb80c6370 con 0x7f6fb815ca30 2023-08-11T15:53:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.063+0000 7f6fbe498640 1 -- 172.21.15.27:0/4235056839 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fb815fff0 con 0x7f6f9c06a050 2023-08-11T15:53:50.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.065+0000 7f6fadffb640 1 -- 172.21.15.27:0/4235056839 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6fb4003990 con 0x7f6f9c06a050 2023-08-11T15:53:50.068 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 7 to deep-scrub 2023-08-11T15:53:50.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 -- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f9c06a050 msgr2=0x7f6f9c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:50.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f9c06a050 0x7f6f9c06c510 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4036000 tx=0x7f6fb402e000 comp rx=0 tx=0).stop 2023-08-11T15:53:50.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 -- 172.21.15.27:0/4235056839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815ca30 msgr2=0x7f6fb81f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:50.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815ca30 0x7f6fb81f5770 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000d7d0 tx=0x7f6fb000dc90 comp rx=0 tx=0).stop 2023-08-11T15:53:50.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 -- 172.21.15.27:0/4235056839 shutdown_connections 2023-08-11T15:53:50.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f6f9c06a050 0x7f6f9c06c510 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fb815ca30 0x7f6fb81f5770 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb815c5d0 0x7f6fb81f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 --2- 172.21.15.27:0/4235056839 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fb815c1f0 0x7f6fb814cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 -- 172.21.15.27:0/4235056839 >> 172.21.15.27:0/4235056839 conn(0x7f6fb806a250 msgr2=0x7f6fb80b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.067+0000 7f6fbe498640 1 -- 172.21.15.27:0/4235056839 shutdown_connections 2023-08-11T15:53:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.068+0000 7f6fbe498640 1 -- 172.21.15.27:0/4235056839 wait complete. 2023-08-11T15:53:50.090 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:53:50.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.316+0000 7f5257fff640 1 Processor -- start 2023-08-11T15:53:50.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.316+0000 7f5257fff640 1 -- start start 2023-08-11T15:53:50.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.318+0000 7f5257fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f52580cbb70 0x7f525806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.318+0000 7f5257fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52581584e0 0x7f52580caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.318+0000 7f5257fff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525806c9f0 con 0x7f5258158930 2023-08-11T15:53:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.318+0000 7f5257fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52580cd8c0 con 0x7f52581584e0 2023-08-11T15:53:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.318+0000 7f5257fff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52580cc110 con 0x7f52580cbb70 2023-08-11T15:53:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.318+0000 7f524ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52581584e0 0x7f52580caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.318+0000 7f524ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52581584e0 0x7f52580caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35246/0 (socket says 172.21.15.27:35246) 2023-08-11T15:53:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.318+0000 7f524ffff640 1 -- 172.21.15.27:0/3112290078 learned_addr learned my addr 172.21.15.27:0/3112290078 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.319+0000 7f5256ffd640 1 --2- 172.21.15.27:0/3112290078 >> v2:172.21.15.119:3300/0 conn(0x7f52580cbb70 0x7f525806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.319+0000 7f524ffff640 1 -- 172.21.15.27:0/3112290078 >> v2:172.21.15.119:3300/0 conn(0x7f52580cbb70 msgr2=0x7f525806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.319+0000 7f524ffff640 1 --2- 172.21.15.27:0/3112290078 >> v2:172.21.15.119:3300/0 conn(0x7f52580cbb70 0x7f525806c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.319+0000 7f524ffff640 1 -- 172.21.15.27:0/3112290078 >> v1:172.21.15.119:6789/0 conn(0x7f5258158930 legacy=0x7f52580cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.319+0000 7f524ffff640 1 -- 172.21.15.27:0/3112290078 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580cc3f0 con 0x7f52581584e0 2023-08-11T15:53:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.320+0000 7f524ffff640 1 --2- 172.21.15.27:0/3112290078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 0x7f52580caa20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f525806dae0 tx=0x7f524002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84e594bbe5b32bf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.320+0000 7f52567fc640 1 -- 172.21.15.27:0/3112290078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240031870 con 0x7f52581584e0 2023-08-11T15:53:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.321+0000 7f52567fc640 1 -- 172.21.15.27:0/3112290078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52400319d0 con 0x7f52581584e0 2023-08-11T15:53:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.321+0000 7f52567fc640 1 -- 172.21.15.27:0/3112290078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240031d60 con 0x7f52581584e0 2023-08-11T15:53:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.321+0000 7f5257fff640 1 -- 172.21.15.27:0/3112290078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 msgr2=0x7f52580caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.321+0000 7f5257fff640 1 --2- 172.21.15.27:0/3112290078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 0x7f52580caa20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f525806dae0 tx=0x7f524002b910 comp rx=0 tx=0).stop 2023-08-11T15:53:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.321+0000 7f5257fff640 1 -- 172.21.15.27:0/3112290078 shutdown_connections 2023-08-11T15:53:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.321+0000 7f5257fff640 1 --2- 172.21.15.27:0/3112290078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 0x7f52580caa20 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.321+0000 7f5257fff640 1 --2- 172.21.15.27:0/3112290078 >> v2:172.21.15.119:3300/0 conn(0x7f52580cbb70 0x7f525806c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.321+0000 7f5257fff640 1 -- 172.21.15.27:0/3112290078 >> 172.21.15.27:0/3112290078 conn(0x7f52580afb70 msgr2=0x7f52580aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.321+0000 7f5257fff640 1 -- 172.21.15.27:0/3112290078 shutdown_connections 2023-08-11T15:53:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.321+0000 7f5257fff640 1 -- 172.21.15.27:0/3112290078 wait complete. 2023-08-11T15:53:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.323+0000 7f5257fff640 1 Processor -- start 2023-08-11T15:53:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.323+0000 7f5257fff640 1 -- start start 2023-08-11T15:53:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.324+0000 7f5257fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52580cbb70 0x7f52580ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.324+0000 7f5257fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 0x7f52580ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.324+0000 7f5257fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5258158930 0x7f525814cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.324+0000 7f5257fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580c0f80 con 0x7f52581584e0 2023-08-11T15:53:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.324+0000 7f5257fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52581359d0 con 0x7f5258158930 2023-08-11T15:53:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.324+0000 7f5257fff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525814f790 con 0x7f52580cbb70 2023-08-11T15:53:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.324+0000 7f524ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 0x7f52580ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:50.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.324+0000 7f524ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 0x7f52580ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35260/0 (socket says 172.21.15.27:35260) 2023-08-11T15:53:50.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.324+0000 7f524ffff640 1 -- 172.21.15.27:0/2328583585 learned_addr learned my addr 172.21.15.27:0/2328583585 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:53:50.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.325+0000 7f524ffff640 1 -- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52580cbb70 msgr2=0x7f52580ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:50.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.325+0000 7f524ffff640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52580cbb70 0x7f52580ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.325+0000 7f524ffff640 1 -- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5258158930 msgr2=0x7f525814cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:50.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.325+0000 7f524ffff640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5258158930 0x7f525814cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.325+0000 7f524ffff640 1 -- 172.21.15.27:0/2328583585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52400094d0 con 0x7f52581584e0 2023-08-11T15:53:50.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.325+0000 7f524ffff640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 0x7f52580ace70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5240000c00 tx=0x7f52400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:50.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.326+0000 7f5254ff9640 1 -- 172.21.15.27:0/2328583585 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524003abe0 con 0x7f52581584e0 2023-08-11T15:53:50.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.326+0000 7f5254ff9640 1 -- 172.21.15.27:0/2328583585 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524003ad40 con 0x7f52581584e0 2023-08-11T15:53:50.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.326+0000 7f5257fff640 1 -- 172.21.15.27:0/2328583585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52580c4220 con 0x7f52581584e0 2023-08-11T15:53:50.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.326+0000 7f5254ff9640 1 -- 172.21.15.27:0/2328583585 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240044610 con 0x7f52581584e0 2023-08-11T15:53:50.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.326+0000 7f5257fff640 1 -- 172.21.15.27:0/2328583585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5258135f00 con 0x7f52581584e0 2023-08-11T15:53:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.327+0000 7f5257fff640 1 -- 172.21.15.27:0/2328583585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525804e420 con 0x7f52581584e0 2023-08-11T15:53:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.328+0000 7f5254ff9640 1 -- 172.21.15.27:0/2328583585 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f524004d450 con 0x7f52581584e0 2023-08-11T15:53:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.329+0000 7f5254ff9640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f523806a050 0x7f523806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.329+0000 7f5254ff9640 1 -- 172.21.15.27:0/2328583585 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f524002d040 con 0x7f52581584e0 2023-08-11T15:53:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.330+0000 7f5256ffd640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f523806a050 0x7f523806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.331+0000 7f5256ffd640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f523806a050 0x7f523806c510 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f52480098f0 tx=0x7f5248008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.333+0000 7f5254ff9640 1 -- 172.21.15.27:0/2328583585 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f5240076140 con 0x7f52581584e0 2023-08-11T15:53:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.523+0000 7f5257fff640 1 -- 172.21.15.27:0/2328583585 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52580cc9b0 con 0x7f523806a050 2023-08-11T15:53:50.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.531+0000 7f5254ff9640 1 -- 172.21.15.27:0/2328583585 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70495 (secure 0 0 0) 0x7f5248009e70 con 0x7f523806a050 2023-08-11T15:53:50.535 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:53:50.537 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:53:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 -- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f523806a050 msgr2=0x7f523806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f523806a050 0x7f523806c510 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f52480098f0 tx=0x7f5248008040 comp rx=0 tx=0).stop 2023-08-11T15:53:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 -- 172.21.15.27:0/2328583585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 msgr2=0x7f52580ace70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 0x7f52580ace70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5240000c00 tx=0x7f52400046c0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 -- 172.21.15.27:0/2328583585 shutdown_connections 2023-08-11T15:53:50.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f523806a050 0x7f523806c510 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5258158930 0x7f525814cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52581584e0 0x7f52580ace70 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 --2- 172.21.15.27:0/2328583585 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52580cbb70 0x7f52580ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:50.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 -- 172.21.15.27:0/2328583585 >> 172.21.15.27:0/2328583585 conn(0x7f52580afb70 msgr2=0x7f5258153410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:50.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 -- 172.21.15.27:0/2328583585 shutdown_connections 2023-08-11T15:53:50.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:53:50.534+0000 7f5257fff640 1 -- 172.21.15.27:0/2328583585 wait complete. 2023-08-11T15:53:50.557 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2023-08-11T15:53:48.774464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":246124,"kb_used_data":5384,"kb_used_omap":13,"kb_used_meta":240626,"kb_avail":754728596,"statfs":{"total":773094113280,"available":772842082304,"internally_reserved":0,"allocated":5513216,"data_stored":3114648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":246401112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.978729"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:37.809548+0000","last_change":"2023-08-11T15:50:54.667601+0000","last_active":"2023-08-11T15:53:37.809548+0000","last_peered":"2023-08-11T15:53:37.809548+0000","last_clean":"2023-08-11T15:53:37.809548+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:53:37.809548+0000","last_undegraded":"2023-08-11T15:53:37.809548+0000","last_fullsized":"2023-08-11T15:53:37.809548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:37:19.658271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:39.788062+0000","last_change":"2023-08-11T15:50:54.662968+0000","last_active":"2023-08-11T15:53:39.788062+0000","last_peered":"2023-08-11T15:53:39.788062+0000","last_clean":"2023-08-11T15:53:39.788062+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:53:39.788062+0000","last_undegraded":"2023-08-11T15:53:39.788062+0000","last_fullsized":"2023-08-11T15:53:39.788062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:39.787856+0000","last_change":"2023-08-11T15:50:55.272783+0000","last_active":"2023-08-11T15:53:39.787856+0000","last_peered":"2023-08-11T15:53:39.787856+0000","last_clean":"2023-08-11T15:53:39.787856+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:53:39.787856+0000","last_undegraded":"2023-08-11T15:53:39.787856+0000","last_fullsized":"2023-08-11T15:53:39.787856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:22:54.752595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:39.786378+0000","last_change":"2023-08-11T15:50:55.270974+0000","last_active":"2023-08-11T15:53:39.786378+0000","last_peered":"2023-08-11T15:53:39.786378+0000","last_clean":"2023-08-11T15:53:39.786378+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:53:39.786378+0000","last_undegraded":"2023-08-11T15:53:39.786378+0000","last_fullsized":"2023-08-11T15:53:39.786378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:38.771558+0000","last_change":"2023-08-11T15:50:54.660862+0000","last_active":"2023-08-11T15:53:38.771558+0000","last_peered":"2023-08-11T15:53:38.771558+0000","last_clean":"2023-08-11T15:53:38.771558+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:53:38.771558+0000","last_undegraded":"2023-08-11T15:53:38.771558+0000","last_fullsized":"2023-08-11T15:53:38.771558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:16:41.162954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:46.515296+0000","last_change":"2023-08-11T15:53:46.515356+0000","last_active":"2023-08-11T15:53:46.515296+0000","last_peered":"2023-08-11T15:53:46.515296+0000","last_clean":"2023-08-11T15:53:46.515296+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:53:46.515296+0000","last_undegraded":"2023-08-11T15:53:46.515296+0000","last_fullsized":"2023-08-11T15:53:46.515296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:46.515201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:46.515201+0000","last_clean_scrub_stamp":"2023-08-11T15:53:46.515201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:15:01.037710+0000","scrub_duration":0.0024113239999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:39.789077+0000","last_change":"2023-08-11T15:50:54.669675+0000","last_active":"2023-08-11T15:53:39.789077+0000","last_peered":"2023-08-11T15:53:39.789077+0000","last_clean":"2023-08-11T15:53:39.789077+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:53:39.789077+0000","last_undegraded":"2023-08-11T15:53:39.789077+0000","last_fullsized":"2023-08-11T15:53:39.789077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:16:41.162954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:38.771332+0000","last_change":"2023-08-11T15:50:54.663730+0000","last_active":"2023-08-11T15:53:38.771332+0000","last_peered":"2023-08-11T15:53:38.771332+0000","last_clean":"2023-08-11T15:53:38.771332+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:53:38.771332+0000","last_undegraded":"2023-08-11T15:53:38.771332+0000","last_fullsized":"2023-08-11T15:53:38.771332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:53.631279+0000","last_clean_scrub_stamp":"2023-08-11T15:50:53.631279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:22:54.752595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:39.789280+0000","last_change":"2023-08-11T15:50:48.426393+0000","last_active":"2023-08-11T15:53:39.789280+0000","last_peered":"2023-08-11T15:53:39.789280+0000","last_clean":"2023-08-11T15:53:39.789280+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:53:39.789280+0000","last_undegraded":"2023-08-11T15:53:39.789280+0000","last_fullsized":"2023-08-11T15:53:39.789280+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:50:47.133560+0000","last_clean_scrub_stamp":"2023-08-11T15:50:47.133560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:07:51.695008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1134592,"data_stored":832107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.093,"15min":1.1160000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.216,"15min":1.222},"min":{"1min":0.80900000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5509999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.109}]},{"osd":1,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3129999999999999,"15min":1.327},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.7370000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.524}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.177,"15min":1.175},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.232,"5min":1.254,"15min":1.258},"min":{"1min":0.87,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5169999999999999,"5min":2.02,"15min":2.02},"last":1.1890000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2909999999999999,"15min":1.3069999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.671,"5min":1.869,"15min":1.869},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4890000000000001,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.4870000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4219999999999999,"15min":1.4279999999999999},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.885,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.419},{"interface":"front","average":{"1min":1.282,"5min":1.395,"15min":1.4139999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6910000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.2569999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.373,"15min":1.381},"min":{"1min":0.93500000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5449999999999999,"15min":1.5509999999999999},"min":{"1min":1.2170000000000001,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":1.9330000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.6299999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.5029999999999999,"15min":1.5209999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.768,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.544,"15min":1.5580000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.861,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.6080000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0609999999999999,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.625},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1339999999999999,"15min":1.1359999999999999},"min":{"1min":0.751,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.115,"15min":1.1140000000000001},"min":{"1min":0.73599999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.591,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.296},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.66100000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.3340000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2390000000000001,"15min":1.24},"min":{"1min":0.85599999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6020000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.3180000000000001,"15min":1.3400000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5600000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.1040000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.369,"15min":1.377},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.77,"5min":2.774,"15min":2.774},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7330000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.379}]},{"osd":4,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.319,"15min":1.327},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8879999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":1.008},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.468,"15min":1.4910000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.665,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.5169999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.496,"15min":1.5069999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8149999999999999,"5min":3.371,"15min":3.371},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.486,"15min":1.504},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0009999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.155}]},{"osd":7,"last update":"Fri Aug 11 15:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.615,"15min":1.6479999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.762,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.6739999999999999,"15min":1.704},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.921,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.482}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0389999999999999,"15min":1.0269999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.073,"5min":1.1499999999999999,"15min":1.163},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2370000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1419999999999999,"15min":1.1399999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.93999999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.621,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.046}]},{"osd":2,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.262,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.615,"5min":1.827,"15min":1.827},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.194,"5min":1.351,"15min":1.377},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8300000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.3680000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.353,"15min":1.353},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.702,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.607},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.4159999999999999,"15min":1.444},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.782,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.01}]},{"osd":4,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4379999999999999,"15min":1.466},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8779999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5880000000000001,"15min":1.6080000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.847,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":1.746}]},{"osd":6,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.4690000000000001,"15min":1.492},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.796,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":1.131},{"interface":"front","average":{"1min":1.456,"5min":1.6120000000000001,"15min":1.6379999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.2160000000000002,"5min":3.9870000000000001,"15min":3.9870000000000001},"last":1.4890000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:51 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.5820000000000001,"15min":1.605},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9990000000000001,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":1.8420000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.637,"15min":1.677},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.855,"5min":3.367,"15min":3.367},"last":1.5880000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0669999999999999,"15min":1.0609999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5700000000000001,"5min":1.776,"15min":1.776},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.05,"5min":1.115,"15min":1.1259999999999999},"min":{"1min":0.64900000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.4930000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.90000000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1200000000000001,"15min":1.129},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5089999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1879999999999999,"15min":1.2010000000000001},"min":{"1min":0.73199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6659999999999999,"5min":1.756,"15min":1.756},"last":0.94899999999999995}]},{"osd":2,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.26,"15min":1.2769999999999999},"min":{"1min":0.89700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3720000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.153},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2310000000000001,"15min":1.2230000000000001},"min":{"1min":0.88900000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.3959999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.82199999999999995,"5min":0.75,"15min":0.75},"max":{"1min":1.524,"5min":1.736,"15min":1.736},"last":1.206},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.357,"15min":1.3640000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.738,"5min":2.121,"15min":2.121},"last":1.046}]},{"osd":5,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2929999999999999,"15min":1.2949999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8759999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.506},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.746,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6419999999999999,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.3360000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3819999999999999,"15min":1.3839999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.825,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.9450000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.704}]},{"osd":7,"last update":"Fri Aug 11 15:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.405,"15min":1.415},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.617,"5min":2.286,"15min":2.286},"last":1.577},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.577,"15min":1.5940000000000001},"min":{"1min":1.008,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.9139999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.6259999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0589999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97999999999999998},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.478}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0249999999999999,"15min":1.0449999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.458,"5min":1.78,"15min":1.78},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.97599999999999998,"15min":0.99199999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.24,"5min":1.623,"15min":1.623},"last":1.288}]},{"osd":2,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.55,"5min":3.7799999999999998,"15min":3.7799999999999998},"last":1.026},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.202,"15min":1.2290000000000001},"min":{"1min":0.628,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5920000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.95499999999999996}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3620000000000001,"15min":1.3799999999999999},"min":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.849},{"interface":"front","average":{"1min":1.286,"5min":1.369,"15min":1.383},"min":{"1min":0.88500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.7190000000000001,"5min":2.573,"15min":2.573},"last":1.6699999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.5629999999999999,"15min":1.6000000000000001},"min":{"1min":0.99399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":2.0630000000000002},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.514,"15min":1.5700000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6299999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":2.0209999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.534,"15min":1.5580000000000001},"min":{"1min":1.0569999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.873,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.9770000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.6060000000000001,"15min":1.6459999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.758,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":2.1219999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.6499999999999999,"15min":1.6899999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.825,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":2.1699999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.6379999999999999,"15min":1.6879999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.738,"5min":3.4340000000000002,"15min":3.4340000000000002},"last":1.911}]}]},{"osd":2,"up_from":14,"seq":60129542183,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89900000000000002,"15min":0.90300000000000002},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3009999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.024,"15min":1.036},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.405,"5min":1.865,"15min":1.865},"last":0.49099999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.383,"5min":1.468,"15min":1.468},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.079,"15min":1.091},"min":{"1min":0.64900000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4570000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.2},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.879,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.1890000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4139999999999999,"15min":1.4179999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.8759999999999999,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.93100000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.847,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.1299999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.508,"15min":1.5229999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.9730000000000001,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.304,"5min":1.4399999999999999,"15min":1.462},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6990000000000001,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.2330000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7829999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.472,"5min":1.5069999999999999,"15min":1.5129999999999999},"min":{"1min":0.97799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.0289999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.4199999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:52:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.6060000000000001,"15min":1.6279999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9279999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.028},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.593,"15min":1.6140000000000001},"min":{"1min":1.054,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.907,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":1.093}]}]},{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.575,"15min":1.575},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0089999999999999,"15min":1.0149999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.81399999999999995}]},{"osd":2,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.026,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.782,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.0569999999999999,"15min":1.0609999999999999},"min":{"1min":0.64000000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.377,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.622}]},{"osd":3,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.282,"15min":1.292},"min":{"1min":0.70999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6200000000000001,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.3,"5min":1.2709999999999999,"15min":1.266},"min":{"1min":0.62,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.962,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.411,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.7410000000000001,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4199999999999999,"15min":1.4359999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.994,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":1.0669999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1059999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.123},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5009999999999999,"15min":1.51},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.8700000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.90400000000000003}]},{"osd":6,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.524,"15min":1.518},"min":{"1min":1.0529999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.3500000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.876},{"interface":"front","average":{"1min":1.405,"5min":1.458,"15min":1.4670000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.919,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.83099999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.5209999999999999,"15min":1.524},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.0449999999999999,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.621,"5min":1.617,"15min":1.617},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":2.4529999999999998,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":1.04}]}]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1134592,"data_stored":832107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.83799999999999997},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.109,"5min":1.288,"15min":1.288},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":0.92100000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0249999999999999,"15min":1.0129999999999999},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.90600000000000003}]},{"osd":3,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.51200000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3080000000000001,"5min":1.522,"15min":1.522},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.137,"15min":1.149},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6879999999999999,"5min":1.728,"15min":1.728},"last":0.97699999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.323,"15min":1.3360000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.724,"5min":2.23,"15min":2.23},"last":1.097},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2869999999999999,"15min":1.298},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.397}]},{"osd":5,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.333,"15min":1.337},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.889,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.4330000000000001,"15min":1.4490000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.7549999999999999,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":1.478}]},{"osd":6,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.3460000000000001,"15min":1.3700000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.677,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.278},{"interface":"front","average":{"1min":1.347,"5min":1.413,"15min":1.4239999999999999},"min":{"1min":0.877,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.4410000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4990000000000001,"15min":1.5069999999999999},"min":{"1min":0.86199999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.99,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.361},{"interface":"front","average":{"1min":1.337,"5min":1.468,"15min":1.49},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.7729999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.056}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:53:50.564 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-08-11T15:50:53.631279+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=50, tm_sec=53, tm_wday=4, tm_yday=223, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=53, tm_sec=42, tm_wday=4, tm_yday=223, tm_isdst=0) 2023-08-11T15:53:50.565 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-08-11T15:50:53.631279+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=50, tm_sec=53, tm_wday=4, tm_yday=223, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=53, tm_sec=42, tm_wday=4, tm_yday=223, tm_isdst=0) 2023-08-11T15:53:50.566 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-08-11T15:50:53.631279+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=50, tm_sec=53, tm_wday=4, tm_yday=223, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=53, tm_sec=42, tm_wday=4, tm_yday=223, tm_isdst=0) 2023-08-11T15:53:50.566 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-08-11T15:50:53.631279+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=50, tm_sec=53, tm_wday=4, tm_yday=223, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=53, tm_sec=42, tm_wday=4, tm_yday=223, tm_isdst=0) 2023-08-11T15:53:50.567 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-08-11T15:50:53.631279+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=50, tm_sec=53, tm_wday=4, tm_yday=223, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=53, tm_sec=42, tm_wday=4, tm_yday=223, tm_isdst=0) 2023-08-11T15:53:50.567 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-08-11T15:50:53.631279+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=50, tm_sec=53, tm_wday=4, tm_yday=223, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=53, tm_sec=42, tm_wday=4, tm_yday=223, tm_isdst=0) 2023-08-11T15:53:50.568 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-08-11T15:50:53.631279+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=50, tm_sec=53, tm_wday=4, tm_yday=223, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=53, tm_sec=42, tm_wday=4, tm_yday=223, tm_isdst=0) 2023-08-11T15:53:50.569 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-08-11T15:50:47.133560+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=50, tm_sec=47, tm_wday=4, tm_yday=223, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=11, tm_hour=15, tm_min=53, tm_sec=42, tm_wday=4, tm_yday=223, tm_isdst=0) 2023-08-11T15:53:50.569 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-08-11T15:54:10.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:54:10.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.817+0000 7fa1664fa640 1 Processor -- start 2023-08-11T15:54:10.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.817+0000 7fa1664fa640 1 -- start start 2023-08-11T15:54:10.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.818+0000 7fa1664fa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa1600b55c0 0x7fa1600cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:10.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.818+0000 7fa1664fa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa1600cf930 0x7fa1600d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:10.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.818+0000 7fa1664fa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1600cda30 con 0x7fa16015c620 2023-08-11T15:54:10.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.818+0000 7fa1664fa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1600ccc10 con 0x7fa1600cf930 2023-08-11T15:54:10.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.818+0000 7fa1664fa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600cfda0 con 0x7fa1600b55c0 2023-08-11T15:54:10.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.819+0000 7fa15ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa16015c620 0x7fa1600b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50640/0 (socket says 172.21.15.27:50640) 2023-08-11T15:54:10.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.819+0000 7fa15ffff640 1 -- 172.21.15.27:0/922751305 learned_addr learned my addr 172.21.15.27:0/922751305 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:54:10.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.819+0000 7fa164a97640 1 --2- 172.21.15.27:0/922751305 >> v2:172.21.15.27:3300/0 conn(0x7fa1600cf930 0x7fa1600d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:10.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.819+0000 7fa15f7fe640 1 --2- 172.21.15.27:0/922751305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa1600b55c0 0x7fa1600cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:10.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.819+0000 7fa15effd640 1 -- 172.21.15.27:0/922751305 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250245482 0 0) 0x7fa1600cda30 con 0x7fa16015c620 2023-08-11T15:54:10.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.819+0000 7fa15effd640 1 -- 172.21.15.27:0/922751305 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa14c003520 con 0x7fa16015c620 2023-08-11T15:54:10.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.820+0000 7fa164a97640 1 -- 172.21.15.27:0/922751305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa1600b55c0 msgr2=0x7fa1600cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:10.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.820+0000 7fa164a97640 1 --2- 172.21.15.27:0/922751305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa1600b55c0 0x7fa1600cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:10.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.820+0000 7fa164a97640 1 -- 172.21.15.27:0/922751305 >> v1:172.21.15.27:6789/0 conn(0x7fa16015c620 legacy=0x7fa1600b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:10.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.820+0000 7fa164a97640 1 -- 172.21.15.27:0/922751305 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160161110 con 0x7fa1600cf930 2023-08-11T15:54:10.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.820+0000 7fa15f7fe640 1 --2- 172.21.15.27:0/922751305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa1600b55c0 0x7fa1600cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:54:10.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.821+0000 7fa164a97640 1 --2- 172.21.15.27:0/922751305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600cf930 0x7fa1600d0b00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa1580097a0 tx=0x7fa15802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83802a1e98d79a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:10.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.821+0000 7fa15effd640 1 -- 172.21.15.27:0/922751305 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15802d040 con 0x7fa1600cf930 2023-08-11T15:54:10.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.821+0000 7fa15effd640 1 -- 172.21.15.27:0/922751305 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158031810 con 0x7fa1600cf930 2023-08-11T15:54:10.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.821+0000 7fa15effd640 1 -- 172.21.15.27:0/922751305 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158031b80 con 0x7fa1600cf930 2023-08-11T15:54:10.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.822+0000 7fa1664fa640 1 -- 172.21.15.27:0/922751305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600cf930 msgr2=0x7fa1600d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:10.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.822+0000 7fa1664fa640 1 --2- 172.21.15.27:0/922751305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600cf930 0x7fa1600d0b00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa1580097a0 tx=0x7fa15802b8b0 comp rx=0 tx=0).stop 2023-08-11T15:54:10.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.822+0000 7fa1664fa640 1 -- 172.21.15.27:0/922751305 shutdown_connections 2023-08-11T15:54:10.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.822+0000 7fa1664fa640 1 --2- 172.21.15.27:0/922751305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600cf930 0x7fa1600d0b00 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:10.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.822+0000 7fa1664fa640 1 --2- 172.21.15.27:0/922751305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa1600b55c0 0x7fa1600cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:10.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.822+0000 7fa1664fa640 1 -- 172.21.15.27:0/922751305 >> 172.21.15.27:0/922751305 conn(0x7fa160071110 msgr2=0x7fa1600714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.822+0000 7fa1664fa640 1 -- 172.21.15.27:0/922751305 shutdown_connections 2023-08-11T15:54:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.822+0000 7fa1664fa640 1 -- 172.21.15.27:0/922751305 wait complete. 2023-08-11T15:54:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.824+0000 7fa1664fa640 1 Processor -- start 2023-08-11T15:54:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.824+0000 7fa1664fa640 1 -- start start 2023-08-11T15:54:10.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.825+0000 7fa1664fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600b55c0 0x7fa16015f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:10.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.825+0000 7fa1664fa640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa1600cf930 0x7fa160155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:10.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.825+0000 7fa1664fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa16015c620 0x7fa1601414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.825+0000 7fa1664fa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa160156bd0 con 0x7fa1600b55c0 2023-08-11T15:54:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.825+0000 7fa1664fa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa160155fc0 con 0x7fa16015c620 2023-08-11T15:54:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.825+0000 7fa1664fa640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1601562a0 con 0x7fa1600cf930 2023-08-11T15:54:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.825+0000 7fa15ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600b55c0 0x7fa16015f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.825+0000 7fa15ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600b55c0 0x7fa16015f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41938/0 (socket says 172.21.15.27:41938) 2023-08-11T15:54:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.825+0000 7fa15ffff640 1 -- 172.21.15.27:0/2882751483 learned_addr learned my addr 172.21.15.27:0/2882751483 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:54:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.826+0000 7fa15ffff640 1 -- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa1600cf930 msgr2=0x7fa160155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.826+0000 7fa15ffff640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa1600cf930 0x7fa160155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.826+0000 7fa15ffff640 1 -- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa16015c620 msgr2=0x7fa1601414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:10.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.826+0000 7fa15ffff640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa16015c620 0x7fa1601414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:10.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.826+0000 7fa15ffff640 1 -- 172.21.15.27:0/2882751483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa158009400 con 0x7fa1600b55c0 2023-08-11T15:54:10.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.826+0000 7fa15ffff640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600b55c0 0x7fa16015f3f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa15000caa0 tx=0x7fa15000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:10.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.826+0000 7fa15cff9640 1 -- 172.21.15.27:0/2882751483 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15000ac00 con 0x7fa1600b55c0 2023-08-11T15:54:10.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.826+0000 7fa15cff9640 1 -- 172.21.15.27:0/2882751483 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15000ad60 con 0x7fa1600b55c0 2023-08-11T15:54:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.826+0000 7fa15cff9640 1 -- 172.21.15.27:0/2882751483 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150012620 con 0x7fa1600b55c0 2023-08-11T15:54:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.826+0000 7fa1664fa640 1 -- 172.21.15.27:0/2882751483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa160156580 con 0x7fa1600b55c0 2023-08-11T15:54:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.827+0000 7fa1664fa640 1 -- 172.21.15.27:0/2882751483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa160157060 con 0x7fa1600b55c0 2023-08-11T15:54:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.828+0000 7fa1664fa640 1 -- 172.21.15.27:0/2882751483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa160073990 con 0x7fa1600b55c0 2023-08-11T15:54:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.829+0000 7fa15cff9640 1 -- 172.21.15.27:0/2882751483 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa150007070 con 0x7fa1600b55c0 2023-08-11T15:54:10.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.830+0000 7fa15cff9640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa14006a050 0x7fa14006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.830+0000 7fa15cff9640 1 -- 172.21.15.27:0/2882751483 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa150088450 con 0x7fa1600b55c0 2023-08-11T15:54:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.830+0000 7fa15f7fe640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa14006a050 0x7fa14006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.832+0000 7fa15f7fe640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa14006a050 0x7fa14006c510 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa148006f00 tx=0x7fa148008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:10.834+0000 7fa15cff9640 1 -- 172.21.15.27:0/2882751483 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fa150053c80 con 0x7fa1600b55c0 2023-08-11T15:54:11.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.022+0000 7fa1664fa640 1 -- 172.21.15.27:0/2882751483 --> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1600cd100 con 0x7fa14006a050 2023-08-11T15:54:11.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.030+0000 7fa15cff9640 1 -- 172.21.15.27:0/2882751483 <== mgr.4106 v2:172.21.15.119:6832/792514541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70371 (secure 0 0 0) 0x7fa148009d80 con 0x7fa14006a050 2023-08-11T15:54:11.032 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:54:11.033 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-08-11T15:54:11.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 -- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa14006a050 msgr2=0x7fa14006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa14006a050 0x7fa14006c510 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa148006f00 tx=0x7fa148008040 comp rx=0 tx=0).stop 2023-08-11T15:54:11.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 -- 172.21.15.27:0/2882751483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600b55c0 msgr2=0x7fa16015f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600b55c0 0x7fa16015f3f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa15000caa0 tx=0x7fa15000cf60 comp rx=0 tx=0).stop 2023-08-11T15:54:11.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 -- 172.21.15.27:0/2882751483 shutdown_connections 2023-08-11T15:54:11.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fa14006a050 0x7fa14006c510 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa1600cf930 0x7fa160155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa16015c620 0x7fa1601414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 --2- 172.21.15.27:0/2882751483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1600b55c0 0x7fa16015f3f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 -- 172.21.15.27:0/2882751483 >> 172.21.15.27:0/2882751483 conn(0x7fa160071110 msgr2=0x7fa16015a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:11.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.032+0000 7fa1664fa640 1 -- 172.21.15.27:0/2882751483 shutdown_connections 2023-08-11T15:54:11.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.033+0000 7fa1664fa640 1 -- 172.21.15.27:0/2882751483 wait complete. 2023-08-11T15:54:11.052 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2023-08-11T15:54:10.781773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":246096,"kb_used_data":5384,"kb_used_omap":13,"kb_used_meta":240626,"kb_avail":754728624,"statfs":{"total":773094113280,"available":772842110976,"internally_reserved":0,"allocated":5513216,"data_stored":3114648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":246401112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004338"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:53.524993+0000","last_change":"2023-08-11T15:53:53.525124+0000","last_active":"2023-08-11T15:53:53.524993+0000","last_peered":"2023-08-11T15:53:53.524993+0000","last_clean":"2023-08-11T15:53:53.524993+0000","last_became_active":"2023-08-11T15:50:54.666688+0000","last_became_peered":"2023-08-11T15:50:54.666688+0000","last_unstale":"2023-08-11T15:53:53.524993+0000","last_undegraded":"2023-08-11T15:53:53.524993+0000","last_fullsized":"2023-08-11T15:53:53.524993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:53.524828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:53.524828+0000","last_clean_scrub_stamp":"2023-08-11T15:53:53.524828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:50:55.910141+0000","scrub_duration":0.0033363749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:52.473500+0000","last_change":"2023-08-11T15:53:52.473711+0000","last_active":"2023-08-11T15:53:52.473500+0000","last_peered":"2023-08-11T15:53:52.473500+0000","last_clean":"2023-08-11T15:53:52.473500+0000","last_became_active":"2023-08-11T15:50:54.662256+0000","last_became_peered":"2023-08-11T15:50:54.662256+0000","last_unstale":"2023-08-11T15:53:52.473500+0000","last_undegraded":"2023-08-11T15:53:52.473500+0000","last_fullsized":"2023-08-11T15:53:52.473500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:52.473323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:52.473323+0000","last_clean_scrub_stamp":"2023-08-11T15:53:52.473323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:42:30.673744+0000","scrub_duration":0.004653829,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:54.036114+0000","last_change":"2023-08-11T15:53:54.036217+0000","last_active":"2023-08-11T15:53:54.036114+0000","last_peered":"2023-08-11T15:53:54.036114+0000","last_clean":"2023-08-11T15:53:54.036114+0000","last_became_active":"2023-08-11T15:50:55.271998+0000","last_became_peered":"2023-08-11T15:50:55.271998+0000","last_unstale":"2023-08-11T15:53:54.036114+0000","last_undegraded":"2023-08-11T15:53:54.036114+0000","last_fullsized":"2023-08-11T15:53:54.036114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:54.035954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:54.035954+0000","last_clean_scrub_stamp":"2023-08-11T15:53:54.035954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:41:25.288164+0000","scrub_duration":0.0037609449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:57.469297+0000","last_change":"2023-08-11T15:53:57.469406+0000","last_active":"2023-08-11T15:53:57.469297+0000","last_peered":"2023-08-11T15:53:57.469297+0000","last_clean":"2023-08-11T15:53:57.469297+0000","last_became_active":"2023-08-11T15:50:55.270605+0000","last_became_peered":"2023-08-11T15:50:55.270605+0000","last_unstale":"2023-08-11T15:53:57.469297+0000","last_undegraded":"2023-08-11T15:53:57.469297+0000","last_fullsized":"2023-08-11T15:53:57.469297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:57.469135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:57.469135+0000","last_clean_scrub_stamp":"2023-08-11T15:53:57.469135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:46:29.041437+0000","scrub_duration":0.0037607510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:51.164460+0000","last_change":"2023-08-11T15:53:51.164571+0000","last_active":"2023-08-11T15:53:51.164460+0000","last_peered":"2023-08-11T15:53:51.164460+0000","last_clean":"2023-08-11T15:53:51.164460+0000","last_became_active":"2023-08-11T15:50:54.659893+0000","last_became_peered":"2023-08-11T15:50:54.659893+0000","last_unstale":"2023-08-11T15:53:51.164460+0000","last_undegraded":"2023-08-11T15:53:51.164460+0000","last_fullsized":"2023-08-11T15:53:51.164460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:51.164304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:51.164304+0000","last_clean_scrub_stamp":"2023-08-11T15:53:51.164304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:53:39.192277+0000","scrub_duration":0.0040121719999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:46.515296+0000","last_change":"2023-08-11T15:53:46.515356+0000","last_active":"2023-08-11T15:53:46.515296+0000","last_peered":"2023-08-11T15:53:46.515296+0000","last_clean":"2023-08-11T15:53:46.515296+0000","last_became_active":"2023-08-11T15:50:54.660151+0000","last_became_peered":"2023-08-11T15:50:54.660151+0000","last_unstale":"2023-08-11T15:53:46.515296+0000","last_undegraded":"2023-08-11T15:53:46.515296+0000","last_fullsized":"2023-08-11T15:53:46.515296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:46.515201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:46.515201+0000","last_clean_scrub_stamp":"2023-08-11T15:53:46.515201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:15:01.037710+0000","scrub_duration":0.0024113239999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:52.021852+0000","last_change":"2023-08-11T15:53:52.021967+0000","last_active":"2023-08-11T15:53:52.021852+0000","last_peered":"2023-08-11T15:53:52.021852+0000","last_clean":"2023-08-11T15:53:52.021852+0000","last_became_active":"2023-08-11T15:50:54.668863+0000","last_became_peered":"2023-08-11T15:50:54.668863+0000","last_unstale":"2023-08-11T15:53:52.021852+0000","last_undegraded":"2023-08-11T15:53:52.021852+0000","last_fullsized":"2023-08-11T15:53:52.021852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:52.021696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:52.021696+0000","last_clean_scrub_stamp":"2023-08-11T15:53:52.021696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:14:41.304730+0000","scrub_duration":0.0041886620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:52.119820+0000","last_change":"2023-08-11T15:53:52.119925+0000","last_active":"2023-08-11T15:53:52.119820+0000","last_peered":"2023-08-11T15:53:52.119820+0000","last_clean":"2023-08-11T15:53:52.119820+0000","last_became_active":"2023-08-11T15:50:54.662530+0000","last_became_peered":"2023-08-11T15:50:54.662530+0000","last_unstale":"2023-08-11T15:53:52.119820+0000","last_undegraded":"2023-08-11T15:53:52.119820+0000","last_fullsized":"2023-08-11T15:53:52.119820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:53:52.119662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:53:52.119662+0000","last_clean_scrub_stamp":"2023-08-11T15:53:52.119662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:32:49.499207+0000","scrub_duration":0.00398264,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"46'119","reported_seq":229,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-08-11T15:53:51.070788+0000","last_change":"2023-08-11T15:53:51.070987+0000","last_active":"2023-08-11T15:53:51.070788+0000","last_peered":"2023-08-11T15:53:51.070788+0000","last_clean":"2023-08-11T15:53:51.070788+0000","last_became_active":"2023-08-11T15:50:48.423954+0000","last_became_peered":"2023-08-11T15:50:48.423954+0000","last_unstale":"2023-08-11T15:53:51.070788+0000","last_undegraded":"2023-08-11T15:53:51.070788+0000","last_fullsized":"2023-08-11T15:53:51.070788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'119","last_scrub_stamp":"2023-08-11T15:53:51.070603+0000","last_deep_scrub":"46'119","last_deep_scrub_stamp":"2023-08-11T15:53:51.070603+0000","last_clean_scrub_stamp":"2023-08-11T15:53:51.070603+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:06:46.491950+0000","scrub_duration":0.0092644790000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1134592,"data_stored":832107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:53:49 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0880000000000001,"15min":1.1140000000000001},"min":{"1min":0.63,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5680000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.236},{"interface":"front","average":{"1min":1.139,"5min":1.1990000000000001,"15min":1.216},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7350000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.2989999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:53:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.292,"15min":1.3200000000000001},"min":{"1min":0.81499999999999995,"5min":0.752,"15min":0.752},"max":{"1min":1.746,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.538},{"interface":"front","average":{"1min":1.355,"5min":1.3360000000000001,"15min":1.333},"min":{"1min":1.004,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.155}]},{"osd":2,"last update":"Fri Aug 11 15:53:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.1990000000000001,"15min":1.1830000000000001},"min":{"1min":0.68000000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.2629999999999999,"15min":1.2609999999999999},"min":{"1min":0.73899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.6259999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:53:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.288,"15min":1.306},"min":{"1min":0.75700000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.849},{"interface":"front","average":{"1min":1.278,"5min":1.1990000000000001,"15min":1.1930000000000001},"min":{"1min":0.86199999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.629,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.4750000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:53:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4199999999999999,"15min":1.4279999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.978,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3899999999999999,"15min":1.4119999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.347}]},{"osd":5,"last update":"Fri Aug 11 15:53:49 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.391,"15min":1.387},"min":{"1min":1.03,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.8130000000000002,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":1.798},{"interface":"front","average":{"1min":1.546,"5min":1.544,"15min":1.55},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.7190000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:53:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.5089999999999999,"15min":1.5229999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.552,"15min":1.5600000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.903}]}]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:53:48 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0529999999999999,"15min":1.0700000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.421,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.353},{"interface":"front","average":{"1min":1.165,"5min":1.1399999999999999,"15min":1.1379999999999999},"min":{"1min":0.66200000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.153}]},{"osd":1,"last update":"Fri Aug 11 15:53:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.1539999999999999,"15min":1.127},"min":{"1min":0.878,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.103},{"interface":"front","average":{"1min":1.101,"5min":1.1200000000000001,"15min":1.1240000000000001},"min":{"1min":0.73799999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.5209999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.214}]},{"osd":2,"last update":"Fri Aug 11 15:53:48 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2450000000000001,"15min":1.242},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.7909999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.446},{"interface":"front","average":{"1min":1.306,"5min":1.3080000000000001,"15min":1.337},"min":{"1min":0.87,"5min":0.624,"15min":0.624},"max":{"1min":2.0230000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.5660000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:53:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3500000000000001,"15min":1.371},"min":{"1min":0.88200000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.875,"5min":2.774,"15min":2.774},"last":1.502},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.3440000000000001,"15min":1.3759999999999999},"min":{"1min":0.80400000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.9039999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.5389999999999999}]},{"osd":4,"last update":"Fri Aug 11 15:53:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.298,"15min":1.3200000000000001},"min":{"1min":0.83599999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.694,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":1.048},{"interface":"front","average":{"1min":1.381,"5min":1.444,"15min":1.4830000000000001},"min":{"1min":0.76600000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.802,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.2669999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:53:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.456,"15min":1.494},"min":{"1min":0.65500000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0230000000000001,"5min":3.371,"15min":3.371},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.466,"15min":1.498},"min":{"1min":0.90800000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.972,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.464}]},{"osd":7,"last update":"Fri Aug 11 15:53:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.5940000000000001,"15min":1.641},"min":{"1min":0.874,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.98,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":1.403},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.637,"15min":1.6919999999999999},"min":{"1min":0.71899999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.169,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.3100000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.052,"15min":1.032},"min":{"1min":0.67600000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5409999999999999,"5min":1.552,"15min":1.552},"last":1.155},{"interface":"front","average":{"1min":1.173,"5min":1.151,"15min":1.163},"min":{"1min":0.69299999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.202}]},{"osd":1,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1539999999999999,"15min":1.1439999999999999},"min":{"1min":0.71499999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6080000000000001,"5min":1.895,"15min":1.895},"last":1.474},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1830000000000001,"15min":1.169},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5940000000000001,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.2649999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2330000000000001,"15min":1.276},"min":{"1min":0.748,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5820000000000001,"5min":1.827,"15min":1.827},"last":1.085},{"interface":"front","average":{"1min":1.25,"5min":1.323,"15min":1.3680000000000001},"min":{"1min":0.88400000000000001,"5min":0.746,"15min":0.746},"max":{"1min":1.5860000000000001,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.3480000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3340000000000001,"15min":1.347},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.607,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.323},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.393,"15min":1.4359999999999999},"min":{"1min":1.01,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.603,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.5089999999999999}]},{"osd":4,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.411,"15min":1.4570000000000001},"min":{"1min":0.91500000000000004,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.679,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.02},{"interface":"front","average":{"1min":1.476,"5min":1.5589999999999999,"15min":1.599},"min":{"1min":0.96299999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8009999999999999,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":1.446}]},{"osd":6,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4330000000000001,"15min":1.48},"min":{"1min":0.82299999999999995,"5min":0.622,"15min":0.622},"max":{"1min":1.7250000000000001,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.5720000000000001,"15min":1.625},"min":{"1min":1.173,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7749999999999999,"5min":3.9870000000000001,"15min":3.9870000000000001},"last":1.242}]},{"osd":7,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.5509999999999999,"15min":1.595},"min":{"1min":1.0880000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8420000000000001,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":1.377},{"interface":"front","average":{"1min":1.482,"5min":1.5940000000000001,"15min":1.6619999999999999},"min":{"1min":1.2509999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.679,"5min":3.367,"15min":3.367},"last":1.302}]}]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0760000000000001,"15min":1.0640000000000001},"min":{"1min":0.78200000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5449999999999999,"5min":1.776,"15min":1.776},"last":1.131},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.871,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.669,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.97399999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1220000000000001,"15min":1.1299999999999999},"min":{"1min":0.78500000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.601,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1859999999999999,"15min":1.2},"min":{"1min":0.78000000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.464,"5min":1.756,"15min":1.756},"last":0.88200000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2529999999999999,"15min":1.2749999999999999},"min":{"1min":0.93600000000000005,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.587,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.552},{"interface":"front","average":{"1min":1.26,"5min":1.2390000000000001,"15min":1.2250000000000001},"min":{"1min":0.76700000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.5609999999999999,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.26}]},{"osd":3,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.1899999999999999,"15min":1.1719999999999999},"min":{"1min":1.1850000000000001,"5min":0.75,"15min":0.75},"max":{"1min":1.5229999999999999,"5min":1.736,"15min":1.736},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3640000000000001,"15min":1.3660000000000001},"min":{"1min":0.89600000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.714,"5min":2.121,"15min":2.121},"last":1.359}]},{"osd":5,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.2949999999999999,"15min":1.296},"min":{"1min":0.877,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6899999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.391},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3200000000000001,"15min":1.3460000000000001},"min":{"1min":0.88800000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.7450000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.605}]},{"osd":6,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.399,"15min":1.389},"min":{"1min":0.98799999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.917,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.458,"15min":1.462},"min":{"1min":1.0620000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.77,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.4990000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.411,"15min":1.417},"min":{"1min":1.1870000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7230000000000001,"5min":2.286,"15min":2.286},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5760000000000001,"15min":1.5940000000000001},"min":{"1min":1.337,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.0489999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.6599999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542188,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.92200000000000004,"15min":0.97199999999999998},"min":{"1min":0.45400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.004,"15min":0.98899999999999999},"min":{"1min":0.64300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.98999999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.008,"15min":1.0389999999999999},"min":{"1min":0.53400000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.4630000000000001,"5min":1.78,"15min":1.78},"last":1.71},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.95999999999999996,"15min":0.98699999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5049999999999999,"5min":1.623,"15min":1.623},"last":0.92200000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1830000000000001,"15min":1.2150000000000001},"min":{"1min":0.54500000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6259999999999999,"5min":3.7799999999999998,"15min":3.7799999999999998},"last":1.109},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.7509999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.165}]},{"osd":4,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3400000000000001,"15min":1.373},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.849,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.359},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.371,"15min":1.383},"min":{"1min":0.90700000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.956,"5min":2.573,"15min":2.573},"last":1.5740000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.53,"15min":1.589},"min":{"1min":0.85899999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0630000000000002,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.4610000000000001,"15min":1.5529999999999999},"min":{"1min":0.83699999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.0209999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":1.411}]},{"osd":6,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.504,"15min":1.548},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.9770000000000001,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.458},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.5600000000000001,"15min":1.631},"min":{"1min":1.038,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.1219999999999999,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":1.659}]},{"osd":7,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.625,"15min":1.6819999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.1699999999999999,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":1.236},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.579,"15min":1.6679999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.911,"5min":3.4340000000000002,"15min":3.4340000000000002},"last":1.5269999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88800000000000001,"15min":0.89900000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5740000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.002,"15min":1.028},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8,"5min":1.865,"15min":1.865},"last":1.389}]},{"osd":1,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.96499999999999997,"15min":0.94199999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.8120000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0609999999999999,"15min":1.085},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":1.024}]},{"osd":3,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98299999999999998,"15min":0.99199999999999999},"min":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5649999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1830000000000001,"15min":1.1950000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.617,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.6379999999999999}]},{"osd":4,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.393,"15min":1.411},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.8580000000000001,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.339,"15min":1.367},"min":{"1min":0.88600000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0150000000000001,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.2789999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.4650000000000001,"15min":1.5089999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.8360000000000001,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":1.6899999999999999},{"interface":"front","average":{"1min":1.29,"5min":1.403,"15min":1.45},"min":{"1min":0.96399999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.7450000000000001,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.3600000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3979999999999999,"15min":1.448},"min":{"1min":0.82899999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7809999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.222},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4930000000000001,"15min":1.508},"min":{"1min":0.93200000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.0720000000000001,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.738}]},{"osd":7,"last update":"Fri Aug 11 15:53:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.554,"15min":1.611},"min":{"1min":0.84099999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8859999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.5569999999999999,"15min":1.6020000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.4489999999999998,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":1.514}]}]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":540672,"data_stored":241739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:53:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.92200000000000004,"15min":0.90800000000000003},"min":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0269999999999999,"15min":1.0209999999999999},"min":{"1min":0.66700000000000004,"5min":0.501,"15min":0.501},"max":{"1min":2.9860000000000002,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":1.3149999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:53:51 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.0409999999999999,"15min":1.0449999999999999},"min":{"1min":0.63400000000000001,"5min":0.499,"15min":0.499},"max":{"1min":4.1760000000000002,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.115,"15min":1.0800000000000001},"min":{"1min":0.622,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":5.649,"5min":5.649,"15min":5.649},"last":1.345}]},{"osd":3,"last update":"Fri Aug 11 15:53:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3,"15min":1.298},"min":{"1min":0.77500000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.8380000000000001,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":1.488},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.282,"15min":1.27},"min":{"1min":0.70999999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":3.0870000000000002,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:53:51 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4119999999999999,"15min":1.4179999999999999},"min":{"1min":0.88500000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.401,"15min":1.429},"min":{"1min":0.88100000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.581,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":1.587}]},{"osd":5,"last update":"Fri Aug 11 15:53:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3380000000000001,"15min":1.339},"min":{"1min":1.0089999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.256},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.5209999999999999,"15min":1.5169999999999999},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":5.4139999999999997,"5min":5.4139999999999997,"15min":5.4139999999999997},"last":1.5669999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:53:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5369999999999999,"15min":1.5229999999999999},"min":{"1min":0.876,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":3.7120000000000002,"5min":3.7120000000000002,"15min":3.7120000000000002},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.4670000000000001,"15min":1.47},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.3109999999999999,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.4279999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:53:51 2023","interfaces":[{"interface":"back","average":{"1min":1.7390000000000001,"5min":1.5629999999999999,"15min":1.538},"min":{"1min":0.93600000000000005,"5min":0.873,"15min":0.873},"max":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.5460000000000003},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":1.0249999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":3.6429999999999998,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":1.393}]}]},{"osd":0,"up_from":14,"seq":60129542189,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1134592,"data_stored":832107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.92800000000000005,"15min":0.92400000000000004},"min":{"1min":0.54800000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5209999999999999,"5min":1.633,"15min":1.633},"last":1.0089999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.98499999999999999,"15min":0.97599999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.617,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0429999999999999,"15min":1.0189999999999999},"min":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.3320000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.94699999999999995,"15min":0.94899999999999995},"min":{"1min":0.57999999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.155,"15min":1.155},"min":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.073}]},{"osd":4,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.325,"15min":1.337},"min":{"1min":0.96899999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8580000000000001,"5min":2.23,"15min":2.23},"last":1.472},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.80600000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.738,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.1739999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.357,"15min":1.345},"min":{"1min":0.85499999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.236},{"interface":"front","average":{"1min":1.401,"5min":1.4219999999999999,"15min":1.4450000000000001},"min":{"1min":0.90500000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.161,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":1.631}]},{"osd":6,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.331,"15min":1.365},"min":{"1min":0.84799999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8839999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4339999999999999,"15min":1.431},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.4330000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:53:52 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.4910000000000001,"15min":1.5049999999999999},"min":{"1min":0.97099999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.351,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.54},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.4470000000000001,"15min":1.4830000000000001},"min":{"1min":0.82599999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.496}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:54:11.055 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-08-11T15:54:11.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.275+0000 7f089eee0640 1 Processor -- start 2023-08-11T15:54:11.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.276+0000 7f089eee0640 1 -- start start 2023-08-11T15:54:11.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.277+0000 7f089eee0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089813ce80 0x7f089813f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.277+0000 7f089eee0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f08980cce30 0x7f089813fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.277+0000 7f089eee0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08980cd470 con 0x7f089813d260 2023-08-11T15:54:11.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.277+0000 7f089eee0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0898140670 con 0x7f08980cce30 2023-08-11T15:54:11.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.277+0000 7f089eee0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08980cd750 con 0x7f089813ce80 2023-08-11T15:54:11.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.277+0000 7f089d6dd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f089813d260 0x7f08980cc720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50642/0 (socket says 172.21.15.27:50642) 2023-08-11T15:54:11.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.277+0000 7f089d6dd640 1 -- 172.21.15.27:0/2362512910 learned_addr learned my addr 172.21.15.27:0/2362512910 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-08-11T15:54:11.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089dede640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089813ce80 0x7f089813f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089e6df640 1 --2- 172.21.15.27:0/2362512910 >> v2:172.21.15.119:3300/0 conn(0x7f08980cce30 0x7f089813fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076286927 0 0) 0x7f08980cd470 con 0x7f089813d260 2023-08-11T15:54:11.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0884003520 con 0x7f089813d260 2023-08-11T15:54:11.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1742870045 0 0) 0x7f0884003520 con 0x7f089813d260 2023-08-11T15:54:11.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089813ce80 msgr2=0x7f089813f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:11.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089cedc640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089813ce80 0x7f089813f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 >> v2:172.21.15.119:3300/0 conn(0x7f08980cce30 msgr2=0x7f089813fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089cedc640 1 --2- 172.21.15.27:0/2362512910 >> v2:172.21.15.119:3300/0 conn(0x7f08980cce30 0x7f089813fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08980ad7d0 con 0x7f089813d260 2023-08-11T15:54:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.278+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2197288807 0 0) 0x7f0888001670 con 0x7f089813d260 2023-08-11T15:54:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 >> v1:172.21.15.27:6789/0 conn(0x7f089813d260 legacy=0x7f08980cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089cedc640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0884003970 0x7f0884003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089cedc640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089813ce80 0x7f08840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089cedc640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 0x7f0884005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0898140670 con 0x7f0884004d00 2023-08-11T15:54:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08980cd750 con 0x7f0884003970 2023-08-11T15:54:11.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088400b980 con 0x7f089813ce80 2023-08-11T15:54:11.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089e6df640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 0x7f0884005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089dede640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0884003970 0x7f0884003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089d6dd640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089813ce80 0x7f08840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089e6df640 1 -- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089813ce80 msgr2=0x7f08840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:11.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089e6df640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089813ce80 0x7f08840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089e6df640 1 -- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0884003970 msgr2=0x7f0884003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:11.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089e6df640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0884003970 0x7f0884003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089e6df640 1 -- 172.21.15.27:0/2362512910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0884003520 con 0x7f0884004d00 2023-08-11T15:54:11.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089d6dd640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089813ce80 0x7f08840047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:54:11.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.279+0000 7f089dede640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0884003970 0x7f0884003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:54:11.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.281+0000 7f089e6df640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 0x7f0884005180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0890009fc0 tx=0x7f08840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2442c36d254ded00 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:11.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.281+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089002cc30 con 0x7f0884004d00 2023-08-11T15:54:11.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.281+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089000f040 con 0x7f0884004d00 2023-08-11T15:54:11.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.281+0000 7f089cedc640 1 -- 172.21.15.27:0/2362512910 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890034440 con 0x7f0884004d00 2023-08-11T15:54:11.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.281+0000 7f089eee0640 1 -- 172.21.15.27:0/2362512910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 msgr2=0x7f0884005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.281+0000 7f089eee0640 1 --2- 172.21.15.27:0/2362512910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 0x7f0884005180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0890009fc0 tx=0x7f08840057f0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.281+0000 7f089e6df640 1 -- 172.21.15.27:0/2362512910 reap_dead start 2023-08-11T15:54:11.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.282+0000 7f089eee0640 1 -- 172.21.15.27:0/2362512910 shutdown_connections 2023-08-11T15:54:11.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.282+0000 7f089eee0640 1 -- 172.21.15.27:0/2362512910 >> 172.21.15.27:0/2362512910 conn(0x7f0898069800 msgr2=0x7f0898069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:11.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.282+0000 7f089eee0640 1 -- 172.21.15.27:0/2362512910 shutdown_connections 2023-08-11T15:54:11.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.282+0000 7f089eee0640 1 -- 172.21.15.27:0/2362512910 wait complete. 2023-08-11T15:54:11.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.284+0000 7f089eee0640 1 Processor -- start 2023-08-11T15:54:11.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.284+0000 7f089eee0640 1 -- start start 2023-08-11T15:54:11.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.285+0000 7f089eee0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 0x7f0898138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.285+0000 7f089eee0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089815b100 0x7f089815b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.285+0000 7f089eee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08981f4fd0 0x7f08981f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.285+0000 7f089eee0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08981f59c0 con 0x7f0884004d00 2023-08-11T15:54:11.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.285+0000 7f089eee0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089815bbc0 con 0x7f08981f4fd0 2023-08-11T15:54:11.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.285+0000 7f089eee0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089815bea0 con 0x7f089815b100 2023-08-11T15:54:11.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.285+0000 7f089dede640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 0x7f0898138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.285+0000 7f089dede640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 0x7f0898138630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41966/0 (socket says 172.21.15.27:41966) 2023-08-11T15:54:11.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.285+0000 7f089dede640 1 -- 172.21.15.27:0/84320724 learned_addr learned my addr 172.21.15.27:0/84320724 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:54:11.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f089dede640 1 -- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089815b100 msgr2=0x7f089815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f089dede640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089815b100 0x7f089815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f089dede640 1 -- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08981f4fd0 msgr2=0x7f08981f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f089dede640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08981f4fd0 0x7f08981f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f089dede640 1 -- 172.21.15.27:0/84320724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0898139450 con 0x7f0884004d00 2023-08-11T15:54:11.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f089dede640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 0x7f0898138630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f08940108b0 tx=0x7f0894010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:11.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f088e7fc640 1 -- 172.21.15.27:0/84320724 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089400eae0 con 0x7f0884004d00 2023-08-11T15:54:11.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f088e7fc640 1 -- 172.21.15.27:0/84320724 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089400ec40 con 0x7f0884004d00 2023-08-11T15:54:11.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f088e7fc640 1 -- 172.21.15.27:0/84320724 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089401c400 con 0x7f0884004d00 2023-08-11T15:54:11.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f089eee0640 1 -- 172.21.15.27:0/84320724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0898139730 con 0x7f0884004d00 2023-08-11T15:54:11.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.286+0000 7f089eee0640 1 -- 172.21.15.27:0/84320724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0898138b70 con 0x7f0884004d00 2023-08-11T15:54:11.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.288+0000 7f089eee0640 1 -- 172.21.15.27:0/84320724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089815c0e0 con 0x7f0884004d00 2023-08-11T15:54:11.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.288+0000 7f088e7fc640 1 -- 172.21.15.27:0/84320724 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f089401c560 con 0x7f0884004d00 2023-08-11T15:54:11.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.289+0000 7f088e7fc640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a050 0x7f087c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.290+0000 7f088e7fc640 1 -- 172.21.15.27:0/84320724 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f0894088a20 con 0x7f0884004d00 2023-08-11T15:54:11.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.290+0000 7f089d6dd640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a050 0x7f087c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.291+0000 7f089d6dd640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a050 0x7f087c06c510 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0898154a60 tx=0x7f0888009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:11.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.293+0000 7f088e7fc640 1 -- 172.21.15.27:0/84320724 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7f0894057d90 con 0x7f0884004d00 2023-08-11T15:54:11.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.490+0000 7f089eee0640 1 -- 172.21.15.27:0/84320724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f0898066e80 con 0x7f0884004d00 2023-08-11T15:54:11.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.501+0000 7f088e7fc640 1 -- 172.21.15.27:0/84320724 <== mon.0 v2:172.21.15.27:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0894057d90 con 0x7f0884004d00 2023-08-11T15:54:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.502+0000 7f088e7fc640 1 -- 172.21.15.27:0/84320724 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0894057920 con 0x7f0884004d00 2023-08-11T15:54:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 -- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a050 msgr2=0x7f087c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a050 0x7f087c06c510 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0898154a60 tx=0x7f0888009040 comp rx=0 tx=0).stop 2023-08-11T15:54:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 -- 172.21.15.27:0/84320724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 msgr2=0x7f0898138630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 0x7f0898138630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f08940108b0 tx=0x7f0894010d70 comp rx=0 tx=0).stop 2023-08-11T15:54:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 -- 172.21.15.27:0/84320724 shutdown_connections 2023-08-11T15:54:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7f087c06a050 0x7f087c06c510 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f089815b100 0x7f089815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08981f4fd0 0x7f08981f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 --2- 172.21.15.27:0/84320724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0884004d00 0x7f0898138630 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 -- 172.21.15.27:0/84320724 >> 172.21.15.27:0/84320724 conn(0x7f0898069800 msgr2=0x7f08980b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:11.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 -- 172.21.15.27:0/84320724 shutdown_connections 2023-08-11T15:54:11.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.505+0000 7f089eee0640 1 -- 172.21.15.27:0/84320724 wait complete. 2023-08-11T15:54:11.524 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-11T15:54:11.525 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:54:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.751+0000 7fca539c2640 1 Processor -- start 2023-08-11T15:54:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.751+0000 7fca539c2640 1 -- start start 2023-08-11T15:54:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.752+0000 7fca539c2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fca4c15c620 0x7fca4c0b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.753+0000 7fca539c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca4c0b82f0 0x7fca4c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.753+0000 7fca539c2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4c0cda30 con 0x7fca4c0b55c0 2023-08-11T15:54:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.753+0000 7fca539c2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4c0ccc10 con 0x7fca4c15c620 2023-08-11T15:54:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.753+0000 7fca539c2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4c161110 con 0x7fca4c0b82f0 2023-08-11T15:54:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.753+0000 7fca5175e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fca4c15c620 0x7fca4c0b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.753+0000 7fca5175e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fca4c15c620 0x7fca4c0b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41980/0 (socket says 172.21.15.27:41980) 2023-08-11T15:54:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.753+0000 7fca5175e640 1 -- 172.21.15.27:0/2689996935 learned_addr learned my addr 172.21.15.27:0/2689996935 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:54:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.753+0000 7fca50f5d640 1 --2- 172.21.15.27:0/2689996935 >> v2:172.21.15.119:3300/0 conn(0x7fca4c0b82f0 0x7fca4c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.753+0000 7fca5175e640 1 -- 172.21.15.27:0/2689996935 >> v2:172.21.15.119:3300/0 conn(0x7fca4c0b82f0 msgr2=0x7fca4c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:11.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.754+0000 7fca5175e640 1 --2- 172.21.15.27:0/2689996935 >> v2:172.21.15.119:3300/0 conn(0x7fca4c0b82f0 0x7fca4c0cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.754+0000 7fca5175e640 1 -- 172.21.15.27:0/2689996935 >> v1:172.21.15.119:6789/0 conn(0x7fca4c0b55c0 legacy=0x7fca4c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:11.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.754+0000 7fca5175e640 1 -- 172.21.15.27:0/2689996935 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0cce20 con 0x7fca4c15c620 2023-08-11T15:54:11.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.755+0000 7fca5175e640 1 --2- 172.21.15.27:0/2689996935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 0x7fca4c0b7db0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0b50e0 tx=0x7fca4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e422cb7d40338547 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:11.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.755+0000 7fca43fff640 1 -- 172.21.15.27:0/2689996935 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48031870 con 0x7fca4c15c620 2023-08-11T15:54:11.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.755+0000 7fca43fff640 1 -- 172.21.15.27:0/2689996935 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fca480319d0 con 0x7fca4c15c620 2023-08-11T15:54:11.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.755+0000 7fca43fff640 1 -- 172.21.15.27:0/2689996935 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48031d80 con 0x7fca4c15c620 2023-08-11T15:54:11.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.756+0000 7fca539c2640 1 -- 172.21.15.27:0/2689996935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 msgr2=0x7fca4c0b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.756+0000 7fca539c2640 1 --2- 172.21.15.27:0/2689996935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 0x7fca4c0b7db0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0b50e0 tx=0x7fca4802b910 comp rx=0 tx=0).stop 2023-08-11T15:54:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.756+0000 7fca539c2640 1 -- 172.21.15.27:0/2689996935 shutdown_connections 2023-08-11T15:54:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.756+0000 7fca539c2640 1 --2- 172.21.15.27:0/2689996935 >> v2:172.21.15.119:3300/0 conn(0x7fca4c0b82f0 0x7fca4c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.756+0000 7fca539c2640 1 --2- 172.21.15.27:0/2689996935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 0x7fca4c0b7db0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.756+0000 7fca539c2640 1 -- 172.21.15.27:0/2689996935 >> 172.21.15.27:0/2689996935 conn(0x7fca4c071110 msgr2=0x7fca4c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:11.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.756+0000 7fca539c2640 1 -- 172.21.15.27:0/2689996935 shutdown_connections 2023-08-11T15:54:11.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.756+0000 7fca539c2640 1 -- 172.21.15.27:0/2689996935 wait complete. 2023-08-11T15:54:11.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.758+0000 7fca539c2640 1 Processor -- start 2023-08-11T15:54:11.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.758+0000 7fca539c2640 1 -- start start 2023-08-11T15:54:11.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.759+0000 7fca539c2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fca4c0b55c0 0x7fca4c14e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.759+0000 7fca539c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4c0b82f0 0x7fca4c15ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.759+0000 7fca539c2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 0x7fca4c0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.759+0000 7fca539c2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c0cca00 con 0x7fca4c15c620 2023-08-11T15:54:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.759+0000 7fca539c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c14eef0 con 0x7fca4c0b82f0 2023-08-11T15:54:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.759+0000 7fca539c2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c0ca480 con 0x7fca4c0b55c0 2023-08-11T15:54:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.759+0000 7fca51f5f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 0x7fca4c0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.759+0000 7fca51f5f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 0x7fca4c0cbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41984/0 (socket says 172.21.15.27:41984) 2023-08-11T15:54:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.759+0000 7fca51f5f640 1 -- 172.21.15.27:0/516888419 learned_addr learned my addr 172.21.15.27:0/516888419 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-08-11T15:54:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca51f5f640 1 -- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fca4c0b55c0 msgr2=0x7fca4c14e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca51f5f640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fca4c0b55c0 0x7fca4c14e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca51f5f640 1 -- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4c0b82f0 msgr2=0x7fca4c15ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:11.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca51f5f640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4c0b82f0 0x7fca4c15ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:11.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca51f5f640 1 -- 172.21.15.27:0/516888419 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca480094d0 con 0x7fca4c15c620 2023-08-11T15:54:11.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca51f5f640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 0x7fca4c0cbb10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fca4400c780 tx=0x7fca4400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:11.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca41ffb640 1 -- 172.21.15.27:0/516888419 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44003d80 con 0x7fca4c15c620 2023-08-11T15:54:11.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca41ffb640 1 -- 172.21.15.27:0/516888419 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fca4400ce30 con 0x7fca4c15c620 2023-08-11T15:54:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca41ffb640 1 -- 172.21.15.27:0/516888419 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca440052d0 con 0x7fca4c15c620 2023-08-11T15:54:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca539c2640 1 -- 172.21.15.27:0/516888419 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c0ca760 con 0x7fca4c15c620 2023-08-11T15:54:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.760+0000 7fca539c2640 1 -- 172.21.15.27:0/516888419 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c0cad20 con 0x7fca4c15c620 2023-08-11T15:54:11.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.762+0000 7fca539c2640 1 -- 172.21.15.27:0/516888419 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4c073990 con 0x7fca4c15c620 2023-08-11T15:54:11.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.763+0000 7fca41ffb640 1 -- 172.21.15.27:0/516888419 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fca44006b10 con 0x7fca4c15c620 2023-08-11T15:54:11.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.764+0000 7fca41ffb640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fca3006a050 0x7fca3006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.764+0000 7fca41ffb640 1 -- 172.21.15.27:0/516888419 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fca44087900 con 0x7fca4c15c620 2023-08-11T15:54:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.765+0000 7fca5175e640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fca3006a050 0x7fca3006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.766+0000 7fca5175e640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fca3006a050 0x7fca3006c510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fca48036000 tx=0x7fca48030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.769+0000 7fca41ffb640 1 -- 172.21.15.27:0/516888419 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178096 (secure 0 0 0) 0x7fca440530c0 con 0x7fca4c15c620 2023-08-11T15:54:12.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:11.999+0000 7fca539c2640 1 -- 172.21.15.27:0/516888419 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca4c06b6f0 con 0x7fca4c15c620 2023-08-11T15:54:12.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.000+0000 7fca41ffb640 1 -- 172.21.15.27:0/516888419 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fca44017030 con 0x7fca4c15c620 2023-08-11T15:54:12.002 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-08-11T15:54:12.002 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2294518038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2294518038},{"type":"v1","addr":"172.21.15.119:6836","nonce":2294518038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5188,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2104666724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2104666724},{"type":"v1","addr":"172.21.15.119:6837","nonce":2104666724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2379252997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2379252997},{"type":"v1","addr":"172.21.15.27:6833","nonce":2379252997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":6024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1096450395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1096450395},{"type":"v1","addr":"172.21.15.27:6835","nonce":1096450395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2023-08-11T15:54:12.003 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 20 2023-08-11T15:54:12.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.002+0000 7fca539c2640 1 -- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fca3006a050 msgr2=0x7fca3006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:12.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.002+0000 7fca539c2640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fca3006a050 0x7fca3006c510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fca48036000 tx=0x7fca48030000 comp rx=0 tx=0).stop 2023-08-11T15:54:12.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.002+0000 7fca539c2640 1 -- 172.21.15.27:0/516888419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 msgr2=0x7fca4c0cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:12.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.002+0000 7fca539c2640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 0x7fca4c0cbb10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fca4400c780 tx=0x7fca4400cc40 comp rx=0 tx=0).stop 2023-08-11T15:54:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.002+0000 7fca539c2640 1 -- 172.21.15.27:0/516888419 shutdown_connections 2023-08-11T15:54:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.003+0000 7fca539c2640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:6832/792514541,v1:172.21.15.119:6833/792514541] conn(0x7fca3006a050 0x7fca3006c510 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.003+0000 7fca539c2640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4c15c620 0x7fca4c0cbb10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.003+0000 7fca539c2640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4c0b82f0 0x7fca4c15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.003+0000 7fca539c2640 1 --2- 172.21.15.27:0/516888419 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fca4c0b55c0 0x7fca4c14e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.003+0000 7fca539c2640 1 -- 172.21.15.27:0/516888419 >> 172.21.15.27:0/516888419 conn(0x7fca4c071110 msgr2=0x7fca4c0d25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.003+0000 7fca539c2640 1 -- 172.21.15.27:0/516888419 shutdown_connections 2023-08-11T15:54:12.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-08-11T15:54:12.003+0000 7fca539c2640 1 -- 172.21.15.27:0/516888419 wait complete. 2023-08-11T15:54:12.022 DEBUG:tasks.cephfs.filesystem:already dead... 2023-08-11T15:54:12.022 INFO:teuthology.misc:Shutting down mds daemons... 2023-08-11T15:54:12.023 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-11T15:54:12.023 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.042 INFO:tasks.ceph.mds.a:Stopped 2023-08-11T15:54:12.043 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-11T15:54:12.043 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.105 INFO:tasks.ceph.mds.c:Stopped 2023-08-11T15:54:12.105 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-11T15:54:12.106 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.134 INFO:tasks.ceph.mds.b:Stopped 2023-08-11T15:54:12.135 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-11T15:54:12.135 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.196 INFO:tasks.ceph.mds.d:Stopped 2023-08-11T15:54:12.197 INFO:teuthology.misc:Shutting down osd daemons... 2023-08-11T15:54:12.197 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-08-11T15:54:12.198 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.242 INFO:tasks.ceph.osd.0:Stopped 2023-08-11T15:54:12.243 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-08-11T15:54:12.243 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.309 INFO:tasks.ceph.osd.1:Stopped 2023-08-11T15:54:12.311 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-08-11T15:54:12.311 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.362 INFO:tasks.ceph.osd.2:Stopped 2023-08-11T15:54:12.363 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-08-11T15:54:12.363 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.423 INFO:tasks.ceph.osd.3:Stopped 2023-08-11T15:54:12.423 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-08-11T15:54:12.424 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.490 INFO:tasks.ceph.osd.4:Stopped 2023-08-11T15:54:12.491 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-08-11T15:54:12.491 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.564 INFO:tasks.ceph.osd.5:Stopped 2023-08-11T15:54:12.565 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-08-11T15:54:12.565 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.616 INFO:tasks.ceph.osd.6:Stopped 2023-08-11T15:54:12.617 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-08-11T15:54:12.617 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.693 INFO:tasks.ceph.osd.7:Stopped 2023-08-11T15:54:12.693 INFO:teuthology.misc:Shutting down mgr daemons... 2023-08-11T15:54:12.694 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-08-11T15:54:12.694 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.756 INFO:tasks.ceph.mgr.y:Stopped 2023-08-11T15:54:12.757 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-08-11T15:54:12.758 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.852 INFO:tasks.ceph.mgr.x:Stopped 2023-08-11T15:54:12.852 INFO:teuthology.misc:Shutting down mon daemons... 2023-08-11T15:54:12.853 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-08-11T15:54:12.853 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.889 INFO:tasks.ceph.mon.a:Stopped 2023-08-11T15:54:12.889 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-08-11T15:54:12.890 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.931 INFO:tasks.ceph.mon.b:Stopped 2023-08-11T15:54:12.932 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-08-11T15:54:12.932 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:54:12.977 INFO:tasks.ceph.mon.c:Stopped 2023-08-11T15:54:12.978 INFO:tasks.ceph:Checking cluster log for badness... 2023-08-11T15:54:12.979 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-08-11T15:54:13.099 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:54:13.099 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-08-11T15:54:13.541 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:54:13.542 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-08-11T15:54:13.814 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:54:13.815 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-08-11T15:54:14.160 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:54:14.161 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-08-11T15:54:14.462 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:54:14.462 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-08-11T15:54:14.886 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:54:14.887 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-08-11T15:54:15.156 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:54:15.156 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-08-11T15:54:15.434 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:54:15.434 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-08-11T15:54:15.691 INFO:tasks.ceph:Archiving mon data... 2023-08-11T15:54:15.693 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066/data/mon.a.tgz 2023-08-11T15:54:15.693 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2023-08-11T15:54:15.760 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.QOA2o0Auo2 2023-08-11T15:54:15.761 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QOA2o0Auo2 2023-08-11T15:54:15.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.QOA2o0Auo2 2023-08-11T15:54:16.140 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.QOA2o0Auo2 is 324KB 2023-08-11T15:54:16.258 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.QOA2o0Auo2 2023-08-11T15:54:16.322 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066/data/mon.b.tgz 2023-08-11T15:54:16.323 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2023-08-11T15:54:16.384 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.DsKTsmeL4t 2023-08-11T15:54:16.385 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.DsKTsmeL4t 2023-08-11T15:54:16.595 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.DsKTsmeL4t 2023-08-11T15:54:16.748 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.DsKTsmeL4t is 428KB 2023-08-11T15:54:16.856 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.DsKTsmeL4t 2023-08-11T15:54:16.914 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066/data/mon.c.tgz 2023-08-11T15:54:16.914 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2023-08-11T15:54:16.967 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.a4f7oniGao 2023-08-11T15:54:16.967 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.a4f7oniGao 2023-08-11T15:54:17.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.a4f7oniGao 2023-08-11T15:54:17.309 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.a4f7oniGao is 428KB 2023-08-11T15:54:17.419 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.a4f7oniGao 2023-08-11T15:54:17.478 INFO:tasks.ceph:Cleaning ceph cluster... 2023-08-11T15:54:17.478 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-08-11T15:54:17.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-08-11T15:54:17.596 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-08-11T15:54:17.596 DEBUG:teuthology.orchestra.run.smithi027:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-08-11T15:54:17.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-08-11T15:54:17.711 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-08-11T15:54:17.718 INFO:teuthology.orchestra.run.smithi027.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-08-11T15:54:17.729 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:54:17.731 INFO:tasks.ceph:Archiving crash dumps... 2023-08-11T15:54:17.733 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066/remote/smithi027/crash 2023-08-11T15:54:17.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-08-11T15:54:17.779 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:54:17.868 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066/remote/smithi119/crash 2023-08-11T15:54:17.869 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-08-11T15:54:17.965 INFO:tasks.ceph:Shutting down logrotate 2023-08-11T15:54:17.965 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:54:17.974 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-11T15:54:18.114 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-08-11T15:54:18.227 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-08-11T15:54:18.344 INFO:tasks.ceph:Compressing logs... 2023-08-11T15:54:18.345 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-11T15:54:18.354 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-11T15:54:18.440 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89196.log 2023-08-11T15:54:18.441 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.96551.log 2023-08-11T15:54:18.443 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.89196.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.89196.log.gz 2023-08-11T15:54:18.443 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.89236.log 2023-08-11T15:54:18.445 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.1.96551.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-08-11T15:54:18.445 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.89236.log: 88.9% -- replaced with /var/log/ceph/ceph-client.1.96551.log.gz 2023-08-11T15:54:18.445 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89236.log.gz 2023-08-11T15:54:18.446 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89287.log 2023-08-11T15:54:18.446 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-08-11T15:54:18.447 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.89287.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-08-11T15:54:18.447 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89287.log.gz 2023-08-11T15:54:18.448 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-08-11T15:54:18.449 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89407.log 2023-08-11T15:54:18.451 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.95804.log 2023-08-11T15:54:18.454 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.89407.log: gzip -5 --verbose -- 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89407.log.gz 2023-08-11T15:54:18.454 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph.audit.log 2023-08-11T15:54:18.455 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.3.95804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89116.log 2023-08-11T15:54:18.457 INFO:teuthology.orchestra.run.smithi119.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.3.95804.log.gz 2023-08-11T15:54:18.458 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89156.log 2023-08-11T15:54:18.459 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101434.log 2023-08-11T15:54:18.459 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96343.log 2023-08-11T15:54:18.460 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.89116.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89116.log.gz 2023-08-11T15:54:18.460 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89327.log 2023-08-11T15:54:18.463 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104150.log 2023-08-11T15:54:18.463 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101434.log.gz 2023-08-11T15:54:18.464 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96343.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-08-11T15:54:18.464 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96343.log.gz 2023-08-11T15:54:18.464 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95787.log 2023-08-11T15:54:18.464 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104150.log.gz 2023-08-11T15:54:18.465 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.89156.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-08-11T15:54:18.466 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89156.log.gz 2023-08-11T15:54:18.466 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.1.95285.log 2023-08-11T15:54:18.467 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.89327.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89327.log.gz 2023-08-11T15:54:18.467 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-08-11T15:54:18.468 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.1.95285.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-08-11T15:54:18.468 INFO:teuthology.orchestra.run.smithi119.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.1.95285.log.gz 2023-08-11T15:54:18.476 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89367.log 2023-08-11T15:54:18.477 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.5.log: 91.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-08-11T15:54:18.477 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-08-11T15:54:18.477 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.89367.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89367.log.gz 2023-08-11T15:54:18.477 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-08-11T15:54:18.478 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-08-11T15:54:18.478 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-08-11T15:54:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102428.log 2023-08-11T15:54:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.95787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89169.log 2023-08-11T15:54:18.481 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95787.log.gz 2023-08-11T15:54:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102428.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102428.log.gz 2023-08-11T15:54:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104666.log 2023-08-11T15:54:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.89169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95829.log 2023-08-11T15:54:18.481 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89169.log.gz 2023-08-11T15:54:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104666.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104666.log.gz 2023-08-11T15:54:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103898.log 2023-08-11T15:54:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.95829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103394.log 2023-08-11T15:54:18.482 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95829.log.gz 2023-08-11T15:54:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103898.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103898.log.gz 2023-08-11T15:54:18.483 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94423.log 2023-08-11T15:54:18.483 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100247.log 2023-08-11T15:54:18.483 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103394.log.gz 2023-08-11T15:54:18.483 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94423.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94423.log.gz 2023-08-11T15:54:18.483 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96013.log 2023-08-11T15:54:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96891.log 2023-08-11T15:54:18.484 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100247.log.gz 2023-08-11T15:54:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96013.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96013.log.gz 2023-08-11T15:54:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105174.log 2023-08-11T15:54:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107837.log 2023-08-11T15:54:18.485 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96891.log.gz 2023-08-11T15:54:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105174.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105174.log.gz 2023-08-11T15:54:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103520.log 2023-08-11T15:54:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99953.log 2023-08-11T15:54:18.485 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107837.log.gz 2023-08-11T15:54:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103520.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103520.log.gz 2023-08-11T15:54:18.488 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107244.log 2023-08-11T15:54:18.488 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.99953.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-08-11T15:54:18.488 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99953.log.gz 2023-08-11T15:54:18.488 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107244.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107244.log.gz 2023-08-11T15:54:18.489 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102344.log 2023-08-11T15:54:18.489 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105646.log 2023-08-11T15:54:18.489 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94213.log 2023-08-11T15:54:18.489 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102344.log.gz 2023-08-11T15:54:18.489 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101846.log 2023-08-11T15:54:18.490 INFO:teuthology.orchestra.run.smithi027.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.105646.log.gz 2023-08-11T15:54:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94213.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94213.log.gz 2023-08-11T15:54:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96511.log 2023-08-11T15:54:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94999.log 2023-08-11T15:54:18.492 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101846.log.gz 2023-08-11T15:54:18.493 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph.log: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-08-11T15:54:18.493 INFO:teuthology.orchestra.run.smithi119.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-08-11T15:54:18.493 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94933.log 2023-08-11T15:54:18.494 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96511.log.gz 2023-08-11T15:54:18.494 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97223.log 2023-08-11T15:54:18.495 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94999.log.gz 2023-08-11T15:54:18.495 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102854.log 2023-08-11T15:54:18.495 INFO:teuthology.orchestra.run.smithi027.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.94933.log.gz 2023-08-11T15:54:18.496 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102218.log 2023-08-11T15:54:18.497 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97223.log.gz 2023-08-11T15:54:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102854.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102854.log.gz 2023-08-11T15:54:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-08-11T15:54:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102670.log 2023-08-11T15:54:18.499 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102218.log.gz 2023-08-11T15:54:18.500 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94681.log 2023-08-11T15:54:18.501 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102670.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102670.log.gz 2023-08-11T15:54:18.502 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107921.log 2023-08-11T15:54:18.503 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107328.log 2023-08-11T15:54:18.504 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94681.log.gz 2023-08-11T15:54:18.504 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107921.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107921.log.gz 2023-08-11T15:54:18.505 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107160.log 2023-08-11T15:54:18.506 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107328.log: gzip 88.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.107328.log.gz --verbose 2023-08-11T15:54:18.507 INFO:teuthology.orchestra.run.smithi027.stderr: -- /var/log/ceph/ceph-client.admin.107711.log 2023-08-11T15:54:18.508 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107160.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107160.log.gz 2023-08-11T15:54:18.508 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96884.log 2023-08-11T15:54:18.509 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107711.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107711.log.gz 2023-08-11T15:54:18.510 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106335.log 2023-08-11T15:54:18.511 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101966.log 2023-08-11T15:54:18.512 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96884.log.gz 2023-08-11T15:54:18.512 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106335.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106335.log.gz 2023-08-11T15:54:18.513 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104792.log 2023-08-11T15:54:18.514 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101966.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101966.log.gz 2023-08-11T15:54:18.516 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94891.log 2023-08-11T15:54:18.517 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96553.log 2023-08-11T15:54:18.518 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104792.log.gz 2023-08-11T15:54:18.519 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94891.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94891.log.gz 2023-08-11T15:54:18.519 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101087.log 2023-08-11T15:54:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96553.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96553.log.gz 2023-08-11T15:54:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105044.log 2023-08-11T15:54:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103562.log 2023-08-11T15:54:18.523 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101087.log.gz 2023-08-11T15:54:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105044.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105044.log.gz 2023-08-11T15:54:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100793.log 2023-08-11T15:54:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103562.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103562.log.gz 2023-08-11T15:54:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96301.log 2023-08-11T15:54:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105520.log 2023-08-11T15:54:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96301.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100793.log.gz 2023-08-11T15:54:18.531 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96301.log.gz 2023-08-11T15:54:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104750.log 2023-08-11T15:54:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105520.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105520.log.gz 2023-08-11T15:54:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97525.log 2023-08-11T15:54:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100205.log 2023-08-11T15:54:18.535 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104750.log.gz 2023-08-11T15:54:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97525.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97525.log.gz 2023-08-11T15:54:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96469.log 2023-08-11T15:54:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100205.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100205.log.gz 2023-08-11T15:54:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105352.log 2023-08-11T15:54:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93769.log 2023-08-11T15:54:18.537 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96469.log.gz 2023-08-11T15:54:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105352.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105352.log.gz 2023-08-11T15:54:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96223.log 2023-08-11T15:54:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.93769.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93769.log.gz 2023-08-11T15:54:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106866.log 2023-08-11T15:54:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101678.log 2023-08-11T15:54:18.540 INFO:teuthology.orchestra.run.smithi119.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-08-11T15:54:18.541 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96223.log.gz 2023-08-11T15:54:18.541 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106866.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106866.log.gz 2023-08-11T15:54:18.542 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95685.log 2023-08-11T15:54:18.543 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101678.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101678.log.gz 2023-08-11T15:54:18.543 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96595.log 2023-08-11T15:54:18.544 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.95685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.99612.log 2023-08-11T15:54:18.545 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95685.log.gz 2023-08-11T15:54:18.546 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96595.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96595.log.gz 2023-08-11T15:54:18.546 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101476.log 2023-08-11T15:54:18.547 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.2.99612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99869.log 2023-08-11T15:54:18.548 INFO:teuthology.orchestra.run.smithi027.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.2.99612.log.gz 2023-08-11T15:54:18.550 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99759.log 2023-08-11T15:54:18.550 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101476.log.gz 2023-08-11T15:54:18.550 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.99869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100877.log 2023-08-11T15:54:18.551 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99869.log.gz 2023-08-11T15:54:18.551 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.99759.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99759.log.gz 2023-08-11T15:54:18.552 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102050.log 2023-08-11T15:54:18.552 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100877.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100877.log.gz 2023-08-11T15:54:18.553 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106950.log 2023-08-11T15:54:18.554 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101392.log 2023-08-11T15:54:18.555 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102050.log.gz 2023-08-11T15:54:18.556 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106950.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106950.log.gz 2023-08-11T15:54:18.557 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102176.log 2023-08-11T15:54:18.557 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101392.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101392.log.gz 2023-08-11T15:54:18.557 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106419.log 2023-08-11T15:54:18.558 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108005.log 2023-08-11T15:54:18.559 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102176.log.gz 2023-08-11T15:54:18.560 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106419.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106419.log.gz 2023-08-11T15:54:18.560 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103478.log 2023-08-11T15:54:18.561 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108005.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108005.log.gz 2023-08-11T15:54:18.562 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-08-11T15:54:18.563 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103478.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103478.log.gz 2023-08-11T15:54:18.563 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100667.log 2023-08-11T15:54:18.565 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108401.log 2023-08-11T15:54:18.566 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100667.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100667.log.gz -5 2023-08-11T15:54:18.567 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97981.log 2023-08-11T15:54:18.568 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94553.log 2023-08-11T15:54:18.568 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108401.log.gz 2023-08-11T15:54:18.569 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97981.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.97981.log.gz 2023-08-11T15:54:18.570 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104024.log 2023-08-11T15:54:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100709.log 2023-08-11T15:54:18.571 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94553.log.gz 2023-08-11T15:54:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104024.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104024.log.gz 2023-08-11T15:54:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94723.log 2023-08-11T15:54:18.577 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96637.log 2023-08-11T15:54:18.578 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100709.log.gz 2023-08-11T15:54:18.579 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94723.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94723.log.gz 2023-08-11T15:54:18.579 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104108.log 2023-08-11T15:54:18.579 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96637.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96637.log.gz 2023-08-11T15:54:18.579 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96139.log 2023-08-11T15:54:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104108.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104108.log.gz 2023-08-11T15:54:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89269.log 2023-08-11T15:54:18.581 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96139.log: gzip 88.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.96139.log.gz -- 2023-08-11T15:54:18.581 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.103940.log 2023-08-11T15:54:18.583 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.89269.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.89269.log.gz 2023-08-11T15:54:18.583 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96896.log 2023-08-11T15:54:18.585 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103940.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103940.log.gz 2023-08-11T15:54:18.585 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104294.log 2023-08-11T15:54:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96896.log: 90.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-08-11T15:54:18.587 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96896.log.gz 2023-08-11T15:54:18.587 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105712.log 2023-08-11T15:54:18.588 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104294.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104294.log.gz 2023-08-11T15:54:18.589 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95871.log 2023-08-11T15:54:18.589 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98527.log 2023-08-11T15:54:18.589 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105712.log.gz 2023-08-11T15:54:18.590 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.95871.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95871.log.gz 2023-08-11T15:54:18.591 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101536.log 2023-08-11T15:54:18.591 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98527.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98527.log.gz 2023-08-11T15:54:18.592 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103064.log 2023-08-11T15:54:18.593 INFO:teuthology.orchestra.run.smithi119.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-08-11T15:54:18.593 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101536.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101536.log.gzgzip 2023-08-11T15:54:18.594 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.105478.log 2023-08-11T15:54:18.595 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103064.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103064.log.gz 2023-08-11T15:54:18.595 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101924.log 2023-08-11T15:54:18.596 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105478.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105478.log.gz 2023-08-11T15:54:18.597 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103106.log 2023-08-11T15:54:18.598 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101924.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101924.log.gz 2023-08-11T15:54:18.598 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105224.log 2023-08-11T15:54:18.600 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103106.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103106.log.gz 2023-08-11T15:54:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98065.log 2023-08-11T15:54:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105224.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105224.log.gz 2023-08-11T15:54:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101350.log 2023-08-11T15:54:18.603 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98065.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98065.log.gz 2023-08-11T15:54:18.603 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101045.log 2023-08-11T15:54:18.605 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101350.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101350.log.gz 2023-08-11T15:54:18.605 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103730.log 2023-08-11T15:54:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101045.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101045.log.gz 2023-08-11T15:54:18.607 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97231.log 2023-08-11T15:54:18.608 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103730.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103730.log.gz 2023-08-11T15:54:18.608 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97566.log 2023-08-11T15:54:18.610 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97231.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97231.log.gz 2023-08-11T15:54:18.610 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95745.log 2023-08-11T15:54:18.611 INFO:teuthology.orchestra.run.smithi119.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-08-11T15:54:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93925.log 2023-08-11T15:54:18.612 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97566.log.gz 2023-08-11T15:54:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.95745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96902.log 2023-08-11T15:54:18.613 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95745.log.gz 2023-08-11T15:54:18.614 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.93925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100919.log 2023-08-11T15:54:18.615 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93925.log.gz 2023-08-11T15:54:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96902.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96902.log.gz 2023-08-11T15:54:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96721.log 2023-08-11T15:54:18.617 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100919.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100919.log.gz 2023-08-11T15:54:18.618 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102260.log 2023-08-11T15:54:18.619 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96721.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96721.log.gz 2023-08-11T15:54:18.620 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105562.log 2023-08-11T15:54:18.621 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102260.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102260.log.gz 2023-08-11T15:54:18.621 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98317.log 2023-08-11T15:54:18.622 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105562.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105562.log.gz 2023-08-11T15:54:18.623 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96097.log 2023-08-11T15:54:18.624 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100751.log 2023-08-11T15:54:18.625 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98317.log.gz 2023-08-11T15:54:18.625 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96097.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96097.log.gz 2023-08-11T15:54:18.626 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88901.log 2023-08-11T15:54:18.627 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100751.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100751.log.gz 2023-08-11T15:54:18.627 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98359.log 2023-08-11T15:54:18.629 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.88901.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88901.log.gz 2023-08-11T15:54:18.629 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107515.log 2023-08-11T15:54:18.630 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103184.log 2023-08-11T15:54:18.631 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98359.log.gz 2023-08-11T15:54:18.632 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107515.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107515.log.gz 2023-08-11T15:54:18.633 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98443.log 2023-08-11T15:54:18.634 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103184.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103184.log.gz 2023-08-11T15:54:18.634 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94807.log 2023-08-11T15:54:18.635 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98443.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98443.log.gz 2023-08-11T15:54:18.635 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103436.log 2023-08-11T15:54:18.636 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100415.log 2023-08-11T15:54:18.636 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94807.log.gz 2023-08-11T15:54:18.637 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103436.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103436.log.gz 2023-08-11T15:54:18.638 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97567.log 2023-08-11T15:54:18.639 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100415.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100415.log.gz 2023-08-11T15:54:18.639 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-08-11T15:54:18.640 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97567.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97567.log.gz 2023-08-11T15:54:18.641 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105086.log 2023-08-11T15:54:18.642 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93967.log 2023-08-11T15:54:18.643 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105086.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105086.log.gz 2023-08-11T15:54:18.644 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96385.log 2023-08-11T15:54:18.644 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-08-11T15:54:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.93967.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.93967.log.gz --verbose 2023-08-11T15:54:18.645 INFO:teuthology.orchestra.run.smithi027.stderr: -- /var/log/ceph/ceph-client.admin.106992.log 2023-08-11T15:54:18.646 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96385.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96385.log.gz 2023-08-11T15:54:18.647 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102754.log 2023-08-11T15:54:18.648 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104066.log 2023-08-11T15:54:18.648 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106992.log.gz 2023-08-11T15:54:18.649 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102754.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102754.log.gz 2023-08-11T15:54:18.650 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102586.log 2023-08-11T15:54:18.651 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104066.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104066.log.gz -5 2023-08-11T15:54:18.651 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.100625.log 2023-08-11T15:54:18.652 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102586.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102586.log.gz 2023-08-11T15:54:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108173.log 2023-08-11T15:54:18.654 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96901.log 2023-08-11T15:54:18.654 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100625.log.gz 2023-08-11T15:54:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108173.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108173.log.gz 2023-08-11T15:54:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95913.log 2023-08-11T15:54:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96901.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96901.log.gz 2023-08-11T15:54:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102302.log 2023-08-11T15:54:18.658 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.95913.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95913.log.gz 2023-08-11T15:54:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108131.log 2023-08-11T15:54:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103268.log 2023-08-11T15:54:18.660 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102302.log.gz 2023-08-11T15:54:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108131.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.107034.log 2023-08-11T15:54:18.662 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108131.log.gz 2023-08-11T15:54:18.663 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103268.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103268.log.gz 2023-08-11T15:54:18.663 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96679.log 2023-08-11T15:54:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107034.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107034.log.gz 2023-08-11T15:54:18.665 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106581.log 2023-08-11T15:54:18.666 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101594.log 2023-08-11T15:54:18.666 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96679.log.gz 2023-08-11T15:54:18.667 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106581.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106581.log.gz 2023-08-11T15:54:18.668 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97771.log 2023-08-11T15:54:18.669 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101594.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101594.log.gz 2023-08-11T15:54:18.669 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-08-11T15:54:18.670 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97771.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97771.log.gz 2023-08-11T15:54:18.671 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102134.log 2023-08-11T15:54:18.672 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94849.log 2023-08-11T15:54:18.673 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102134.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102134.log.gz 2023-08-11T15:54:18.674 INFO:teuthology.orchestra.run.smithi027.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-08-11T15:54:18.674 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106908.log 2023-08-11T15:54:18.675 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94849.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94849.log.gz 2023-08-11T15:54:18.675 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102386.log 2023-08-11T15:54:18.676 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106908.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106908.log.gz 2023-08-11T15:54:18.677 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106791.log 2023-08-11T15:54:18.678 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108693.log 2023-08-11T15:54:18.678 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102386.log.gz 2023-08-11T15:54:18.679 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106791.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106791.log.gz 2023-08-11T15:54:18.680 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107431.log 2023-08-11T15:54:18.681 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108693.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108693.log.gz 2023-08-11T15:54:18.681 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101636.log 2023-08-11T15:54:18.682 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97484.log 2023-08-11T15:54:18.683 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107431.log.gz 2023-08-11T15:54:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101636.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101636.log.gz 2023-08-11T15:54:18.685 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94339.log 2023-08-11T15:54:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97484.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97484.log.gz 2023-08-11T15:54:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108485.log 2023-08-11T15:54:18.687 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94339.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94339.log.gz 2023-08-11T15:54:18.688 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100037.log 2023-08-11T15:54:18.689 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108485.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108485.log.gz 2023-08-11T15:54:18.690 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99205.log 2023-08-11T15:54:18.691 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100037.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100037.log.gz 2023-08-11T15:54:18.691 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106146.log 2023-08-11T15:54:18.692 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.99205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106461.log 2023-08-11T15:54:18.693 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99205.log.gz 2023-08-11T15:54:18.694 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106146.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106146.log.gz 2023-08-11T15:54:18.694 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97897.log 2023-08-11T15:54:18.695 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106461.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106461.log.gz 2023-08-11T15:54:18.695 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107753.log 2023-08-11T15:54:18.697 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103352.log 2023-08-11T15:54:18.697 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97897.log.gz 2023-08-11T15:54:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107753.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107753.log.gz 2023-08-11T15:54:18.699 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102008.log 2023-08-11T15:54:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103352.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103352.log.gz 2023-08-11T15:54:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104420.log 2023-08-11T15:54:18.701 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102008.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102008.log.gz 2023-08-11T15:54:18.702 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100961.log 2023-08-11T15:54:18.703 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107202.log 2023-08-11T15:54:18.703 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104420.log.gz 2023-08-11T15:54:18.704 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100961.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100961.log.gz 2023-08-11T15:54:18.705 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102796.log 2023-08-11T15:54:18.706 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107202.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107202.log.gz 2023-08-11T15:54:18.706 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97230.log 2023-08-11T15:54:18.707 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102796.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102796.log.gz 2023-08-11T15:54:18.707 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107557.log 2023-08-11T15:54:18.709 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5/var/log/ceph/ceph-client.admin.97230.log: --verbose -- /var/log/ceph/ceph-client.admin.107473.log 2023-08-11T15:54:18.709 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97230.log.gz 2023-08-11T15:54:18.710 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107557.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107557.log.gz 2023-08-11T15:54:18.710 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107118.log 2023-08-11T15:54:18.711 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100331.log 2023-08-11T15:54:18.712 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107118.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107473.log.gz 2023-08-11T15:54:18.714 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107118.log.gz 2023-08-11T15:54:18.714 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.82884.log 2023-08-11T15:54:18.715 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100331.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100331.log.gz 2023-08-11T15:54:18.715 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104562.log 2023-08-11T15:54:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.tmp-client.admin.82884.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.82884.log.gzgzip 2023-08-11T15:54:18.716 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.103772.log 2023-08-11T15:54:18.717 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104562.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104562.log.gz 2023-08-11T15:54:18.717 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106749.log 2023-08-11T15:54:18.718 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103772.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103772.log.gz 2023-08-11T15:54:18.718 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101003.log 2023-08-11T15:54:18.719 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106749.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106749.log.gz 2023-08-11T15:54:18.720 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97813.log 2023-08-11T15:54:18.721 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94170.log 2023-08-11T15:54:18.722 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101003.log.gz 2023-08-11T15:54:18.723 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97813.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97813.log.gz 2023-08-11T15:54:18.723 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95643.log 2023-08-11T15:54:18.724 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94170.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94170.log.gz 2023-08-11T15:54:18.725 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104624.log 2023-08-11T15:54:18.726 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.95643.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95643.log.gz 2023-08-11T15:54:18.727 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103604.log 2023-08-11T15:54:18.728 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104462.log 2023-08-11T15:54:18.729 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104624.log.gz 2023-08-11T15:54:18.729 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103604.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103604.log.gz 2023-08-11T15:54:18.730 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105310.log 2023-08-11T15:54:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104462.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104462.log.gz 2023-08-11T15:54:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108569.log 2023-08-11T15:54:18.732 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105310.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105310.log.gz 2023-08-11T15:54:18.733 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99717.log 2023-08-11T15:54:18.733 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108569.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108569.log.gz 2023-08-11T15:54:18.734 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94511.log 2023-08-11T15:54:18.734 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.99717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98485.log 2023-08-11T15:54:18.735 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99717.log.gz 2023-08-11T15:54:18.736 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94511.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94511.log.gz 2023-08-11T15:54:18.736 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100289.log 2023-08-11T15:54:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98485.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98485.log.gz 2023-08-11T15:54:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103646.log 2023-08-11T15:54:18.739 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100289.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100289.log.gz 2023-08-11T15:54:18.740 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104960.log 2023-08-11T15:54:18.741 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103646.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103646.log.gz 2023-08-11T15:54:18.741 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104336.log 2023-08-11T15:54:18.742 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104960.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104960.log.gz 2023-08-11T15:54:18.743 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102980.log 2023-08-11T15:54:18.744 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104336.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104336.log.gz 2023-08-11T15:54:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108611.log 2023-08-11T15:54:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102980.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102980.log.gz 2023-08-11T15:54:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96893.log 2023-08-11T15:54:18.747 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108611.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108611.log.gz 2023-08-11T15:54:18.748 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94467.log 2023-08-11T15:54:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96893.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96893.log.gz 2023-08-11T15:54:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98149.log 2023-08-11T15:54:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94467.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94467.log.gz 2023-08-11T15:54:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100541.log 2023-08-11T15:54:18.752 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98149.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98149.log.gz 2023-08-11T15:54:18.753 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96181.log 2023-08-11T15:54:18.754 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100541.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100541.log.gz 2023-08-11T15:54:18.754 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97855.log 2023-08-11T15:54:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96181.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96181.log.gz 2023-08-11T15:54:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106503.log 2023-08-11T15:54:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97855.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.97855.log.gz 2023-08-11T15:54:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103310.log 2023-08-11T15:54:18.759 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106503.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106503.log.gz 2023-08-11T15:54:18.760 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108317.log 2023-08-11T15:54:18.761 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100499.log 2023-08-11T15:54:18.761 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103310.log.gz 2023-08-11T15:54:18.762 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108257.log 2023-08-11T15:54:18.763 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108317.log.gz 2023-08-11T15:54:18.764 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100499.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100499.log.gz 2023-08-11T15:54:18.765 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106623.log 2023-08-11T15:54:18.766 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108257.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108257.log.gzgzip 2023-08-11T15:54:18.766 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.102628.log 2023-08-11T15:54:18.767 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106623.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106623.log.gz 2023-08-11T15:54:18.768 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100835.log 2023-08-11T15:54:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102628.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102628.log.gz 2023-08-11T15:54:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100583.log 2023-08-11T15:54:18.771 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100835.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100835.log.gz 2023-08-11T15:54:18.771 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98275.log 2023-08-11T15:54:18.772 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99100.log 2023-08-11T15:54:18.772 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100583.log.gz 2023-08-11T15:54:18.774 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98275.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98275.log.gz 2023-08-11T15:54:18.774 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103814.log 2023-08-11T15:54:18.775 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.99100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97235.log 2023-08-11T15:54:18.776 INFO:teuthology.orchestra.run.smithi027.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.0.99100.log.gz 2023-08-11T15:54:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103814.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103814.log.gz 2023-08-11T15:54:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108359.log 2023-08-11T15:54:18.778 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101762.log 2023-08-11T15:54:18.779 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97235.log.gz 2023-08-11T15:54:18.780 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108359.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108359.log.gz 2023-08-11T15:54:18.780 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95971.log 2023-08-11T15:54:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107963.log 2023-08-11T15:54:18.782 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101762.log.gz 2023-08-11T15:54:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.95971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99995.log 2023-08-11T15:54:18.783 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95971.log.gz 2023-08-11T15:54:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107963.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.107963.log.gz 2023-08-11T15:54:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96903.log 2023-08-11T15:54:18.786 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.99995.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99995.log.gz 2023-08-11T15:54:18.787 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103982.log 2023-08-11T15:54:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96903.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96903.log.gz 2023-08-11T15:54:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101308.log 2023-08-11T15:54:18.790 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103982.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103982.log.gz 2023-08-11T15:54:18.790 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104708.log 2023-08-11T15:54:18.791 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101308.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101308.log.gz 2023-08-11T15:54:18.791 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100079.log 2023-08-11T15:54:18.793 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104708.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104708.log.gz 2023-08-11T15:54:18.793 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94639.log 2023-08-11T15:54:18.794 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100079.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100079.log.gz 2023-08-11T15:54:18.795 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94381.log 2023-08-11T15:54:18.796 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94639.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94639.log.gz 2023-08-11T15:54:18.796 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105266.log 2023-08-11T15:54:18.797 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94381.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94381.log.gz 2023-08-11T15:54:18.798 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100163.log 2023-08-11T15:54:18.799 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97221.log 2023-08-11T15:54:18.799 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105266.log.gz 2023-08-11T15:54:18.800 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100163.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100163.log.gz 2023-08-11T15:54:18.801 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93869.log 2023-08-11T15:54:18.802 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97221.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97221.log.gz 2023-08-11T15:54:18.802 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107286.log 2023-08-11T15:54:18.804 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.93869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97234.log 2023-08-11T15:54:18.804 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93869.log.gz 2023-08-11T15:54:18.805 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107286.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107286.log.gz 2023-08-11T15:54:18.805 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97527.log 2023-08-11T15:54:18.807 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97234.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97234.log.gz 2023-08-11T15:54:18.807 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105436.log 2023-08-11T15:54:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97527.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97527.log.gz 2023-08-11T15:54:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97239.log 2023-08-11T15:54:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105436.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105436.log.gz 2023-08-11T15:54:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101720.log 2023-08-11T15:54:18.811 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97239.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97239.log.gz 2023-08-11T15:54:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106006.log 2023-08-11T15:54:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101720.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101720.log.gz 2023-08-11T15:54:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-08-11T15:54:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.106006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100121.log 2023-08-11T15:54:18.817 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94128.log 2023-08-11T15:54:18.818 INFO:teuthology.orchestra.run.smithi027.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.106006.log.gz 2023-08-11T15:54:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102938.log 2023-08-11T15:54:18.819 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100121.log.gz 2023-08-11T15:54:18.821 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104200.log 2023-08-11T15:54:18.821 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94128.log.gz 2023-08-11T15:54:18.822 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102938.log: 93.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-08-11T15:54:18.822 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102938.log.gz 2023-08-11T15:54:18.823 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-08-11T15:54:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104200.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104200.log.gz 2023-08-11T15:54:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102092.log 2023-08-11T15:54:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106377.log 2023-08-11T15:54:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96427.log 2023-08-11T15:54:18.828 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102092.log.gz 2023-08-11T15:54:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106377.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-08-11T15:54:18.829 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106377.log.gz 2023-08-11T15:54:18.833 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96427.log: gzip -5 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96427.log.gz 2023-08-11T15:54:18.834 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102544.log 2023-08-11T15:54:18.834 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99911.log 2023-08-11T15:54:18.835 INFO:teuthology.orchestra.run.smithi027.stderr: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-08-11T15:54:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102544.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102544.log.gz 2023-08-11T15:54:18.836 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105002.log 2023-08-11T15:54:18.840 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.99911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101129.log 2023-08-11T15:54:18.840 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99911.log.gz 2023-08-11T15:54:18.841 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104252.log 2023-08-11T15:54:18.841 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105002.log.gz 2023-08-11T15:54:18.841 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107879.log 2023-08-11T15:54:18.841 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101129.log.gz 2023-08-11T15:54:18.841 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104252.log: gzip -5 --verbose 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104252.log.gz 2023-08-11T15:54:18.842 INFO:teuthology.orchestra.run.smithi027.stderr: -- /var/log/ceph/ceph-client.admin.104918.log 2023-08-11T15:54:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107879.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107879.log.gz 2023-08-11T15:54:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98569.log 2023-08-11T15:54:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100457.log 2023-08-11T15:54:18.843 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104918.log.gz 2023-08-11T15:54:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103688.log 2023-08-11T15:54:18.845 INFO:teuthology.orchestra.run.smithi027.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.98569.log.gz 2023-08-11T15:54:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100457.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100457.log.gz 2023-08-11T15:54:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101804.log 2023-08-11T15:54:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103688.log: gzip -5 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103688.log.gz 2023-08-11T15:54:18.847 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.107076.log 2023-08-11T15:54:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101804.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101804.log.gz 2023-08-11T15:54:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94595.log 2023-08-11T15:54:18.851 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107076.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107076.log.gz 2023-08-11T15:54:18.851 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.106293.log 2023-08-11T15:54:18.853 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94595.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94595.log.gz 2023-08-11T15:54:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108443.log 2023-08-11T15:54:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106293.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106293.log.gz 2023-08-11T15:54:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96889.log 2023-08-11T15:54:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108443.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108443.log.gz 2023-08-11T15:54:18.859 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100373.log 2023-08-11T15:54:18.859 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96889.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96889.log.gz 2023-08-11T15:54:18.859 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107795.log 2023-08-11T15:54:18.860 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.100373.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100373.log.gz 2023-08-11T15:54:18.861 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96055.log 2023-08-11T15:54:18.869 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103856.log 2023-08-11T15:54:18.869 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107795.log.gz 2023-08-11T15:54:18.870 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.96055.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96055.log.gz 2023-08-11T15:54:18.870 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107388.log 2023-08-11T15:54:18.870 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103856.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103856.log.gz 2023-08-11T15:54:18.871 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-08-11T15:54:18.871 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.107388.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107388.log.gz 2023-08-11T15:54:18.871 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102470.log 2023-08-11T15:54:18.871 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101171.log 2023-08-11T15:54:18.872 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102470.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102470.log.gz 2023-08-11T15:54:18.873 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97584.log 2023-08-11T15:54:18.873 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101171.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108215.log 2023-08-11T15:54:18.873 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101171.log.gz 2023-08-11T15:54:18.874 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98107.log 2023-08-11T15:54:18.874 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97584.log.gz 2023-08-11T15:54:18.875 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98401.log 2023-08-11T15:54:18.876 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108215.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108215.log.gz/var/log/ceph/ceph-client.admin.98107.log: 2023-08-11T15:54:18.877 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105394.log 2023-08-11T15:54:18.877 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98107.log.gz 2023-08-11T15:54:18.878 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98697.log 2023-08-11T15:54:18.880 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98401.log: /var/log/ceph/ceph-client.admin.105394.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103226.log 2023-08-11T15:54:18.880 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105394.log.gz 2023-08-11T15:54:18.880 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98401.log.gz 2023-08-11T15:54:18.881 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98697.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98697.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106707.log 2023-08-11T15:54:18.882 INFO:teuthology.orchestra.run.smithi027.stderr: 2023-08-11T15:54:18.883 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98233.log 2023-08-11T15:54:18.883 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103226.log.gz 2023-08-11T15:54:18.885 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106707.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104876.log 2023-08-11T15:54:18.885 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106707.log.gz 2023-08-11T15:54:18.886 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98191.log 2023-08-11T15:54:18.886 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98233.log.gz 2023-08-11T15:54:18.888 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108651.log 2023-08-11T15:54:18.888 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104876.log.gz 2023-08-11T15:54:18.889 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.98191.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98191.log.gz 2023-08-11T15:54:18.890 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97594.log 2023-08-11T15:54:18.891 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108047.log 2023-08-11T15:54:18.891 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108651.log.gz 2023-08-11T15:54:18.893 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97939.log 2023-08-11T15:54:18.894 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97594.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97594.log.gz 2023-08-11T15:54:18.894 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98023.log 2023-08-11T15:54:18.895 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108047.log.gz 2023-08-11T15:54:18.895 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105130.log 2023-08-11T15:54:18.897 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97939.log: /var/log/ceph/ceph-client.admin.98023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97238.log 2023-08-11T15:54:18.898 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98023.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97939.log.gz 2023-08-11T15:54:18.898 INFO:teuthology.orchestra.run.smithi027.stderr: 2023-08-11T15:54:18.899 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.105130.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105130.log.gz 2023-08-11T15:54:18.899 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108089.log 2023-08-11T15:54:18.900 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104834.log 2023-08-11T15:54:18.902 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97238.log: /var/log/ceph/ceph-client.admin.108089.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94765.log 2023-08-11T15:54:18.903 INFO:teuthology.orchestra.run.smithi027.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108089.log.gz 2023-08-11T15:54:18.903 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97238.log.gz 2023-08-11T15:54:18.904 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105604.log 2023-08-11T15:54:18.904 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104834.log.gz 2023-08-11T15:54:18.905 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94047.log 2023-08-11T15:54:18.907 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94765.log: /var/log/ceph/ceph-client.admin.105604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97529.log 2023-08-11T15:54:18.908 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105604.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.94765.log.gz 2023-08-11T15:54:18.908 INFO:teuthology.orchestra.run.smithi027.stderr: 2023-08-11T15:54:18.909 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94047.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94047.log.gz 2023-08-11T15:54:18.909 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94297.log 2023-08-11T15:54:18.910 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94255.log 2023-08-11T15:54:18.912 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.97529.log: /var/log/ceph/ceph-client.admin.94297.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103022.log 2023-08-11T15:54:18.913 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94297.log.gz 2023-08-11T15:54:18.913 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97529.log.gz 2023-08-11T15:54:18.914 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.94255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95419.log 2023-08-11T15:54:18.914 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94255.log.gz 2023-08-11T15:54:18.915 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101266.log 2023-08-11T15:54:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.103022.log: /var/log/ceph/ceph-client.admin.95419.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102896.log 2023-08-11T15:54:18.917 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103022.log.gz 2023-08-11T15:54:18.918 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.101266.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101266.log.gz 2023-08-11T15:54:18.919 INFO:teuthology.orchestra.run.smithi027.stderr: 92.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102712.log 2023-08-11T15:54:18.920 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95419.log.gz 2023-08-11T15:54:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104378.log 2023-08-11T15:54:18.922 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.102896.log: /var/log/ceph/ceph-client.admin.102712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108527.log 2023-08-11T15:54:18.922 INFO:teuthology.orchestra.run.smithi027.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.102712.log.gz 2023-08-11T15:54:18.923 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102896.log.gz 2023-08-11T15:54:18.924 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.104378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106665.log 2023-08-11T15:54:18.924 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104378.log.gz 2023-08-11T15:54:18.925 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.108527.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108527.log.gz 2023-08-11T15:54:18.927 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.106665.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106665.log.gz 2023-08-11T15:54:19.198 INFO:teuthology.orchestra.run.smithi119.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-08-11T15:54:19.226 INFO:teuthology.orchestra.run.smithi119.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-08-11T15:54:19.235 INFO:teuthology.orchestra.run.smithi119.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-08-11T15:54:19.244 INFO:teuthology.orchestra.run.smithi027.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-08-11T15:54:19.250 INFO:teuthology.orchestra.run.smithi119.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-08-11T15:54:19.250 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-08-11T15:54:19.250 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m0.844s 2023-08-11T15:54:19.250 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m3.206s 2023-08-11T15:54:19.251 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m0.403s 2023-08-11T15:54:19.300 INFO:teuthology.orchestra.run.smithi027.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-08-11T15:54:19.341 INFO:teuthology.orchestra.run.smithi027.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-08-11T15:54:19.351 INFO:teuthology.orchestra.run.smithi027.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-08-11T15:54:19.601 INFO:teuthology.orchestra.run.smithi027.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-08-11T15:54:19.606 INFO:teuthology.orchestra.run.smithi027.stderr: 2023-08-11T15:54:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m1.197s 2023-08-11T15:54:19.607 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m3.595s 2023-08-11T15:54:19.607 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m1.247s 2023-08-11T15:54:19.608 INFO:tasks.ceph:Archiving logs... 2023-08-11T15:54:19.609 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066/remote/smithi027/log 2023-08-11T15:54:19.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-11T15:54:20.343 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066/remote/smithi119/log 2023-08-11T15:54:20.344 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-11T15:54:20.834 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-08-11T15:54:20.849 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-08-11T15:54:20.850 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-11T15:54:20.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-11T15:54:21.019 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-08-11T15:54:21.032 INFO:teuthology.task.clock:Checking final clock skew... 2023-08-11T15:54:21.032 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T15:54:21.042 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T15:54:21.095 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2023-08-11T15:54:21.099 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2023-08-11T15:54:21.106 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T15:54:21.106 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2023-08-11T15:54:21.106 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 +758us[ +755us] +/- 85ms 2023-08-11T15:54:21.107 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -1053us[-1057us] +/- 80ms 2023-08-11T15:54:21.107 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv03.front.sepia.ceph.com 3 6 377 8 +258us[ +254us] +/- 67ms 2023-08-11T15:54:21.107 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:54:21.108 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T15:54:21.109 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2023-08-11T15:54:21.109 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 7 +753us[ +753us] +/- 85ms 2023-08-11T15:54:21.109 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -1027us[-1027us] +/- 80ms 2023-08-11T15:54:21.109 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv03.front.sepia.ceph.com 3 6 377 11 +280us[ +284us] +/- 67ms 2023-08-11T15:54:21.110 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:54:21.110 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-08-11T15:54:21.122 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-08-11T15:54:21.123 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-08-11T15:54:21.133 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-11T15:54:21.437 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-11T15:54:21.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T15:54:21.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:54:21.819 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T15:54:21.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:54:21.929 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-08-11T15:54:21.939 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-08-11T15:54:21.949 INFO:teuthology.task.internal:Duration was 1356.755533 seconds 2023-08-11T15:54:21.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-08-11T15:54:21.967 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-08-11T15:54:21.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-11T15:54:21.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-11T15:54:22.153 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T15:54:22.165 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T15:54:22.494 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-08-11T15:54:22.495 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2023-08-11T15:54:22.496 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-11T15:54:22.572 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2023-08-11T15:54:22.573 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-11T15:54:22.656 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-08-11T15:54:22.657 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-11T15:54:22.667 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-11T15:54:22.778 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-08-11T15:54:22.779 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-11T15:54:22.785 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-11T15:54:23.553 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-08-11T15:54:23.563 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-08-11T15:54:23.564 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-11T15:54:23.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-11T15:54:23.771 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-08-11T15:54:23.782 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-11T15:54:23.791 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-11T15:54:23.902 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2023-08-11T15:54:23.910 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2023-08-11T15:54:23.975 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-11T15:54:24.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:54:24.032 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-11T15:54:24.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:54:24.088 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-08-11T15:54:24.102 INFO:teuthology.task.internal:Transferring archived files... 2023-08-11T15:54:24.103 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066/remote/smithi027 2023-08-11T15:54:24.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-11T15:54:24.235 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066/remote/smithi119 2023-08-11T15:54:24.235 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-11T15:54:24.352 INFO:teuthology.task.internal:Removing archive directory... 2023-08-11T15:54:24.353 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-11T15:54:24.363 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-11T15:54:24.419 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-08-11T15:54:24.432 INFO:teuthology.task.internal:Not uploading archives. 2023-08-11T15:54:24.432 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-08-11T15:54:24.448 INFO:teuthology.task.internal:Tidying up after the test... 2023-08-11T15:54:24.449 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-11T15:54:24.458 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-11T15:54:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: 263245 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 11 15:54 /home/ubuntu/cephtest 2023-08-11T15:54:24.516 INFO:teuthology.orchestra.run.smithi119.stdout: 263230 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 11 15:54 /home/ubuntu/cephtest 2023-08-11T15:54:24.519 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-08-11T15:54:24.529 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-08-11T15:54:24.561 INFO:teuthology.nuke:Checking targets against current locks 2023-08-11T15:54:24.592 DEBUG:teuthology.nuke:shortname: smithi027 2023-08-11T15:54:24.593 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T15:54:24.619 DEBUG:teuthology.nuke:shortname: smithi119 2023-08-11T15:54:24.620 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T15:54:24.623 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 15:21:08.929080', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIeYzfE3NPS3k8lesPvSilWbv9aC2F0ABFsxvm0DjICD7uFmKzpbjIT4K0oyiq4G8JVQbSTXUg2NPF/FJWCfmoc='} 2023-08-11T15:54:24.649 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366066', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 15:21:08.930423', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGd8+I5tYPlLin+mFo6DMZQ8wbSlDqoZJKOfAPw6zm6bEe2ff7EDCAnT3VVnHewnFpsguxwJp+ip/NUNaR9JwnI='} 2023-08-11T15:54:24.671 INFO:teuthology.orchestra.console.smithi027:Power off 2023-08-11T15:54:24.672 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T15:54:24.699 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off 2023-08-11T15:54:24.699 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T15:54:24.721 INFO:teuthology.orchestra.console.smithi119:Power off 2023-08-11T15:54:24.721 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T15:54:24.732 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2023-08-11T15:54:24.743 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2023-08-11T15:54:24.744 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T15:54:24.769 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-08-11T15:54:28.734 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T15:54:28.770 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T15:54:28.848 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2023-08-11T15:54:28.885 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-08-11T15:54:32.852 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T15:54:32.887 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T15:54:32.965 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2023-08-11T15:54:32.999 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-08-11T15:54:36.968 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T15:54:37.001 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T15:54:37.081 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2023-08-11T15:54:37.114 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2023-08-11T15:54:37.182 INFO:teuthology.orchestra.console.smithi027:Power off completed 2023-08-11T15:54:37.215 INFO:teuthology.orchestra.console.smithi119:Power off completed 2023-08-11T15:54:37.324 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} duration: 1356.755532503128 failure_reason: 'Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)' flavor: default mon.a-kernel-sha1: a7fb1265323db972dd333f71b9a53e9479f62e37 mon.b-kernel-sha1: a7fb1265323db972dd333f71b9a53e9479f62e37 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=7de88daded054cecbc1edfe153bc8472 status: fail success: false 2023-08-11T15:54:37.325 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T15:54:37.409 INFO:teuthology.run:FAIL